2023-07-21T21:48:03.616 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T21:48:03.617 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722 --name rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/orig.config.yaml 2023-07-21T21:48:03.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T21:48:08.780 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722 branch: wip-rishabh-2023Jul13-base-2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7347722' kernel: client: sha1: distro flavor: default kdb: true sha1: c685da6f56cc77f2cfb9c4221be3960522616caf ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-base-2 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-base-2 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-base-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi194.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 79491cb69dd37a1eafb93b3471998174bdfa2fd9 timestamp: 2023-07-21_21:30:53 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-21T21:48:08.780 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa; will attempt to use it 2023-07-21T21:48:08.781 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks 2023-07-21T21:48:08.781 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-21T21:48:08.783 INFO:teuthology.task.internal:Checking packages... 2023-07-21T21:48:08.811 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea' 2023-07-21T21:48:08.811 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-21T21:48:08.812 INFO:teuthology.packaging:ref: None 2023-07-21T21:48:08.812 INFO:teuthology.packaging:tag: None 2023-07-21T21:48:08.812 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-base-2 2023-07-21T21:48:08.812 INFO:teuthology.packaging:sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T21:48:08.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-base-2 2023-07-21T21:48:14.270 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4735.gbffe049d 2023-07-21T21:48:14.271 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-21T21:48:14.277 INFO:teuthology.task.internal:no buildpackages task found 2023-07-21T21:48:14.277 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-21T21:48:14.289 INFO:teuthology.task.internal:Saving configuration 2023-07-21T21:48:14.311 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-21T21:48:14.316 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T21:48:14.345 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.017564', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:48:14.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.019731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:48:14.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.021060', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:48:14.395 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-21T21:48:14.401 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-21T21:48:14.402 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-21T21:48:14.402 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-21T21:48:14.402 INFO:teuthology.run_tasks:Running task console_log... 2023-07-21T21:48:14.554 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fca8a9a35e0>, signals=[15]) 2023-07-21T21:48:14.555 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-21T21:48:14.562 INFO:teuthology.task.internal:Opening connections... 2023-07-21T21:48:14.562 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T21:48:14.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:48:14.634 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2023-07-21T21:48:14.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:48:14.702 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2023-07-21T21:48:14.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:48:14.771 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-21T21:48:14.778 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-07-21T21:48:14.797 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-07-21T21:48:14.798 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-07-21T21:48:14.856 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:48:14.856 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:48:14.857 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-07-21T21:48:14.857 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-07-21T21:48:14.857 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-07-21T21:48:14.857 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:48:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:48:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:48:14.859 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:48:14.859 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:48:14.859 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:48:14.859 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:48:14.860 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-07-21T21:48:14.889 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2023-07-21T21:48:14.908 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2023-07-21T21:48:14.908 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2023-07-21T21:48:14.965 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:48:14.966 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:48:14.966 INFO:teuthology.orchestra.run.smithi134.stdout:ID="rhel" 2023-07-21T21:48:14.966 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="fedora" 2023-07-21T21:48:14.966 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8.6" 2023-07-21T21:48:14.966 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:48:14.967 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:48:14.968 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:48:14.968 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:48:14.968 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:48:14.968 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2023-07-21T21:48:14.988 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2023-07-21T21:48:15.007 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2023-07-21T21:48:15.007 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2023-07-21T21:48:15.064 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:48:15.064 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:ID="rhel" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="fedora" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8.6" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2023-07-21T21:48:15.065 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:48:15.066 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:48:15.067 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:48:15.067 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2023-07-21T21:48:15.088 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-21T21:48:15.099 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-21T21:48:15.104 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-21T21:48:15.104 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:48:15.107 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:48:15.109 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:48:15.126 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-21T21:48:15.132 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-21T21:48:15.132 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:48:15.166 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:48:15.168 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:48:15.195 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-21T21:48:15.282 INFO:teuthology.run_tasks:Running task kernel... 2023-07-21T21:48:15.296 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'} 2023-07-21T21:48:15.298 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-07-21T21:48:15.298 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-07-21T21:48:15.299 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:48:15.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:48:15.319 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-07-21T21:48:15.379 INFO:teuthology.orchestra.run.smithi093.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T21:48:15.380 INFO:teuthology.task.kernel:Running kernel on smithi093: 4.18.0-372.9.1.el8.x86_64 2023-07-21T21:48:15.380 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum install -y kernel 2023-07-21T21:48:16.624 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:48:16.624 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-07-21T21:48:18.401 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 6.0 kB/s | 3.5 kB 00:00 2023-07-21T21:48:24.067 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 16 MB 00:05 2023-07-21T21:48:27.526 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 421 kB/s | 24 kB 00:00 2023-07-21T21:48:28.237 INFO:teuthology.orchestra.run.smithi093.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-21T21:48:28.295 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-21T21:48:28.296 INFO:teuthology.orchestra.run.smithi093.stdout:Nothing to do. 2023-07-21T21:48:28.296 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-21T21:48:28.426 DEBUG:teuthology.orchestra.run.smithi093:> echo no | sudo yum reinstall kernel || true 2023-07-21T21:48:28.753 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:48:28.753 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-07-21T21:48:29.019 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:01 ago on Fri 21 Jul 2023 09:48:27 PM UTC. 2023-07-21T21:48:29.060 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T21:48:29.097 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2023-07-21T21:48:29.142 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum reinstall -y kernel || true 2023-07-21T21:48:29.471 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:48:29.471 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-07-21T21:48:29.725 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:02 ago on Fri 21 Jul 2023 09:48:27 PM UTC. 2023-07-21T21:48:29.767 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T21:48:29.793 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2023-07-21T21:48:29.841 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q kernel | sort -rV | head -n 1 2023-07-21T21:48:29.922 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-21T21:48:29.922 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-21T21:48:29.922 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-21T21:48:29.923 DEBUG:teuthology.parallel:result is None 2023-07-21T21:48:29.923 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-21T21:48:29.930 INFO:teuthology.task.internal:Creating test directory... 2023-07-21T21:48:29.930 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:48:29.933 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:48:29.935 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:48:29.954 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-21T21:48:29.961 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-21T21:48:29.968 INFO:teuthology.task.internal:Creating archive directory... 2023-07-21T21:48:29.969 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:48:29.989 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:48:29.994 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:48:30.097 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-21T21:48:30.103 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-21T21:48:30.104 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:48:30.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:48:30.123 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:48:30.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:48:30.142 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:48:30.144 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:48:30.165 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:48:30.179 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.196 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.198 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.230 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.247 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.265 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:48:30.266 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-21T21:48:30.274 INFO:teuthology.task.internal:Configuring sudo... 2023-07-21T21:48:30.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:48:30.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:48:30.278 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:48:30.307 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-21T21:48:30.318 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-21T21:48:30.318 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:48:30.343 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:48:30.347 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:48:30.364 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:48:35.449 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:48:35.489 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:48:40.583 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:48:40.616 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T21:48:40.618 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:48:40.691 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:48:44.670 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:48:44.710 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:48:49.804 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:48:49.836 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T21:48:49.837 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:48:49.942 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:48:59.997 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:00.046 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:05.099 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:05.132 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T21:49:05.132 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:49:05.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2023-07-21T21:49:05.207 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2023-07-21T21:49:05.210 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2023-07-21T21:49:05.272 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:05.313 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:05.696 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-21T21:49:05.705 INFO:teuthology.task.internal:Starting timer... 2023-07-21T21:49:05.705 INFO:teuthology.run_tasks:Running task pcp... 2023-07-21T21:49:05.734 INFO:teuthology.run_tasks:Running task selinux... 2023-07-21T21:49:05.759 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-21T21:49:05.761 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2023-07-21T21:49:05.829 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2023-07-21T21:49:05.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2023-07-21T21:49:05.917 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2023-07-21T21:49:05.918 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2023-07-21T21:49:05.979 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2023-07-21T21:49:05.980 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-21T21:49:05.981 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2023-07-21T21:49:06.007 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2023-07-21T21:49:06.008 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2023-07-21T21:49:06.035 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2023-07-21T21:49:06.035 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2023-07-21T21:49:06.059 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2023-07-21T21:49:06.060 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi093.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2023-07-21T21:49:06.061 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:06.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:06.087 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:06.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:06.115 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:06.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:06.143 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-21T21:49:06.143 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:06.170 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:06.196 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:06.224 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-21T21:49:06.237 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-21T21:49:06.238 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-21T21:49:06.259 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-21T21:49:06.261 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi093.front.sepia.ceph.com,smithi134.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-21T21:59:12.588 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi093.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2023-07-21T21:59:12.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-07-21T21:59:12.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:17.665 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-07-21T21:59:17.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-07-21T21:59:17.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-07-21T21:59:17.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:17.807 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-07-21T21:59:17.875 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-07-21T21:59:17.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2023-07-21T21:59:17.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:17.945 DEBUG:teuthology.orchestra.run.smithi194:> true 2023-07-21T21:59:18.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2023-07-21T21:59:18.017 INFO:teuthology.run_tasks:Running task clock... 2023-07-21T21:59:18.027 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-21T21:59:18.027 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T21:59:18.028 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T21:59:18.030 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T21:59:18.030 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T21:59:18.032 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T21:59:18.033 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T21:59:18.066 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T21:59:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T21:59:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T21:59:18.084 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T21:59:18.091 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T21:59:18.106 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T21:59:18.155 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2023-07-21T21:59:18.168 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2023-07-21T21:59:18.170 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2023-07-21T21:59:18.177 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2023-07-21T21:59:18.182 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2023-07-21T21:59:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T21:59:18.189 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2023-07-21T21:59:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T21:59:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T21:59:18.203 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T21:59:18.211 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T21:59:18.218 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T21:59:18.273 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-07-21T21:59:18.276 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T21:59:18.276 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-07-21T21:59:18.277 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.277 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.277 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.277 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.287 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.306 INFO:teuthology.orchestra.run.smithi194.stderr:bash: ntpq: command not found 2023-07-21T21:59:18.309 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T21:59:18.309 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2023-07-21T21:59:18.309 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.309 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.309 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.310 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T21:59:18.310 INFO:teuthology.run_tasks:Running task pexec... 2023-07-21T21:59:18.324 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-21T21:59:18.325 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T21:59:18.325 DEBUG:teuthology.orchestra.run.smithi093:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T21:59:18.326 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi134.front.sepia.ceph.com 2023-07-21T21:59:18.326 DEBUG:teuthology.orchestra.run.smithi134:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T21:59:18.326 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi194.front.sepia.ceph.com 2023-07-21T21:59:18.327 DEBUG:teuthology.orchestra.run.smithi194:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T21:59:18.697 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:18.703 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:18.722 INFO:teuthology.orchestra.run.smithi194.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:19.549 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:17 ago on Fri 21 Jul 2023 09:57:02 PM UTC. 2023-07-21T21:59:19.560 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:31 ago on Fri 21 Jul 2023 09:57:48 PM UTC. 2023-07-21T21:59:19.576 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:08 ago on Fri 21 Jul 2023 09:57:11 PM UTC. 2023-07-21T21:59:21.255 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-21T21:59:21.256 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-21T21:59:21.256 INFO:teuthology.orchestra.run.smithi093.stdout: Package Architecture Version Repository Size 2023-07-21T21:59:21.256 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-21T21:59:21.256 INFO:teuthology.orchestra.run.smithi093.stdout:Resetting modules: 2023-07-21T21:59:21.256 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools 2023-07-21T21:59:21.257 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:21.257 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-21T21:59:21.257 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-07-21T21:59:21.257 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:21.296 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout:Resetting modules: 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools 2023-07-21T21:59:21.297 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:21.298 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-21T21:59:21.298 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-21T21:59:21.298 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:21.310 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2023-07-21T21:59:21.311 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2023-07-21T21:59:21.311 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2023-07-21T21:59:21.311 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2023-07-21T21:59:21.311 INFO:teuthology.orchestra.run.smithi194.stdout:Resetting modules: 2023-07-21T21:59:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools 2023-07-21T21:59:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:21.312 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2023-07-21T21:59:21.312 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2023-07-21T21:59:21.312 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:22.219 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:19 ago on Fri 21 Jul 2023 09:57:02 PM UTC. 2023-07-21T21:59:22.263 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:33 ago on Fri 21 Jul 2023 09:57:48 PM UTC. 2023-07-21T21:59:22.305 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:10 ago on Fri 21 Jul 2023 09:57:11 PM UTC. 2023-07-21T21:59:25.843 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-21T21:59:26.034 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-21T21:59:26.049 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2023-07-21T21:59:26.326 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:26.516 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:26.529 INFO:teuthology.orchestra.run.smithi194.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:27.175 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:24 ago on Fri 21 Jul 2023 09:57:02 PM UTC. 2023-07-21T21:59:27.362 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:38 ago on Fri 21 Jul 2023 09:57:48 PM UTC. 2023-07-21T21:59:27.380 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:15 ago on Fri 21 Jul 2023 09:57:11 PM UTC. 2023-07-21T21:59:29.037 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-21T21:59:29.072 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-21T21:59:29.120 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2023-07-21T21:59:29.128 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-21T21:59:29.128 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-07-21T21:59:29.128 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-21T21:59:29.128 INFO:teuthology.orchestra.run.smithi093.stdout:Installing group/module packages: 2023-07-21T21:59:29.128 INFO:teuthology.orchestra.run.smithi093.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T21:59:29.129 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout:Installing module profiles: 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools/common 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-07-21T21:59:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools rhel8 2023-07-21T21:59:29.131 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:29.131 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-21T21:59:29.131 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================================ 2023-07-21T21:59:29.131 INFO:teuthology.orchestra.run.smithi093.stdout:Install 10 Packages 2023-07-21T21:59:29.131 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:29.132 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 18 M 2023-07-21T21:59:29.132 INFO:teuthology.orchestra.run.smithi093.stdout:Installed size: 65 M 2023-07-21T21:59:29.132 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout:Installing group/module packages: 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T21:59:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-07-21T21:59:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout:Installing module profiles: 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools/common 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools rhel8 2023-07-21T21:59:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:29.170 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-21T21:59:29.170 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-21T21:59:29.170 INFO:teuthology.orchestra.run.smithi134.stdout:Install 10 Packages 2023-07-21T21:59:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:29.171 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 18 M 2023-07-21T21:59:29.171 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 65 M 2023-07-21T21:59:29.171 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-07-21T21:59:29.213 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================================ 2023-07-21T21:59:29.213 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2023-07-21T21:59:29.213 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================================ 2023-07-21T21:59:29.213 INFO:teuthology.orchestra.run.smithi194.stdout:Installing group/module packages: 2023-07-21T21:59:29.213 INFO:teuthology.orchestra.run.smithi194.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2023-07-21T21:59:29.214 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout:Installing module profiles: 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools/common 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout:Enabling module streams: 2023-07-21T21:59:29.215 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools rhel8 2023-07-21T21:59:29.216 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:29.216 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2023-07-21T21:59:29.216 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================================ 2023-07-21T21:59:29.216 INFO:teuthology.orchestra.run.smithi194.stdout:Install 10 Packages 2023-07-21T21:59:29.216 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:29.217 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 18 M 2023-07-21T21:59:29.217 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 65 M 2023-07-21T21:59:29.217 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2023-07-21T21:59:29.227 INFO:teuthology.orchestra.run.smithi093.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 974 kB/s | 94 kB 00:00 2023-07-21T21:59:29.252 INFO:teuthology.orchestra.run.smithi093.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 204 kB/s | 25 kB 00:00 2023-07-21T21:59:29.259 INFO:teuthology.orchestra.run.smithi134.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 276 kB/s | 25 kB 00:00 2023-07-21T21:59:29.277 INFO:teuthology.orchestra.run.smithi093.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 278 kB/s | 41 kB 00:00 2023-07-21T21:59:29.285 INFO:teuthology.orchestra.run.smithi134.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 354 kB/s | 41 kB 00:00 2023-07-21T21:59:29.300 INFO:teuthology.orchestra.run.smithi194.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 294 kB/s | 25 kB 00:00 2023-07-21T21:59:29.303 INFO:teuthology.orchestra.run.smithi093.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-07-21T21:59:29.310 INFO:teuthology.orchestra.run.smithi134.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 668 kB/s | 94 kB 00:00 2023-07-21T21:59:29.317 INFO:teuthology.orchestra.run.smithi194.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 402 kB/s | 41 kB 00:00 2023-07-21T21:59:29.335 INFO:teuthology.orchestra.run.smithi134.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-07-21T21:59:29.337 INFO:teuthology.orchestra.run.smithi093.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T21:59:29.342 INFO:teuthology.orchestra.run.smithi194.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 740 kB/s | 94 kB 00:00 2023-07-21T21:59:29.362 INFO:teuthology.orchestra.run.smithi093.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T21:59:29.367 INFO:teuthology.orchestra.run.smithi194.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 738 kB/s | 49 kB 00:00 2023-07-21T21:59:29.369 INFO:teuthology.orchestra.run.smithi134.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T21:59:29.394 INFO:teuthology.orchestra.run.smithi134.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T21:59:29.401 INFO:teuthology.orchestra.run.smithi194.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T21:59:29.426 INFO:teuthology.orchestra.run.smithi194.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T21:59:29.428 INFO:teuthology.orchestra.run.smithi093.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.3 MB/s | 213 kB 00:00 2023-07-21T21:59:29.451 INFO:teuthology.orchestra.run.smithi194.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-07-21T21:59:29.469 INFO:teuthology.orchestra.run.smithi134.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.1 MB/s | 213 kB 00:00 2023-07-21T21:59:29.554 INFO:teuthology.orchestra.run.smithi093.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 29 MB/s | 8.0 MB 00:00 2023-07-21T21:59:29.561 INFO:teuthology.orchestra.run.smithi134.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 32 MB/s | 8.0 MB 00:00 2023-07-21T21:59:29.577 INFO:teuthology.orchestra.run.smithi194.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-07-21T21:59:29.621 INFO:teuthology.orchestra.run.smithi093.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-21T21:59:29.628 INFO:teuthology.orchestra.run.smithi134.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 14 MB/s | 2.2 MB 00:00 2023-07-21T21:59:29.702 INFO:teuthology.orchestra.run.smithi194.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-07-21T21:59:29.718 INFO:teuthology.orchestra.run.smithi093.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-07-21T21:59:29.719 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-07-21T21:59:29.719 INFO:teuthology.orchestra.run.smithi093.stdout:Total 31 MB/s | 18 MB 00:00 2023-07-21T21:59:29.753 INFO:teuthology.orchestra.run.smithi134.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-07-21T21:59:29.754 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-07-21T21:59:29.755 INFO:teuthology.orchestra.run.smithi134.stdout:Total 31 MB/s | 18 MB 00:00 2023-07-21T21:59:29.794 INFO:teuthology.orchestra.run.smithi194.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 19 MB/s | 7.0 MB 00:00 2023-07-21T21:59:29.794 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2023-07-21T21:59:29.795 INFO:teuthology.orchestra.run.smithi194.stdout:Total 31 MB/s | 18 MB 00:00 2023-07-21T21:59:29.912 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-07-21T21:59:29.931 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-07-21T21:59:29.931 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-07-21T21:59:29.954 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-07-21T21:59:29.973 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-07-21T21:59:29.973 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-07-21T21:59:29.996 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2023-07-21T21:59:30.014 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2023-07-21T21:59:30.015 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2023-07-21T21:59:30.112 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-07-21T21:59:30.115 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-07-21T21:59:30.154 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-07-21T21:59:30.158 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-07-21T21:59:30.196 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2023-07-21T21:59:30.200 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2023-07-21T21:59:30.628 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-07-21T21:59:30.672 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-07-21T21:59:30.710 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2023-07-21T21:59:30.820 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T21:59:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T21:59:30.896 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T21:59:31.014 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:31.101 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:31.209 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T21:59:31.216 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T21:59:31.286 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T21:59:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T21:59:31.425 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T21:59:31.548 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T21:59:31.768 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T21:59:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T21:59:31.887 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T21:59:32.524 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T21:59:32.541 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T21:59:32.650 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T21:59:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T21:59:33.432 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T21:59:33.543 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T21:59:33.650 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T21:59:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T21:59:33.785 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T21:59:33.814 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T21:59:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T21:59:33.958 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T21:59:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.033 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.168 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.280 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.280 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T21:59:34.281 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T21:59:34.282 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T21:59:34.282 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T21:59:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T21:59:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T21:59:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T21:59:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T21:59:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T21:59:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T21:59:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T21:59:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T21:59:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T21:59:34.436 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T21:59:34.436 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T21:59:34.436 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T21:59:34.436 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T21:59:34.436 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T21:59:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T21:59:35.408 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:02:32 ago on Fri 21 Jul 2023 09:57:02 PM UTC. 2023-07-21T21:59:35.408 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T21:59:35.408 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:46 ago on Fri 21 Jul 2023 09:57:48 PM UTC. 2023-07-21T21:59:35.621 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T21:59:35.622 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:24 ago on Fri 21 Jul 2023 09:57:11 PM UTC. 2023-07-21T21:59:39.255 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-07-21T21:59:39.313 INFO:teuthology.orchestra.run.smithi194.stdout:Installed products updated. 2023-07-21T21:59:39.335 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.447 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T21:59:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T21:59:39.449 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-21T21:59:39.517 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:39.517 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2023-07-21T21:59:39.517 INFO:teuthology.orchestra.run.smithi194.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.517 INFO:teuthology.orchestra.run.smithi194.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.518 INFO:teuthology.orchestra.run.smithi194.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.519 INFO:teuthology.orchestra.run.smithi194.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T21:59:39.519 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T21:59:39.519 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2023-07-21T21:59:39.533 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:39.533 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-07-21T21:59:39.533 INFO:teuthology.orchestra.run.smithi093.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.534 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T21:59:39.535 INFO:teuthology.orchestra.run.smithi093.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T21:59:39.535 INFO:teuthology.orchestra.run.smithi093.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T21:59:39.535 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T21:59:39.535 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-21T21:59:39.608 DEBUG:teuthology.parallel:result is None 2023-07-21T21:59:39.666 DEBUG:teuthology.parallel:result is None 2023-07-21T21:59:39.678 DEBUG:teuthology.parallel:result is None 2023-07-21T21:59:39.679 INFO:teuthology.run_tasks:Running task install... 2023-07-21T21:59:39.687 DEBUG:teuthology.task.install:project ceph 2023-07-21T21:59:39.687 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'}} 2023-07-21T21:59:39.687 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'} 2023-07-21T21:59:39.688 INFO:teuthology.task.install:Using flavor: default 2023-07-21T21:59:39.693 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-21T21:59:39.694 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-21T21:59:39.694 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T21:59:39.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T21:59:39.697 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T21:59:39.697 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T21:59:39.698 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T21:59:39.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T21:59:40.710 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T21:59:40.710 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T21:59:40.717 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T21:59:40.718 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T21:59:40.721 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T21:59:40.721 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T21:59:41.163 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T21:59:41.164 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T21:59:41.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T21:59:41.166 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T21:59:41.167 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T21:59:41.167 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T21:59:41.172 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T21:59:41.173 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T21:59:41.173 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T21:59:41.197 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T21:59:41.197 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T21:59:41.199 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T21:59:41.199 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T21:59:41.206 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T21:59:41.206 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T21:59:41.269 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T21:59:41.270 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T21:59:41.275 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T21:59:41.349 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T21:59:41.351 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T21:59:41.356 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T21:59:41.416 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2023-07-21T21:59:41.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2023-07-21T21:59:41.420 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2023-07-21T21:59:41.422 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2023-07-21T21:59:41.424 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2023-07-21T21:59:41.426 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2023-07-21T21:59:41.793 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:41.797 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:41.802 INFO:teuthology.orchestra.run.smithi194.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:42.136 INFO:teuthology.orchestra.run.smithi134.stdout:44 files removed 2023-07-21T21:59:42.145 INFO:teuthology.orchestra.run.smithi194.stdout:44 files removed 2023-07-21T21:59:42.152 INFO:teuthology.orchestra.run.smithi093.stdout:44 files removed 2023-07-21T21:59:42.183 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T21:59:42.191 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T21:59:42.198 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T21:59:42.514 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:42.522 INFO:teuthology.orchestra.run.smithi194.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:42.530 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-07-21T21:59:44.095 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 68 kB/s | 81 kB 00:01 2023-07-21T21:59:44.096 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 68 kB/s | 81 kB 00:01 2023-07-21T21:59:44.097 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 69 kB/s | 81 kB 00:01 2023-07-21T21:59:45.122 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 9.5 kB/s | 9.4 kB 00:00 2023-07-21T21:59:45.137 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 9.3 kB/s | 9.4 kB 00:01 2023-07-21T21:59:45.140 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 9.3 kB/s | 9.4 kB 00:01 2023-07-21T21:59:46.114 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T21:59:46.149 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T21:59:46.154 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T21:59:46.798 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 5.2 kB/s | 3.5 kB 00:00 2023-07-21T21:59:46.854 INFO:teuthology.orchestra.run.smithi194.stdout:Copr repo for python3-asyncssh owned by ceph 5.1 kB/s | 3.5 kB 00:00 2023-07-21T21:59:47.097 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 3.7 kB/s | 3.5 kB 00:00 2023-07-21T21:59:50.418 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 16 MB 00:03 2023-07-21T21:59:51.229 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 16 MB 00:04 2023-07-21T21:59:54.017 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 209 kB/s | 24 kB 00:00 2023-07-21T21:59:54.713 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 502 kB/s | 24 kB 00:00 2023-07-21T21:59:54.885 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-07-21T21:59:55.581 INFO:teuthology.orchestra.run.smithi194.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-07-21T22:00:03.470 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 53 MB 00:01 2023-07-21T22:00:03.686 INFO:teuthology.orchestra.run.smithi194.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-21T22:00:14.773 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:00:14.774 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:00:14.788 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.789 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.791 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:00:14.792 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:00:14.794 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:00:14.798 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.799 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:00:14.835 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:00:14.836 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:00:14.852 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.853 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.854 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:00:14.855 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:00:14.858 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:00:14.861 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:00:14.862 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:00:14.933 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2023-07-21T22:00:14.945 INFO:teuthology.orchestra.run.smithi194.stdout:=========================================================================================================================== 2023-07-21T22:00:14.945 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout:=========================================================================================================================== 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:00:14.946 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:00:14.947 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:00:14.948 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:00:14.949 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:00:14.950 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:00:14.951 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:00:14.952 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:00:14.953 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:00:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:00:14.955 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:00:14.956 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:00:14.957 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:00:14.958 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:00:14.959 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:00:14.960 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:00:14.961 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:00:14.962 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:00:14.963 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout:Enabling module streams: 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql 10 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout:=========================================================================================================================== 2023-07-21T22:00:14.964 INFO:teuthology.orchestra.run.smithi194.stdout:Install 128 Packages 2023-07-21T22:00:14.965 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 3 Packages 2023-07-21T22:00:14.965 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:00:14.965 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 205 M 2023-07-21T22:00:14.966 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2023-07-21T22:00:14.993 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-07-21T22:00:15.006 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-21T22:00:15.006 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-07-21T22:00:15.006 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-21T22:00:15.007 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2023-07-21T22:00:15.007 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:00:15.007 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:00:15.008 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:00:15.008 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:00:15.008 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:00:15.008 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:00:15.009 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:00:15.009 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:00:15.009 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:00:15.010 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:00:15.010 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:00:15.010 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:00:15.010 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:00:15.011 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:00:15.011 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:00:15.011 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:00:15.013 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:00:15.013 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:00:15.013 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:00:15.014 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:00:15.015 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:00:15.016 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:00:15.017 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:00:15.018 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:00:15.019 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:00:15.020 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:00:15.021 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:00:15.022 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:00:15.023 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:00:15.023 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:00:15.023 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:00:15.023 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:00:15.024 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:00:15.025 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:00:15.026 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:00:15.027 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:00:15.028 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:00:15.029 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql 10 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-07-21T22:00:15.030 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-07-21T22:00:15.031 INFO:teuthology.orchestra.run.smithi093.stdout:Install 128 Packages 2023-07-21T22:00:15.031 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 3 Packages 2023-07-21T22:00:15.031 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:00:15.031 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 205 M 2023-07-21T22:00:15.032 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-07-21T22:00:15.789 INFO:teuthology.orchestra.run.smithi093.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2023-07-21T22:00:16.016 INFO:teuthology.orchestra.run.smithi194.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-07-21T22:00:16.439 INFO:teuthology.orchestra.run.smithi093.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 1.3 MB/s | 848 kB 00:00 2023-07-21T22:00:16.556 INFO:teuthology.orchestra.run.smithi093.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 1.4 MB/s | 165 kB 00:00 2023-07-21T22:00:16.657 INFO:teuthology.orchestra.run.smithi194.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 1.3 MB/s | 848 kB 00:00 2023-07-21T22:00:16.774 INFO:teuthology.orchestra.run.smithi194.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 1.4 MB/s | 165 kB 00:00 2023-07-21T22:00:17.231 INFO:teuthology.orchestra.run.smithi093.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 3.4 MB/s | 2.3 MB 00:00 2023-07-21T22:00:17.307 INFO:teuthology.orchestra.run.smithi093.stdout:(5/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-07-21T22:00:17.316 INFO:teuthology.orchestra.run.smithi194.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 4.2 MB/s | 2.3 MB 00:00 2023-07-21T22:00:17.455 INFO:teuthology.orchestra.run.smithi194.stdout:(5/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 2.9 MB/s | 5.4 MB 00:01 2023-07-21T22:00:17.725 INFO:teuthology.orchestra.run.smithi194.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 3.8 MB/s | 1.5 MB 00:00 2023-07-21T22:00:17.765 INFO:teuthology.orchestra.run.smithi093.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-07-21T22:00:18.217 INFO:teuthology.orchestra.run.smithi194.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 6.1 MB/s | 4.6 MB 00:00 2023-07-21T22:00:18.465 INFO:teuthology.orchestra.run.smithi093.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 4.0 MB/s | 4.6 MB 00:01 2023-07-21T22:00:20.059 INFO:teuthology.orchestra.run.smithi194.stdout:(8/131): ceph-radosgw-18.0.0-4735.gbffe049d.el8 7.3 MB/s | 13 MB 00:01 2023-07-21T22:00:20.150 INFO:teuthology.orchestra.run.smithi194.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 266 kB/s | 24 kB 00:00 2023-07-21T22:00:20.410 INFO:teuthology.orchestra.run.smithi194.stdout:(10/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 6.6 MB/s | 18 MB 00:02 2023-07-21T22:00:20.543 INFO:teuthology.orchestra.run.smithi194.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.9 MB/s | 258 kB 00:00 2023-07-21T22:00:20.660 INFO:teuthology.orchestra.run.smithi194.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 266 kB/s | 31 kB 00:00 2023-07-21T22:00:20.827 INFO:teuthology.orchestra.run.smithi194.stdout:(13/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 4.5 MB/s | 771 kB 00:00 2023-07-21T22:00:20.935 INFO:teuthology.orchestra.run.smithi194.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 1.2 MB/s | 137 kB 00:00 2023-07-21T22:00:21.036 INFO:teuthology.orchestra.run.smithi194.stdout:(15/131): librados-devel-18.0.0-4735.gbffe049d. 1.4 MB/s | 139 kB 00:00 2023-07-21T22:00:21.161 INFO:teuthology.orchestra.run.smithi194.stdout:(16/131): libradosstriper1-18.0.0-4735.gbffe049 3.7 MB/s | 473 kB 00:00 2023-07-21T22:00:21.582 INFO:teuthology.orchestra.run.smithi093.stdout:(8/131): ceph-common-18.0.0-4735.gbffe049d.el8. 3.5 MB/s | 22 MB 00:06 2023-07-21T22:00:21.682 INFO:teuthology.orchestra.run.smithi093.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 244 kB/s | 24 kB 00:00 2023-07-21T22:00:22.361 INFO:teuthology.orchestra.run.smithi194.stdout:(17/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 5.6 MB/s | 6.7 MB 00:01 2023-07-21T22:00:22.486 INFO:teuthology.orchestra.run.smithi194.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 358 kB/s | 45 kB 00:00 2023-07-21T22:00:22.491 INFO:teuthology.orchestra.run.smithi093.stdout:(10/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 3.4 MB/s | 13 MB 00:04 2023-07-21T22:00:22.616 INFO:teuthology.orchestra.run.smithi093.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 2.0 MB/s | 258 kB 00:00 2023-07-21T22:00:22.620 INFO:teuthology.orchestra.run.smithi194.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 891 kB/s | 119 kB 00:00 2023-07-21T22:00:22.750 INFO:teuthology.orchestra.run.smithi093.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 232 kB/s | 31 kB 00:00 2023-07-21T22:00:22.778 INFO:teuthology.orchestra.run.smithi194.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.2 MB/s | 197 kB 00:00 2023-07-21T22:00:22.921 INFO:teuthology.orchestra.run.smithi194.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.5 MB/s | 370 kB 00:00 2023-07-21T22:00:23.046 INFO:teuthology.orchestra.run.smithi194.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.8 MB/s | 364 kB 00:00 2023-07-21T22:00:23.066 INFO:teuthology.orchestra.run.smithi093.stdout:(13/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 2.4 MB/s | 771 kB 00:00 2023-07-21T22:00:23.191 INFO:teuthology.orchestra.run.smithi093.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 1.1 MB/s | 137 kB 00:00 2023-07-21T22:00:23.305 INFO:teuthology.orchestra.run.smithi194.stdout:(23/131): ceph-common-18.0.0-4735.gbffe049d.el8 2.9 MB/s | 22 MB 00:07 2023-07-21T22:00:23.308 INFO:teuthology.orchestra.run.smithi093.stdout:(15/131): librados-devel-18.0.0-4735.gbffe049d. 1.2 MB/s | 139 kB 00:00 2023-07-21T22:00:23.329 INFO:teuthology.orchestra.run.smithi194.stdout:(24/131): python3-rgw-18.0.0-4735.gbffe049d.el8 376 kB/s | 107 kB 00:00 2023-07-21T22:00:23.430 INFO:teuthology.orchestra.run.smithi194.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 756 kB/s | 94 kB 00:00 2023-07-21T22:00:23.534 INFO:teuthology.orchestra.run.smithi093.stdout:(16/131): libradosstriper1-18.0.0-4735.gbffe049 2.1 MB/s | 473 kB 00:00 2023-07-21T22:00:23.639 INFO:teuthology.orchestra.run.smithi194.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 861 kB/s | 179 kB 00:00 2023-07-21T22:00:23.764 INFO:teuthology.orchestra.run.smithi194.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 184 kB/s | 23 kB 00:00 2023-07-21T22:00:23.889 INFO:teuthology.orchestra.run.smithi194.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 1.0 MB/s | 129 kB 00:00 2023-07-21T22:00:24.098 INFO:teuthology.orchestra.run.smithi194.stdout:(29/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 4.5 MB/s | 3.4 MB 00:00 2023-07-21T22:00:24.509 INFO:teuthology.orchestra.run.smithi093.stdout:(17/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 2.6 MB/s | 18 MB 00:06 2023-07-21T22:00:24.634 INFO:teuthology.orchestra.run.smithi093.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 358 kB/s | 45 kB 00:00 2023-07-21T22:00:24.856 INFO:teuthology.orchestra.run.smithi194.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 1.8 MB/s | 1.7 MB 00:00 2023-07-21T22:00:24.893 INFO:teuthology.orchestra.run.smithi093.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 460 kB/s | 119 kB 00:00 2023-07-21T22:00:25.045 INFO:teuthology.orchestra.run.smithi194.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.2 MB/s | 240 kB 00:00 2023-07-21T22:00:25.151 INFO:teuthology.orchestra.run.smithi093.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 762 kB/s | 197 kB 00:00 2023-07-21T22:00:25.215 INFO:teuthology.orchestra.run.smithi194.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 280 kB/s | 47 kB 00:00 2023-07-21T22:00:25.418 INFO:teuthology.orchestra.run.smithi093.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 1.4 MB/s | 370 kB 00:00 2023-07-21T22:00:25.582 INFO:teuthology.orchestra.run.smithi194.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 40 kB/s | 15 kB 00:00 2023-07-21T22:00:25.719 INFO:teuthology.orchestra.run.smithi093.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 1.2 MB/s | 364 kB 00:00 2023-07-21T22:00:25.824 INFO:teuthology.orchestra.run.smithi194.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-07-21T22:00:25.866 INFO:teuthology.orchestra.run.smithi194.stdout:(35/131): ceph-volume-18.0.0-4735.gbffe049d.el8 910 kB/s | 258 kB 00:00 2023-07-21T22:00:25.869 INFO:teuthology.orchestra.run.smithi093.stdout:(23/131): python3-rgw-18.0.0-4735.gbffe049d.el8 712 kB/s | 107 kB 00:00 2023-07-21T22:00:25.992 INFO:teuthology.orchestra.run.smithi194.stdout:(36/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 978 kB/s | 164 kB 00:00 2023-07-21T22:00:25.994 INFO:teuthology.orchestra.run.smithi093.stdout:(24/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 756 kB/s | 94 kB 00:00 2023-07-21T22:00:26.025 INFO:teuthology.orchestra.run.smithi194.stdout:(37/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 113 kB/s | 18 kB 00:00 2023-07-21T22:00:26.461 INFO:teuthology.orchestra.run.smithi093.stdout:(25/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 2.3 MB/s | 6.7 MB 00:02 2023-07-21T22:00:26.578 INFO:teuthology.orchestra.run.smithi093.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.5 MB/s | 179 kB 00:00 2023-07-21T22:00:26.695 INFO:teuthology.orchestra.run.smithi093.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 198 kB/s | 23 kB 00:00 2023-07-21T22:00:26.708 INFO:teuthology.orchestra.run.smithi194.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 86 kB/s | 59 kB 00:00 2023-07-21T22:00:26.804 INFO:teuthology.orchestra.run.smithi093.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 1.2 MB/s | 129 kB 00:00 2023-07-21T22:00:26.942 INFO:teuthology.orchestra.run.smithi194.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 322 kB/s | 75 kB 00:00 2023-07-21T22:00:27.001 INFO:teuthology.orchestra.run.smithi194.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 304 kB/s | 306 kB 00:01 2023-07-21T22:00:27.184 INFO:teuthology.orchestra.run.smithi194.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 238 kB/s | 44 kB 00:00 2023-07-21T22:00:27.418 INFO:teuthology.orchestra.run.smithi194.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 795 kB/s | 185 kB 00:00 2023-07-21T22:00:27.460 INFO:teuthology.orchestra.run.smithi194.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 920 kB/s | 476 kB 00:00 2023-07-21T22:00:27.479 INFO:teuthology.orchestra.run.smithi093.stdout:(29/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 2.3 MB/s | 3.4 MB 00:01 2023-07-21T22:00:27.513 INFO:teuthology.orchestra.run.smithi093.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 2.4 MB/s | 1.7 MB 00:00 2023-07-21T22:00:27.593 INFO:teuthology.orchestra.run.smithi194.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 188 kB/s | 33 kB 00:00 2023-07-21T22:00:27.635 INFO:teuthology.orchestra.run.smithi194.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 93 kB/s | 16 kB 00:00 2023-07-21T22:00:27.655 INFO:teuthology.orchestra.run.smithi093.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.7 MB/s | 240 kB 00:00 2023-07-21T22:00:27.805 INFO:teuthology.orchestra.run.smithi093.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 316 kB/s | 47 kB 00:00 2023-07-21T22:00:27.810 INFO:teuthology.orchestra.run.smithi194.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 796 kB/s | 173 kB 00:00 2023-07-21T22:00:27.894 INFO:teuthology.orchestra.run.smithi194.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-07-21T22:00:27.955 INFO:teuthology.orchestra.run.smithi093.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 97 kB/s | 15 kB 00:00 2023-07-21T22:00:28.122 INFO:teuthology.orchestra.run.smithi093.stdout:(34/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.5 MB/s | 258 kB 00:00 2023-07-21T22:00:28.272 INFO:teuthology.orchestra.run.smithi093.stdout:(35/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 1.1 MB/s | 164 kB 00:00 2023-07-21T22:00:28.394 INFO:teuthology.orchestra.run.smithi194.stdout:(48/131): ceph-test-18.0.0-4735.gbffe049d.el8.x 5.8 MB/s | 48 MB 00:08 2023-07-21T22:00:28.420 INFO:teuthology.orchestra.run.smithi194.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 21 kB/s | 11 kB 00:00 2023-07-21T22:00:28.422 INFO:teuthology.orchestra.run.smithi093.stdout:(36/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 120 kB/s | 18 kB 00:00 2023-07-21T22:00:28.437 INFO:teuthology.orchestra.run.smithi194.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 130 kB/s | 81 kB 00:00 2023-07-21T22:00:28.587 INFO:teuthology.orchestra.run.smithi194.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 91 kB/s | 18 kB 00:00 2023-07-21T22:00:28.645 INFO:teuthology.orchestra.run.smithi194.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 234 kB/s | 49 kB 00:00 2023-07-21T22:00:28.821 INFO:teuthology.orchestra.run.smithi194.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 251 kB/s | 59 kB 00:00 2023-07-21T22:00:28.854 INFO:teuthology.orchestra.run.smithi194.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 289 kB/s | 60 kB 00:00 2023-07-21T22:00:29.013 INFO:teuthology.orchestra.run.smithi194.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 98 kB/s | 16 kB 00:00 2023-07-21T22:00:29.038 INFO:teuthology.orchestra.run.smithi194.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-07-21T22:00:29.205 INFO:teuthology.orchestra.run.smithi194.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 175 kB/s | 34 kB 00:00 2023-07-21T22:00:29.264 INFO:teuthology.orchestra.run.smithi194.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 872 kB/s | 196 kB 00:00 2023-07-21T22:00:29.381 INFO:teuthology.orchestra.run.smithi194.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 351 kB/s | 61 kB 00:00 2023-07-21T22:00:29.414 INFO:teuthology.orchestra.run.smithi194.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 125 kB/s | 19 kB 00:00 2023-07-21T22:00:29.548 INFO:teuthology.orchestra.run.smithi194.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 143 kB/s | 24 kB 00:00 2023-07-21T22:00:29.581 INFO:teuthology.orchestra.run.smithi194.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 175 kB/s | 29 kB 00:00 2023-07-21T22:00:29.707 INFO:teuthology.orchestra.run.smithi194.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 174 kB/s | 27 kB 00:00 2023-07-21T22:00:29.799 INFO:teuthology.orchestra.run.smithi194.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-07-21T22:00:29.899 INFO:teuthology.orchestra.run.smithi194.stdout:(65/131): python3-kubernetes-11.0.0-6.el8.noarc 807 kB/s | 1.2 MB 00:01 2023-07-21T22:00:29.949 INFO:teuthology.orchestra.run.smithi194.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-07-21T22:00:29.974 INFO:teuthology.orchestra.run.smithi194.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 349 kB/s | 61 kB 00:00 2023-07-21T22:00:30.083 INFO:teuthology.orchestra.run.smithi194.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 464 kB/s | 85 kB 00:00 2023-07-21T22:00:30.133 INFO:teuthology.orchestra.run.smithi194.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-07-21T22:00:30.158 INFO:teuthology.orchestra.run.smithi194.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 110 kB/s | 23 kB 00:00 2023-07-21T22:00:30.234 INFO:teuthology.orchestra.run.smithi194.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-07-21T22:00:30.276 INFO:teuthology.orchestra.run.smithi194.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-21T22:00:30.326 INFO:teuthology.orchestra.run.smithi194.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-21T22:00:30.376 INFO:teuthology.orchestra.run.smithi194.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-21T22:00:30.418 INFO:teuthology.orchestra.run.smithi194.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 938 kB/s | 39 kB 00:00 2023-07-21T22:00:30.485 INFO:teuthology.orchestra.run.smithi194.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-21T22:00:30.519 INFO:teuthology.orchestra.run.smithi194.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 3.2 MB/s | 1.7 MB 00:00 2023-07-21T22:00:30.577 INFO:teuthology.orchestra.run.smithi194.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.7 MB/s | 538 kB 00:00 2023-07-21T22:00:30.719 INFO:teuthology.orchestra.run.smithi194.stdout:(79/131): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-07-21T22:00:30.750 INFO:teuthology.orchestra.run.smithi194.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 695 kB/s | 157 kB 00:00 2023-07-21T22:00:30.770 INFO:teuthology.orchestra.run.smithi194.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-07-21T22:00:30.781 INFO:teuthology.orchestra.run.smithi093.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 2.2 MB/s | 7.4 MB 00:03 2023-07-21T22:00:30.804 INFO:teuthology.orchestra.run.smithi194.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-07-21T22:00:30.837 INFO:teuthology.orchestra.run.smithi194.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 398 kB/s | 27 kB 00:00 2023-07-21T22:00:30.871 INFO:teuthology.orchestra.run.smithi194.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2023-07-21T22:00:30.896 INFO:teuthology.orchestra.run.smithi194.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 750 kB/s | 44 kB 00:00 2023-07-21T22:00:30.946 INFO:teuthology.orchestra.run.smithi194.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-21T22:00:31.072 INFO:teuthology.orchestra.run.smithi194.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2023-07-21T22:00:31.139 INFO:teuthology.orchestra.run.smithi194.stdout:(88/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-07-21T22:00:31.197 INFO:teuthology.orchestra.run.smithi194.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-07-21T22:00:31.223 INFO:teuthology.orchestra.run.smithi194.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 425 kB/s | 64 kB 00:00 2023-07-21T22:00:31.247 INFO:teuthology.orchestra.run.smithi194.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-07-21T22:00:31.265 INFO:teuthology.orchestra.run.smithi194.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-07-21T22:00:31.290 INFO:teuthology.orchestra.run.smithi194.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-07-21T22:00:31.340 INFO:teuthology.orchestra.run.smithi194.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-07-21T22:00:31.365 INFO:teuthology.orchestra.run.smithi194.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-07-21T22:00:31.407 INFO:teuthology.orchestra.run.smithi194.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:00:31.482 INFO:teuthology.orchestra.run.smithi194.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-07-21T22:00:31.566 INFO:teuthology.orchestra.run.smithi194.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-07-21T22:00:31.642 INFO:teuthology.orchestra.run.smithi194.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-07-21T22:00:31.692 INFO:teuthology.orchestra.run.smithi194.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-07-21T22:00:31.725 INFO:teuthology.orchestra.run.smithi194.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-07-21T22:00:31.759 INFO:teuthology.orchestra.run.smithi194.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-07-21T22:00:31.784 INFO:teuthology.orchestra.run.smithi194.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-07-21T22:00:31.801 INFO:teuthology.orchestra.run.smithi194.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-21T22:00:31.818 INFO:teuthology.orchestra.run.smithi194.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-07-21T22:00:31.835 INFO:teuthology.orchestra.run.smithi194.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2023-07-21T22:00:31.860 INFO:teuthology.orchestra.run.smithi194.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-21T22:00:31.909 INFO:teuthology.orchestra.run.smithi194.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 574 kB/s | 43 kB 00:00 2023-07-21T22:00:31.944 INFO:teuthology.orchestra.run.smithi194.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2023-07-21T22:00:31.969 INFO:teuthology.orchestra.run.smithi194.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi194.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-21T22:00:32.028 INFO:teuthology.orchestra.run.smithi194.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-07-21T22:00:32.054 INFO:teuthology.orchestra.run.smithi194.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-21T22:00:32.121 INFO:teuthology.orchestra.run.smithi194.stdout:(114/131): postgresql-server-10.21-2.module+el8 7.9 MB/s | 5.1 MB 00:00 2023-07-21T22:00:32.146 INFO:teuthology.orchestra.run.smithi194.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 195 kB/s | 18 kB 00:00 2023-07-21T22:00:32.180 INFO:teuthology.orchestra.run.smithi194.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-07-21T22:00:32.205 INFO:teuthology.orchestra.run.smithi194.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T22:00:32.230 INFO:teuthology.orchestra.run.smithi194.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-07-21T22:00:32.247 INFO:teuthology.orchestra.run.smithi194.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-21T22:00:32.272 INFO:teuthology.orchestra.run.smithi194.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-21T22:00:32.297 INFO:teuthology.orchestra.run.smithi194.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-21T22:00:32.353 INFO:teuthology.orchestra.run.smithi194.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.2 MB/s | 98 kB 00:00 2023-07-21T22:00:32.381 INFO:teuthology.orchestra.run.smithi194.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 691 kB/s | 58 kB 00:00 2023-07-21T22:00:32.407 INFO:teuthology.orchestra.run.smithi194.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 731 kB/s | 39 kB 00:00 2023-07-21T22:00:32.440 INFO:teuthology.orchestra.run.smithi194.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-21T22:00:32.465 INFO:teuthology.orchestra.run.smithi194.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-21T22:00:32.491 INFO:teuthology.orchestra.run.smithi194.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-21T22:00:32.541 INFO:teuthology.orchestra.run.smithi194.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.4 MB/s | 2.3 MB 00:00 2023-07-21T22:00:32.591 INFO:teuthology.orchestra.run.smithi194.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T22:00:33.966 INFO:teuthology.orchestra.run.smithi194.stdout:(130/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 2.4 MB/s | 3.6 MB 00:01 2023-07-21T22:00:34.150 INFO:teuthology.orchestra.run.smithi194.stdout:(131/131): librados2-18.0.0-4735.gbffe049d.el8. 2.2 MB/s | 3.7 MB 00:01 2023-07-21T22:00:34.155 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:00:34.155 INFO:teuthology.orchestra.run.smithi194.stdout:Total 11 MB/s | 205 MB 00:19 2023-07-21T22:00:35.015 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2023-07-21T22:00:35.136 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2023-07-21T22:00:35.137 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2023-07-21T22:00:35.622 INFO:teuthology.orchestra.run.smithi093.stdout:(38/131): ceph-test-18.0.0-4735.gbffe049d.el8.x 3.4 MB/s | 48 MB 00:13 2023-07-21T22:00:36.390 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2023-07-21T22:00:36.394 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2023-07-21T22:00:41.352 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2023-07-21T22:00:41.516 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:00:41.693 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:00:42.318 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T22:00:42.476 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:00:42.574 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:00:43.267 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:00:43.451 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:00:43.878 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:00:44.101 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:00:44.216 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:00:44.418 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:00:44.512 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:00:44.668 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:00:44.913 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:00:45.095 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:00:45.482 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:00:45.746 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:00:45.884 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:00:45.997 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:00:46.192 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:00:46.275 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:00:46.400 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:00:46.545 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:00:46.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:00:47.330 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:00:47.542 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:00:48.139 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:00:48.317 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:00:49.069 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:00:49.340 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T22:00:49.559 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T22:00:49.778 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T22:00:49.931 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T22:00:50.124 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T22:00:50.344 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:00:50.523 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:00:50.669 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:00:51.577 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:00:51.888 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T22:00:52.168 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T22:00:52.450 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T22:00:52.633 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:00:54.384 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:00:54.717 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T22:00:54.919 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:00:55.134 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:00:55.305 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:00:55.495 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:00:55.656 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T22:00:55.810 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T22:00:56.001 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T22:00:56.153 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T22:00:56.357 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T22:00:56.572 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T22:00:56.922 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T22:00:57.168 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T22:00:57.384 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T22:00:57.586 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T22:00:57.837 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T22:00:58.157 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T22:00:58.331 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T22:00:58.491 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T22:00:58.703 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T22:00:58.868 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T22:00:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T22:00:59.339 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T22:00:59.618 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T22:00:59.861 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T22:01:00.104 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T22:01:00.264 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T22:01:00.458 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T22:01:00.631 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:01:00.695 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:01:00.961 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T22:01:01.143 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T22:01:01.340 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:01:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:01:01.635 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T22:01:01.832 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T22:01:01.997 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:01:02.192 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:01:02.424 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:01:02.655 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:01:02.802 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:01:02.942 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:01:03.191 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:01:03.385 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:01:03.601 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:01:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:01:03.912 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:01:04.157 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:01:04.377 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:01:04.562 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:01:04.785 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:01:04.988 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:01:05.138 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:01:05.355 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:01:06.468 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:01:06.775 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:01:06.989 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:01:07.306 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:01:07.604 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:01:07.813 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:01:08.639 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:01:08.788 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:01:08.858 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:01:09.047 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:01:12.702 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:01:13.042 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:01:13.653 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:01:13.943 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:01:14.026 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:01:14.137 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:01:33.002 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:01:33.002 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /run 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2023-07-21T22:01:33.003 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2023-07-21T22:01:33.004 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2023-07-21T22:01:33.004 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:01:33.368 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:01:34.934 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:01:35.140 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:01:35.365 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:01:35.568 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:01:36.032 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:01:36.193 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:01:36.210 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:01:37.858 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:01:38.026 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:01:38.174 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:01:38.356 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:01:38.562 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:01:38.775 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:01:39.064 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:01:40.021 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:01:40.184 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:01:40.273 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:01:40.625 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:01:40.861 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:01:40.896 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:01:41.016 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:01:41.028 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:01:42.306 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:01:42.507 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:01:42.541 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:01:42.725 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:01:42.767 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:01:42.942 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:01:43.287 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:01:43.475 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:01:48.000 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:01:48.279 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:01:48.482 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:01:48.672 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:01:49.579 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T22:01:49.905 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T22:01:50.068 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 119/134 2023-07-21T22:01:50.303 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:01:50.461 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:01:50.625 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:01:52.761 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:01:53.024 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:01:53.042 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:01:53.253 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:01:53.506 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:01:53.817 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:01:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:01:54.468 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T22:01:54.656 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:01:54.736 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:01:54.923 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:01:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:01:54.936 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:01:55.126 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:01:55.127 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:01:55.328 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:01:55.555 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:01:56.063 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:01:56.063 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:01:56.063 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:01:56.063 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:01:56.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:01:56.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:01:56.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:01:56.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:01:56.064 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:01:56.065 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:01:56.066 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:01:56.066 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:01:56.066 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:01:56.066 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:01:56.067 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:01:56.067 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:01:56.068 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:01:56.068 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:01:56.068 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:01:56.069 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:01:56.069 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:01:56.069 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:01:56.069 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:01:56.070 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:01:56.078 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:01:56.078 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:01:56.079 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:01:56.080 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:01:56.081 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:01:56.082 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:01:56.083 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T22:01:56.084 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T22:01:56.085 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T22:01:56.086 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T22:01:56.087 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T22:01:56.088 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T22:01:56.089 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T22:01:56.090 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T22:01:56.091 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:01:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:01:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:01:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:01:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:01:56.092 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:01:58.625 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:01:58.625 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:55 ago on Fri 21 Jul 2023 10:00:03 PM UTC. 2023-07-21T22:02:01.367 INFO:teuthology.orchestra.run.smithi194.stdout:Installed products updated. 2023-07-21T22:02:01.583 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:02:01.583 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2023-07-21T22:02:01.583 INFO:teuthology.orchestra.run.smithi194.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:02:01.583 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.584 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.585 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.586 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:02:01.587 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:02:01.588 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:02:01.589 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:02:01.590 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:02:01.591 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:02:01.592 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.593 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:02:01.594 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:02:01.595 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:02:01.596 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.597 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:02:01.598 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:02:01.599 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:02:01.600 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:02:01.601 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:02:01.601 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:02:01.601 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2023-07-21T22:02:01.798 DEBUG:teuthology.parallel:result is None 2023-07-21T22:02:37.918 INFO:teuthology.orchestra.run.smithi093.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:02:38.685 INFO:teuthology.orchestra.run.smithi093.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 kB/s | 306 kB 02:10 2023-07-21T22:02:41.889 INFO:teuthology.orchestra.run.smithi093.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:02:42.177 INFO:teuthology.orchestra.run.smithi093.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 458 B/s | 59 kB 02:11 2023-07-21T22:02:46.380 INFO:teuthology.orchestra.run.smithi093.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:02:46.643 INFO:teuthology.orchestra.run.smithi093.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 586 B/s | 75 kB 02:11 2023-07-21T22:02:46.901 INFO:teuthology.orchestra.run.smithi093.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 717 kB/s | 185 kB 00:00 2023-07-21T22:02:46.993 INFO:teuthology.orchestra.run.smithi093.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2023-07-21T22:02:47.102 INFO:teuthology.orchestra.run.smithi093.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-07-21T22:02:47.301 INFO:teuthology.orchestra.run.smithi093.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 868 kB/s | 173 kB 00:00 2023-07-21T22:02:47.677 INFO:teuthology.orchestra.run.smithi093.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.0 MB/s | 384 kB 00:00 2023-07-21T22:02:47.811 INFO:teuthology.orchestra.run.smithi093.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 609 kB/s | 81 kB 00:00 2023-07-21T22:02:47.936 INFO:teuthology.orchestra.run.smithi093.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-07-21T22:02:48.045 INFO:teuthology.orchestra.run.smithi093.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-07-21T22:02:48.620 INFO:teuthology.orchestra.run.smithi093.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-07-21T22:02:48.720 INFO:teuthology.orchestra.run.smithi093.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 488 kB/s | 49 kB 00:00 2023-07-21T22:02:48.821 INFO:teuthology.orchestra.run.smithi093.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 586 kB/s | 59 kB 00:00 2023-07-21T22:02:48.921 INFO:teuthology.orchestra.run.smithi093.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 603 kB/s | 60 kB 00:00 2023-07-21T22:02:49.113 INFO:teuthology.orchestra.run.smithi093.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-07-21T22:02:49.213 INFO:teuthology.orchestra.run.smithi093.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-07-21T22:02:49.305 INFO:teuthology.orchestra.run.smithi093.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 368 kB/s | 34 kB 00:00 2023-07-21T22:02:49.389 INFO:teuthology.orchestra.run.smithi093.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-07-21T22:02:49.480 INFO:teuthology.orchestra.run.smithi093.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 672 kB/s | 61 kB 00:00 2023-07-21T22:02:49.581 INFO:teuthology.orchestra.run.smithi093.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-07-21T22:02:49.664 INFO:teuthology.orchestra.run.smithi093.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 287 kB/s | 24 kB 00:00 2023-07-21T22:02:49.756 INFO:teuthology.orchestra.run.smithi093.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 319 kB/s | 29 kB 00:00 2023-07-21T22:02:50.198 INFO:teuthology.orchestra.run.smithi093.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 62 kB/s | 27 kB 00:00 2023-07-21T22:02:50.357 INFO:teuthology.orchestra.run.smithi093.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-07-21T22:02:50.515 INFO:teuthology.orchestra.run.smithi093.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-07-21T22:02:50.607 INFO:teuthology.orchestra.run.smithi093.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 669 kB/s | 61 kB 00:00 2023-07-21T22:02:50.699 INFO:teuthology.orchestra.run.smithi093.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 932 kB/s | 85 kB 00:00 2023-07-21T22:02:50.783 INFO:teuthology.orchestra.run.smithi093.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-07-21T22:02:51.483 INFO:teuthology.orchestra.run.smithi093.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 2.5 MB/s | 1.7 MB 00:00 2023-07-21T22:02:51.541 INFO:teuthology.orchestra.run.smithi093.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-07-21T22:02:51.817 INFO:teuthology.orchestra.run.smithi093.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 50 MB/s | 14 MB 00:00 2023-07-21T22:02:51.858 INFO:teuthology.orchestra.run.smithi093.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T22:02:51.892 INFO:teuthology.orchestra.run.smithi093.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-21T22:02:51.934 INFO:teuthology.orchestra.run.smithi093.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T22:02:51.976 INFO:teuthology.orchestra.run.smithi093.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-21T22:02:52.010 INFO:teuthology.orchestra.run.smithi093.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-07-21T22:02:52.052 INFO:teuthology.orchestra.run.smithi093.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 16 MB/s | 688 kB 00:00 2023-07-21T22:02:52.094 INFO:teuthology.orchestra.run.smithi093.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-07-21T22:02:52.127 INFO:teuthology.orchestra.run.smithi093.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-07-21T22:02:52.161 INFO:teuthology.orchestra.run.smithi093.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-07-21T22:02:52.203 INFO:teuthology.orchestra.run.smithi093.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2023-07-21T22:02:52.346 INFO:teuthology.orchestra.run.smithi093.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-07-21T22:02:52.378 INFO:teuthology.orchestra.run.smithi093.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 820 kB/s | 27 kB 00:00 2023-07-21T22:02:52.420 INFO:teuthology.orchestra.run.smithi093.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T22:02:52.454 INFO:teuthology.orchestra.run.smithi093.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-07-21T22:02:52.554 INFO:teuthology.orchestra.run.smithi093.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 37 MB/s | 3.7 MB 00:00 2023-07-21T22:02:52.596 INFO:teuthology.orchestra.run.smithi093.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-07-21T22:02:52.629 INFO:teuthology.orchestra.run.smithi093.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-07-21T22:02:52.663 INFO:teuthology.orchestra.run.smithi093.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-07-21T22:02:52.697 INFO:teuthology.orchestra.run.smithi093.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.7 MB/s | 126 kB 00:00 2023-07-21T22:02:52.730 INFO:teuthology.orchestra.run.smithi093.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 602 kB/s | 20 kB 00:00 2023-07-21T22:02:52.764 INFO:teuthology.orchestra.run.smithi093.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 3.0 MB/s | 103 kB 00:00 2023-07-21T22:02:52.806 INFO:teuthology.orchestra.run.smithi093.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 17 MB/s | 709 kB 00:00 2023-07-21T22:02:52.923 INFO:teuthology.orchestra.run.smithi093.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 42 MB/s | 4.9 MB 00:00 2023-07-21T22:02:52.965 INFO:teuthology.orchestra.run.smithi093.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-21T22:02:53.073 INFO:teuthology.orchestra.run.smithi093.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 44 MB/s | 4.7 MB 00:00 2023-07-21T22:02:53.115 INFO:teuthology.orchestra.run.smithi093.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:02:53.181 INFO:teuthology.orchestra.run.smithi093.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 5.3 MB/s | 354 kB 00:00 2023-07-21T22:02:53.299 INFO:teuthology.orchestra.run.smithi093.stdout:(98/131): postgresql-server-10.21-2.module+el8. 43 MB/s | 5.1 MB 00:00 2023-07-21T22:02:53.349 INFO:teuthology.orchestra.run.smithi093.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-07-21T22:02:53.408 INFO:teuthology.orchestra.run.smithi093.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 26 MB/s | 1.5 MB 00:00 2023-07-21T22:02:53.442 INFO:teuthology.orchestra.run.smithi093.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-07-21T22:02:53.483 INFO:teuthology.orchestra.run.smithi093.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-07-21T22:02:53.517 INFO:teuthology.orchestra.run.smithi093.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-07-21T22:02:53.550 INFO:teuthology.orchestra.run.smithi093.stdout:(104/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-07-21T22:02:53.584 INFO:teuthology.orchestra.run.smithi093.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.4 MB/s | 48 kB 00:00 2023-07-21T22:02:53.618 INFO:teuthology.orchestra.run.smithi093.stdout:(106/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-07-21T22:02:53.652 INFO:teuthology.orchestra.run.smithi093.stdout:(107/131): python3-requests-oauthlib-1.0.0-1.el 1.2 MB/s | 43 kB 00:00 2023-07-21T22:02:53.685 INFO:teuthology.orchestra.run.smithi093.stdout:(108/131): python3-pycparser-2.14-14.el8.noarch 3.2 MB/s | 109 kB 00:00 2023-07-21T22:02:53.727 INFO:teuthology.orchestra.run.smithi093.stdout:(109/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-07-21T22:02:53.769 INFO:teuthology.orchestra.run.smithi093.stdout:(110/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 13 MB/s | 560 kB 00:00 2023-07-21T22:02:53.802 INFO:teuthology.orchestra.run.smithi093.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 564 kB/s | 19 kB 00:00 2023-07-21T22:02:53.845 INFO:teuthology.orchestra.run.smithi093.stdout:(112/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-07-21T22:02:53.886 INFO:teuthology.orchestra.run.smithi093.stdout:(113/131): python3-cryptography-3.2.1-5.el8.x86 13 MB/s | 559 kB 00:00 2023-07-21T22:02:53.920 INFO:teuthology.orchestra.run.smithi093.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 538 kB/s | 18 kB 00:00 2023-07-21T22:02:53.962 INFO:teuthology.orchestra.run.smithi093.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 14 MB/s | 605 kB 00:00 2023-07-21T22:02:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:(116/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-07-21T22:02:54.096 INFO:teuthology.orchestra.run.smithi093.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 35 MB/s | 2.3 MB 00:00 2023-07-21T22:02:54.122 INFO:teuthology.orchestra.run.smithi093.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.8 MB/s | 72 kB 00:00 2023-07-21T22:02:54.155 INFO:teuthology.orchestra.run.smithi093.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-07-21T22:02:54.188 INFO:teuthology.orchestra.run.smithi093.stdout:(120/131): elfutils-libelf-devel-0.186-1.el8.x8 1.8 MB/s | 60 kB 00:00 2023-07-21T22:02:54.222 INFO:teuthology.orchestra.run.smithi093.stdout:(121/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.9 MB/s | 98 kB 00:00 2023-07-21T22:02:54.247 INFO:teuthology.orchestra.run.smithi093.stdout:(122/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 2.3 MB/s | 58 kB 00:00 2023-07-21T22:02:54.272 INFO:teuthology.orchestra.run.smithi093.stdout:(123/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.5 MB/s | 39 kB 00:00 2023-07-21T22:02:54.323 INFO:teuthology.orchestra.run.smithi093.stdout:(124/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-07-21T22:02:54.365 INFO:teuthology.orchestra.run.smithi093.stdout:(125/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 4.0 MB/s | 171 kB 00:00 2023-07-21T22:02:54.398 INFO:teuthology.orchestra.run.smithi093.stdout:(126/131): NetworkManager-initscripts-updown-1. 4.1 MB/s | 138 kB 00:00 2023-07-21T22:02:56.140 INFO:teuthology.orchestra.run.smithi093.stdout:(127/131): librados2-18.0.0-4735.gbffe049d.el8. 2.1 MB/s | 3.7 MB 00:01 2023-07-21T22:02:56.682 INFO:teuthology.orchestra.run.smithi093.stdout:(128/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 6.6 MB/s | 3.6 MB 00:00 2023-07-21T22:02:56.732 INFO:teuthology.orchestra.run.smithi093.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T22:04:11.068 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 60 kB/s | 16 MB 04:24 2023-07-21T22:04:14.517 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-07-21T22:04:15.427 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-07-21T22:04:23.369 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-07-21T22:04:34.492 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:34.493 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:34.507 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:34.508 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:34.509 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:04:34.510 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:04:34.513 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:04:34.516 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:04:34.517 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:04:34.647 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-21T22:04:34.659 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:04:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:04:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:04:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:04:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:04:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:04:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:04:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:04:34.667 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:04:34.668 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:04:34.669 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:04:34.670 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:04:34.671 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:04:34.672 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:04:34.673 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:04:34.674 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:04:34.675 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:04:34.676 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:04:34.677 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql 10 2023-07-21T22:04:34.678 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T22:04:34.679 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-21T22:04:34.679 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-21T22:04:34.679 INFO:teuthology.orchestra.run.smithi134.stdout:Install 128 Packages 2023-07-21T22:04:34.679 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 3 Packages 2023-07-21T22:04:34.679 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T22:04:34.680 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 205 M 2023-07-21T22:04:34.680 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-07-21T22:04:35.878 INFO:teuthology.orchestra.run.smithi134.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 13 kB/s | 6.4 kB 00:00 2023-07-21T22:04:37.545 INFO:teuthology.orchestra.run.smithi134.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 509 kB/s | 848 kB 00:01 2023-07-21T22:04:37.828 INFO:teuthology.orchestra.run.smithi134.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 585 kB/s | 165 kB 00:00 2023-07-21T22:04:39.853 INFO:teuthology.orchestra.run.smithi134.stdout:(4/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-07-21T22:04:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:(5/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-07-21T22:04:40.470 INFO:teuthology.orchestra.run.smithi134.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 2.5 MB/s | 1.5 MB 00:00 2023-07-21T22:04:46.262 INFO:teuthology.orchestra.run.smithi134.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 774 kB/s | 4.6 MB 00:06 2023-07-21T22:04:47.370 INFO:teuthology.orchestra.run.smithi134.stdout:(8/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x86 2.6 MB/s | 18 MB 00:06 2023-07-21T22:04:47.529 INFO:teuthology.orchestra.run.smithi134.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 154 kB/s | 24 kB 00:00 2023-07-21T22:04:49.314 INFO:teuthology.orchestra.run.smithi093.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:04:50.044 INFO:teuthology.orchestra.run.smithi093.stdout:(130/131): python3-asyncssh-2.7.0-2.el8.noarch. 3.6 kB/s | 476 kB 02:11 2023-07-21T22:04:50.937 INFO:teuthology.orchestra.run.smithi134.stdout:(10/131): ceph-common-18.0.0-4735.gbffe049d.el8 1.4 MB/s | 22 MB 00:15 2023-07-21T22:04:51.312 INFO:teuthology.orchestra.run.smithi134.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 689 kB/s | 258 kB 00:00 2023-07-21T22:04:51.454 INFO:teuthology.orchestra.run.smithi134.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 219 kB/s | 31 kB 00:00 2023-07-21T22:04:52.138 INFO:teuthology.orchestra.run.smithi134.stdout:(13/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 1.1 MB/s | 771 kB 00:00 2023-07-21T22:04:52.263 INFO:teuthology.orchestra.run.smithi134.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 1.1 MB/s | 137 kB 00:00 2023-07-21T22:04:52.413 INFO:teuthology.orchestra.run.smithi134.stdout:(15/131): librados-devel-18.0.0-4735.gbffe049d. 927 kB/s | 139 kB 00:00 2023-07-21T22:04:52.856 INFO:teuthology.orchestra.run.smithi134.stdout:(16/131): libradosstriper1-18.0.0-4735.gbffe049 1.0 MB/s | 473 kB 00:00 2023-07-21T22:04:52.959 INFO:teuthology.orchestra.run.smithi093.stdout:[MIRROR] python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:04:53.318 INFO:teuthology.orchestra.run.smithi093.stdout:(131/131): python3-bcrypt-3.1.6-2.el8.1.x86_64. 340 B/s | 44 kB 02:11 2023-07-21T22:04:53.320 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:04:53.320 INFO:teuthology.orchestra.run.smithi093.stdout:Total 755 kB/s | 205 MB 04:38 2023-07-21T22:04:54.102 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-07-21T22:04:54.222 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-07-21T22:04:54.222 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-07-21T22:04:55.324 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-07-21T22:04:55.329 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-07-21T22:04:56.522 INFO:teuthology.orchestra.run.smithi134.stdout:(17/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 1.8 MB/s | 6.7 MB 00:03 2023-07-21T22:04:56.680 INFO:teuthology.orchestra.run.smithi134.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 283 kB/s | 45 kB 00:00 2023-07-21T22:04:56.847 INFO:teuthology.orchestra.run.smithi134.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 714 kB/s | 119 kB 00:00 2023-07-21T22:04:57.022 INFO:teuthology.orchestra.run.smithi134.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.1 MB/s | 197 kB 00:00 2023-07-21T22:04:57.398 INFO:teuthology.orchestra.run.smithi134.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 988 kB/s | 370 kB 00:00 2023-07-21T22:04:57.673 INFO:teuthology.orchestra.run.smithi134.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 1.3 MB/s | 364 kB 00:00 2023-07-21T22:04:57.823 INFO:teuthology.orchestra.run.smithi134.stdout:(23/131): python3-rgw-18.0.0-4735.gbffe049d.el8 711 kB/s | 107 kB 00:00 2023-07-21T22:04:58.024 INFO:teuthology.orchestra.run.smithi134.stdout:(24/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 1.1 MB/s | 13 MB 00:11 2023-07-21T22:04:58.057 INFO:teuthology.orchestra.run.smithi134.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 404 kB/s | 94 kB 00:00 2023-07-21T22:04:58.216 INFO:teuthology.orchestra.run.smithi134.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-21T22:04:58.382 INFO:teuthology.orchestra.run.smithi134.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 138 kB/s | 23 kB 00:00 2023-07-21T22:04:58.549 INFO:teuthology.orchestra.run.smithi134.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 773 kB/s | 129 kB 00:00 2023-07-21T22:04:59.658 INFO:teuthology.orchestra.run.smithi134.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 1.5 MB/s | 1.7 MB 00:01 2023-07-21T22:05:00.136 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-07-21T22:05:00.305 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:05:00.501 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:05:00.705 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T22:05:00.866 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:05:00.972 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:05:01.192 INFO:teuthology.orchestra.run.smithi134.stdout:(30/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 1.1 MB/s | 3.4 MB 00:03 2023-07-21T22:05:01.216 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:05:01.458 INFO:teuthology.orchestra.run.smithi134.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 903 kB/s | 240 kB 00:00 2023-07-21T22:05:01.615 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:05:01.650 INFO:teuthology.orchestra.run.smithi134.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 247 kB/s | 47 kB 00:00 2023-07-21T22:05:01.800 INFO:teuthology.orchestra.run.smithi134.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 97 kB/s | 15 kB 00:00 2023-07-21T22:05:01.807 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:05:02.084 INFO:teuthology.orchestra.run.smithi134.stdout:(34/131): ceph-volume-18.0.0-4735.gbffe049d.el8 912 kB/s | 258 kB 00:00 2023-07-21T22:05:02.213 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:05:02.276 INFO:teuthology.orchestra.run.smithi134.stdout:(35/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 855 kB/s | 164 kB 00:00 2023-07-21T22:05:02.415 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:05:02.443 INFO:teuthology.orchestra.run.smithi134.stdout:(36/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 108 kB/s | 18 kB 00:00 2023-07-21T22:05:02.544 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:05:02.715 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:05:02.807 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:05:02.844 INFO:teuthology.orchestra.run.smithi134.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-07-21T22:05:03.007 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:05:03.078 INFO:teuthology.orchestra.run.smithi134.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 483 kB/s | 306 kB 00:00 2023-07-21T22:05:03.228 INFO:teuthology.orchestra.run.smithi134.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 501 kB/s | 75 kB 00:00 2023-07-21T22:05:03.235 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:05:03.311 INFO:teuthology.orchestra.run.smithi134.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 126 kB/s | 59 kB 00:00 2023-07-21T22:05:03.418 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:05:03.487 INFO:teuthology.orchestra.run.smithi134.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-07-21T22:05:03.512 INFO:teuthology.orchestra.run.smithi134.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-07-21T22:05:03.679 INFO:teuthology.orchestra.run.smithi134.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 198 kB/s | 33 kB 00:00 2023-07-21T22:05:03.787 INFO:teuthology.orchestra.run.smithi134.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 617 kB/s | 185 kB 00:00 2023-07-21T22:05:03.798 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:05:03.821 INFO:teuthology.orchestra.run.smithi134.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 115 kB/s | 16 kB 00:00 2023-07-21T22:05:03.971 INFO:teuthology.orchestra.run.smithi134.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 940 kB/s | 173 kB 00:00 2023-07-21T22:05:03.996 INFO:teuthology.orchestra.run.smithi134.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-21T22:05:04.027 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:05:04.138 INFO:teuthology.orchestra.run.smithi134.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 487 kB/s | 81 kB 00:00 2023-07-21T22:05:04.164 INFO:teuthology.orchestra.run.smithi134.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 66 kB/s | 11 kB 00:00 2023-07-21T22:05:04.165 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:05:04.262 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:05:04.289 INFO:teuthology.orchestra.run.smithi134.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-21T22:05:04.407 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:05:04.485 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:05:04.581 INFO:teuthology.orchestra.run.smithi134.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 167 kB/s | 49 kB 00:00 2023-07-21T22:05:04.615 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:05:04.623 INFO:teuthology.orchestra.run.smithi134.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 2.5 MB/s | 1.2 MB 00:00 2023-07-21T22:05:04.706 INFO:teuthology.orchestra.run.smithi134.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 468 kB/s | 59 kB 00:00 2023-07-21T22:05:04.739 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:05:04.740 INFO:teuthology.orchestra.run.smithi134.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-07-21T22:05:04.840 INFO:teuthology.orchestra.run.smithi134.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-07-21T22:05:04.857 INFO:teuthology.orchestra.run.smithi134.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-07-21T22:05:04.890 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:05:04.966 INFO:teuthology.orchestra.run.smithi134.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-07-21T22:05:04.991 INFO:teuthology.orchestra.run.smithi134.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-07-21T22:05:05.083 INFO:teuthology.orchestra.run.smithi134.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 526 kB/s | 61 kB 00:00 2023-07-21T22:05:05.100 INFO:teuthology.orchestra.run.smithi134.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-07-21T22:05:05.192 INFO:teuthology.orchestra.run.smithi134.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-07-21T22:05:05.209 INFO:teuthology.orchestra.run.smithi134.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-07-21T22:05:05.301 INFO:teuthology.orchestra.run.smithi134.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-07-21T22:05:05.343 INFO:teuthology.orchestra.run.smithi134.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-07-21T22:05:05.435 INFO:teuthology.orchestra.run.smithi134.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2023-07-21T22:05:05.460 INFO:teuthology.orchestra.run.smithi134.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2023-07-21T22:05:05.502 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:05:05.560 INFO:teuthology.orchestra.run.smithi134.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 683 kB/s | 85 kB 00:00 2023-07-21T22:05:05.586 INFO:teuthology.orchestra.run.smithi134.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-07-21T22:05:05.611 INFO:teuthology.orchestra.run.smithi134.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-21T22:05:05.681 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:05:05.903 INFO:teuthology.orchestra.run.smithi134.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 47 MB/s | 14 MB 00:00 2023-07-21T22:05:05.945 INFO:teuthology.orchestra.run.smithi134.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 4.5 MB/s | 1.7 MB 00:00 2023-07-21T22:05:05.970 INFO:teuthology.orchestra.run.smithi134.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-07-21T22:05:06.020 INFO:teuthology.orchestra.run.smithi134.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 719 kB/s | 54 kB 00:00 2023-07-21T22:05:06.045 INFO:teuthology.orchestra.run.smithi134.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.6 MB/s | 259 kB 00:00 2023-07-21T22:05:06.071 INFO:teuthology.orchestra.run.smithi134.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-21T22:05:06.096 INFO:teuthology.orchestra.run.smithi134.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 771 kB/s | 39 kB 00:00 2023-07-21T22:05:06.130 INFO:teuthology.orchestra.run.smithi134.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-07-21T22:05:06.163 INFO:teuthology.orchestra.run.smithi134.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-07-21T22:05:06.197 INFO:teuthology.orchestra.run.smithi134.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-07-21T22:05:06.222 INFO:teuthology.orchestra.run.smithi134.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-21T22:05:06.256 INFO:teuthology.orchestra.run.smithi134.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-21T22:05:06.276 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:05:06.298 INFO:teuthology.orchestra.run.smithi134.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 637 kB/s | 27 kB 00:00 2023-07-21T22:05:06.340 INFO:teuthology.orchestra.run.smithi134.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T22:05:06.374 INFO:teuthology.orchestra.run.smithi134.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-07-21T22:05:06.432 INFO:teuthology.orchestra.run.smithi134.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-07-21T22:05:06.465 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:05:06.491 INFO:teuthology.orchestra.run.smithi134.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.6 MB/s | 457 kB 00:00 2023-07-21T22:05:06.533 INFO:teuthology.orchestra.run.smithi134.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-21T22:05:06.591 INFO:teuthology.orchestra.run.smithi134.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-07-21T22:05:06.616 INFO:teuthology.orchestra.run.smithi134.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-07-21T22:05:06.642 INFO:teuthology.orchestra.run.smithi134.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-21T22:05:06.659 INFO:teuthology.orchestra.run.smithi134.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 478 kB/s | 20 kB 00:00 2023-07-21T22:05:06.692 INFO:teuthology.orchestra.run.smithi134.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-21T22:05:06.726 INFO:teuthology.orchestra.run.smithi134.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-21T22:05:06.768 INFO:teuthology.orchestra.run.smithi134.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T22:05:06.860 INFO:teuthology.orchestra.run.smithi134.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-07-21T22:05:06.902 INFO:teuthology.orchestra.run.smithi134.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:05:06.969 INFO:teuthology.orchestra.run.smithi134.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-07-21T22:05:06.994 INFO:teuthology.orchestra.run.smithi134.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-07-21T22:05:07.045 INFO:teuthology.orchestra.run.smithi134.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-07-21T22:05:07.128 INFO:teuthology.orchestra.run.smithi134.stdout:(100/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-07-21T22:05:07.170 INFO:teuthology.orchestra.run.smithi134.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-21T22:05:07.175 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:05:07.195 INFO:teuthology.orchestra.run.smithi134.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2023-07-21T22:05:07.221 INFO:teuthology.orchestra.run.smithi134.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-21T22:05:07.246 INFO:teuthology.orchestra.run.smithi134.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-07-21T22:05:07.271 INFO:teuthology.orchestra.run.smithi134.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-21T22:05:07.296 INFO:teuthology.orchestra.run.smithi134.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 959 kB/s | 48 kB 00:00 2023-07-21T22:05:07.322 INFO:teuthology.orchestra.run.smithi134.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-21T22:05:07.347 INFO:teuthology.orchestra.run.smithi134.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-07-21T22:05:07.372 INFO:teuthology.orchestra.run.smithi134.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-21T22:05:07.397 INFO:teuthology.orchestra.run.smithi134.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-21T22:05:07.431 INFO:teuthology.orchestra.run.smithi134.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-21T22:05:07.456 INFO:teuthology.orchestra.run.smithi134.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-07-21T22:05:07.463 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T22:05:07.483 INFO:teuthology.orchestra.run.smithi134.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-21T22:05:07.515 INFO:teuthology.orchestra.run.smithi134.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-07-21T22:05:07.540 INFO:teuthology.orchestra.run.smithi134.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-07-21T22:05:07.574 INFO:teuthology.orchestra.run.smithi134.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T22:05:07.599 INFO:teuthology.orchestra.run.smithi134.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T22:05:07.633 INFO:teuthology.orchestra.run.smithi134.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-07-21T22:05:07.683 INFO:teuthology.orchestra.run.smithi134.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 21 MB/s | 2.3 MB 00:00 2023-07-21T22:05:07.690 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T22:05:07.708 INFO:teuthology.orchestra.run.smithi134.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-07-21T22:05:07.733 INFO:teuthology.orchestra.run.smithi134.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-21T22:05:07.759 INFO:teuthology.orchestra.run.smithi134.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-21T22:05:07.786 INFO:teuthology.orchestra.run.smithi134.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-21T22:05:07.803 INFO:teuthology.orchestra.run.smithi134.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 875 kB/s | 39 kB 00:00 2023-07-21T22:05:07.837 INFO:teuthology.orchestra.run.smithi134.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-07-21T22:05:07.891 INFO:teuthology.orchestra.run.smithi134.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.9 MB/s | 171 kB 00:00 2023-07-21T22:05:07.893 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T22:05:07.921 INFO:teuthology.orchestra.run.smithi134.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.6 MB/s | 138 kB 00:00 2023-07-21T22:05:08.119 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T22:05:08.321 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T22:05:08.533 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:05:08.696 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:05:08.817 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:05:09.742 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:05:09.921 INFO:teuthology.orchestra.run.smithi134.stdout:(128/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 1.8 MB/s | 3.6 MB 00:01 2023-07-21T22:05:09.971 INFO:teuthology.orchestra.run.smithi134.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T22:05:10.086 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T22:05:10.341 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T22:05:10.604 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T22:05:10.789 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:05:10.871 INFO:teuthology.orchestra.run.smithi134.stdout:(130/131): librados2-18.0.0-4735.gbffe049d.el8. 1.2 MB/s | 3.7 MB 00:02 2023-07-21T22:05:11.546 INFO:teuthology.orchestra.run.smithi134.stdout:(131/131): ceph-test-18.0.0-4735.gbffe049d.el8. 2.0 MB/s | 48 MB 00:24 2023-07-21T22:05:11.550 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:05:11.550 INFO:teuthology.orchestra.run.smithi134.stdout:Total 5.6 MB/s | 205 MB 00:36 2023-07-21T22:05:12.340 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-07-21T22:05:12.460 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-07-21T22:05:12.461 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-07-21T22:05:12.545 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:05:12.866 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T22:05:13.059 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:05:13.266 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:05:13.461 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:05:13.583 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-07-21T22:05:13.587 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-07-21T22:05:13.635 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:05:13.796 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T22:05:13.990 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T22:05:14.166 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T22:05:14.326 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T22:05:14.537 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T22:05:14.737 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T22:05:15.110 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T22:05:15.400 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T22:05:15.616 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T22:05:15.828 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T22:05:16.069 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T22:05:16.373 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T22:05:16.580 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T22:05:16.739 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T22:05:16.951 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T22:05:17.100 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T22:05:17.258 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T22:05:17.593 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T22:05:17.841 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T22:05:18.076 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T22:05:18.315 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-07-21T22:05:18.336 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T22:05:18.495 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T22:05:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:05:18.631 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T22:05:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:05:18.797 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:05:18.866 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:05:18.933 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T22:05:19.110 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:05:19.126 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T22:05:19.241 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:05:19.349 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T22:05:19.477 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:05:19.505 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:05:19.602 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:05:19.825 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T22:05:19.875 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:05:19.980 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T22:05:20.035 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:05:20.429 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:05:20.463 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:05:20.626 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:05:20.642 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:05:20.754 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:05:20.909 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:05:20.927 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:05:21.009 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:05:21.120 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:05:21.185 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:05:21.267 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:05:21.399 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:05:21.421 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:05:21.581 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:05:21.637 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:05:21.801 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:05:21.982 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:05:22.018 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:05:22.149 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:05:22.213 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:05:22.286 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:05:22.343 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:05:22.449 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:05:22.528 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:05:22.601 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:05:22.688 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:05:22.775 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:05:22.826 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:05:22.960 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:05:22.983 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:05:23.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:05:23.200 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:05:23.421 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:05:23.579 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:05:23.753 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:05:23.796 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:05:23.959 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:05:24.552 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:05:24.735 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:05:24.903 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:05:25.207 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:05:25.453 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:05:25.730 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:05:25.857 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T22:05:26.085 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T22:05:26.137 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:05:26.312 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T22:05:26.342 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:05:26.490 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T22:05:26.681 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T22:05:26.918 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:05:27.082 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:05:27.122 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:05:27.202 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:05:27.278 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:27.362 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:27.521 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:05:28.081 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T22:05:28.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T22:05:28.868 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T22:05:29.084 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:05:30.828 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:05:31.103 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:31.127 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T22:05:31.312 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:05:31.329 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:31.468 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:05:31.623 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:05:31.828 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:05:31.934 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:31.982 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T22:05:32.126 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T22:05:32.240 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:32.302 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T22:05:32.321 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:32.445 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T22:05:32.445 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:32.690 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T22:05:32.863 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T22:05:33.210 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T22:05:33.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T22:05:33.668 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T22:05:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T22:05:34.104 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T22:05:34.416 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T22:05:34.707 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T22:05:34.905 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T22:05:35.112 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T22:05:35.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T22:05:35.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T22:05:35.708 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T22:05:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T22:05:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T22:05:36.439 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T22:05:36.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T22:05:36.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T22:05:36.891 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:05:36.944 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:05:37.184 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T22:05:37.361 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T22:05:37.524 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:05:37.589 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:05:37.777 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T22:05:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T22:05:38.115 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:05:38.301 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:05:38.549 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:05:38.756 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:05:38.928 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:05:39.059 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:05:39.223 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:05:39.435 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:05:39.600 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:05:39.743 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:05:39.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:05:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:05:40.353 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:05:40.530 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:05:40.764 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:05:40.965 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:05:41.106 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:05:41.299 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:05:42.389 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:05:42.660 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:05:42.906 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:05:43.239 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:05:43.497 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:05:43.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:05:44.466 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:05:44.656 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:44.733 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:44.881 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:05:48.553 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:48.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:49.380 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:49.645 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:49.726 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:49.864 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:51.168 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /run 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2023-07-21T22:05:51.169 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2023-07-21T22:05:51.170 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2023-07-21T22:05:51.170 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2023-07-21T22:05:51.170 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-07-21T22:05:51.170 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:05:51.501 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:53.070 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:53.237 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:53.459 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:53.642 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:54.108 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:54.301 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:54.317 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:56.086 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:05:56.247 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:05:56.364 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:05:56.564 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:05:56.774 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:05:56.985 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:05:57.278 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:05:58.237 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:05:58.366 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:05:58.457 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:05:58.814 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:05:59.038 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:05:59.072 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:05:59.249 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:05:59.260 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:06:00.533 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:06:00.711 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:00.744 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:00.866 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:00.908 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:01.041 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:01.389 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:01.533 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:06.050 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:06.324 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T22:06:08.068 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 119/134 2023-07-21T22:06:08.269 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:06:08.461 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:06:08.559 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:06:08.560 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2023-07-21T22:06:08.560 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2023-07-21T22:06:08.560 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-07-21T22:06:08.560 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2023-07-21T22:06:08.560 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2023-07-21T22:06:08.561 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2023-07-21T22:06:08.561 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2023-07-21T22:06:08.561 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2023-07-21T22:06:08.561 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-07-21T22:06:08.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T22:06:08.599 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:06:08.886 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:06:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:06:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:06:10.748 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:10.843 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:06:11.011 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:06:11.023 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:11.042 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:11.207 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:06:11.464 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:06:11.484 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:06:11.644 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:06:11.661 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:06:11.756 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:06:12.050 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:06:12.173 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:12.181 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:12.361 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T22:06:12.539 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:06:12.629 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:06:12.839 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:06:12.851 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:06:12.851 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:13.059 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:13.059 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:13.261 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:13.291 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:06:13.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:06:13.505 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:13.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:06:13.741 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:06:13.918 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:06:14.029 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:06:14.030 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:06:14.031 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:06:14.032 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:06:14.033 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:06:14.034 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:06:14.034 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:06:14.034 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:06:14.034 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:06:14.034 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:06:14.042 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:06:14.042 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:06:14.042 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:06:14.043 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:06:14.044 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:06:14.045 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:06:14.046 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:06:14.047 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:06:14.048 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:06:14.048 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:06:14.048 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:06:14.048 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:06:14.048 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T22:06:14.049 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T22:06:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T22:06:14.051 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T22:06:14.052 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T22:06:14.053 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T22:06:14.054 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T22:06:14.055 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:06:14.056 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:06:14.057 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:06:14.058 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:06:14.058 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:14.058 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:06:14.130 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:06:14.391 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:06:15.353 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:06:15.493 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:06:15.584 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:06:15.927 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:06:16.120 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:06:16.154 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:06:16.277 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:06:16.287 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:06:16.613 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:16.614 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:06:14 ago on Fri 21 Jul 2023 10:00:02 PM UTC. 2023-07-21T22:06:17.559 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:06:17.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:17.774 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:17.926 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:17.968 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:18.448 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:18.602 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:19.219 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-07-21T22:06:19.433 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:06:19.433 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-07-21T22:06:19.433 INFO:teuthology.orchestra.run.smithi093.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:06:19.434 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.434 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.434 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-07-21T22:06:19.434 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:06:19.435 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:06:19.435 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:06:19.435 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.435 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.436 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.436 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.436 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.437 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.438 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.439 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.439 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.439 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:19.439 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:06:19.439 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:06:19.440 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:06:19.441 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:06:19.442 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:06:19.443 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:19.444 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:06:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:06:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:06:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:06:19.445 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:06:19.446 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:06:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:06:19.448 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:06:19.449 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.450 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:06:19.451 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:06:19.452 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:06:19.453 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:06:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:06:19.454 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:06:19.454 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-07-21T22:06:19.615 DEBUG:teuthology.parallel:result is None 2023-07-21T22:06:23.122 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:23.385 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:06:23.539 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:06:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:06:24.554 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T22:06:24.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T22:06:25.053 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 119/134 2023-07-21T22:06:25.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:06:25.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:06:25.585 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:06:27.743 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:27.975 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:27.992 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:28.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:06:28.466 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:06:28.777 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:06:29.027 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:06:29.150 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:29.158 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:29.323 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T22:06:29.500 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:06:29.586 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:06:29.825 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:06:29.837 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:06:29.837 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:30.036 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:30.036 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:30.255 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:30.432 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:06:30.955 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:06:30.956 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:06:30.957 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:06:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:06:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:06:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:06:30.958 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:06:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:06:30.959 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:06:30.960 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:06:30.961 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:06:30.967 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:06:30.967 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:06:30.967 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:06:30.967 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:06:30.968 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:06:30.969 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:06:30.970 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:06:30.971 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T22:06:30.972 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T22:06:30.973 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T22:06:30.974 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T22:06:30.975 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T22:06:30.975 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T22:06:30.975 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T22:06:30.975 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T22:06:30.975 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T22:06:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T22:06:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T22:06:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T22:06:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T22:06:30.976 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T22:06:30.977 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T22:06:30.978 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T22:06:30.979 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T22:06:30.980 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:30.981 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:06:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:33.388 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:02:10 ago on Fri 21 Jul 2023 10:04:23 PM UTC. 2023-07-21T22:06:35.886 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-07-21T22:06:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T22:06:36.100 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2023-07-21T22:06:36.100 INFO:teuthology.orchestra.run.smithi134.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:06:36.101 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.102 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.103 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.104 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:06:36.105 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:06:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.107 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:06:36.108 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:06:36.109 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:06:36.110 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.111 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:06:36.112 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:06:36.113 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:06:36.114 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:06:36.115 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:06:36.116 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:06:36.117 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.118 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:06:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:06:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:06:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:06:36.119 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:06:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:06:36.120 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-21T22:06:36.120 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-21T22:06:36.290 DEBUG:teuthology.parallel:result is None 2023-07-21T22:06:36.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:37.034 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:06:37.062 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:37.063 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:37.063 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:06:37.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:37.759 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:06:37.787 INFO:teuthology.orchestra.run.smithi134.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:37.787 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:37.788 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:06:37.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:38.422 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi194.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:38.451 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:06:38.451 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:06:38.452 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-21T22:06:38.452 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:06:38.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:06:38.500 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:06:38.501 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:06:38.547 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:06:38.548 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:06:38.597 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-21T22:06:38.597 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:06:38.597 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:06:38.628 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:06:38.697 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:06:38.697 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:06:38.730 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:06:38.800 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:06:38.800 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:06:38.833 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:06:38.904 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-21T22:06:38.904 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:06:38.904 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:06:38.933 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:06:39.002 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:06:39.003 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:06:39.034 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:06:39.104 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:06:39.105 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:06:39.134 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:06:39.203 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-21T22:06:39.204 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:06:39.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:06:39.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:06:39.308 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:06:39.309 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:06:39.340 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:06:39.410 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:06:39.411 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:06:39.441 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:06:39.509 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-21T22:06:39.591 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-21T22:06:39.592 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:39.602 INFO:tasks.cephadm:Cluster fsid is de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:06:39.603 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-21T22:06:39.603 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.93', 'mon.b': '172.21.15.134', 'mon.c': '172.21.15.194'} 2023-07-21T22:06:39.604 INFO:tasks.cephadm:First mon is mon.a on smithi093 2023-07-21T22:06:39.604 INFO:tasks.cephadm:First mgr is x 2023-07-21T22:06:39.604 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-21T22:06:39.605 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname $(hostname -s) 2023-07-21T22:06:39.639 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2023-07-21T22:06:39.672 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2023-07-21T22:06:39.701 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-21T22:06:39.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:45.235 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71611/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4735-gbffe049d', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4735.gbffe049d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-21 20:32:06.130547', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-base-2', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-07-21T22:06:50.784 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-2023Jul13-base-2, sha1 bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:50.785 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm 2023-07-21T22:06:50.786 DEBUG:teuthology.orchestra.run.smithi093:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:52.624 INFO:teuthology.orchestra.run.smithi093.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:06 /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:52.624 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:54.076 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:06 /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:54.077 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:55.881 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:06 /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:55.881 DEBUG:teuthology.orchestra.run.smithi093:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:55.902 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:55.923 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:06:55.960 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea on all hosts... 2023-07-21T22:06:55.960 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:06:55.963 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:06:55.965 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:06:57.224 INFO:teuthology.orchestra.run.smithi093.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:06:57.272 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:06:59.028 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:07:27.811 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-21T22:07:27.812 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:07:27.812 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:07:27.813 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2023-07-21T22:07:27.813 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:07:27.813 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2023-07-21T22:07:27.813 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-21T22:07:52.579 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2023-07-21T22:07:52.579 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:07:52.579 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:07:52.580 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2023-07-21T22:07:52.580 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:07:52.580 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2023-07-21T22:07:52.580 INFO:teuthology.orchestra.run.smithi194.stdout:} 2023-07-21T22:07:54.367 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout: "repo_digests": [ 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:07:54.368 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:07:54.387 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph 2023-07-21T22:07:54.416 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2023-07-21T22:07:54.455 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2023-07-21T22:07:54.491 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /etc/ceph 2023-07-21T22:07:54.517 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2023-07-21T22:07:54.549 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2023-07-21T22:07:54.579 INFO:tasks.cephadm:Writing seed config... 2023-07-21T22:07:54.581 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-21T22:07:54.581 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-21T22:07:54.582 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-21T22:07:54.582 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-21T22:07:54.582 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-21T22:07:54.582 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-21T22:07:54.583 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-21T22:07:54.583 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-21T22:07:54.583 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-21T22:07:54.583 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-21T22:07:54.583 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-21T22:07:54.584 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-21T22:07:54.584 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-21T22:07:54.584 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-21T22:07:54.584 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-21T22:07:54.585 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-21T22:07:54.585 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-21T22:07:54.585 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-07-21T22:07:54.585 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-21T22:07:54.585 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-21T22:07:54.586 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-21T22:07:54.587 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-21T22:07:54.587 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-21T22:07:54.588 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-21T22:07:54.588 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-21T22:07:54.588 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-21T22:07:54.588 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-21T22:07:54.589 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-21T22:07:54.590 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-21T22:07:54.590 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-21T22:07:54.590 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-21T22:07:54.590 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-21T22:07:54.590 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-21T22:07:54.591 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-21T22:07:54.591 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-21T22:07:54.591 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-21T22:07:54.591 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-21T22:07:54.592 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:07:54.592 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-21T22:07:54.609 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = de1cec7a-2812-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-21T22:07:54.609 DEBUG:teuthology.orchestra.run.smithi093:mon.a> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service 2023-07-21T22:07:54.652 DEBUG:teuthology.orchestra.run.smithi093:mgr.x> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-21T22:07:54.677 INFO:journalctl@ceph.mon.a.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:07:54.678 INFO:tasks.cephadm:Bootstrapping... 2023-07-21T22:07:54.678 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea -v bootstrap --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.93 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:07:54.703 INFO:journalctl@ceph.mgr.x.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:07:54.836 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:07:54.836 INFO:teuthology.orchestra.run.smithi093.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', '-v', 'bootstrap', '--fsid', 'de1cec7a-2812-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.93', '--skip-admin-label'] 2023-07-21T22:07:54.980 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:07:54.981 INFO:teuthology.orchestra.run.smithi093.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-21T22:07:54.981 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying podman|docker is present... 2023-07-21T22:07:55.121 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:07:55.122 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying lvm2 is present... 2023-07-21T22:07:55.122 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying time synchronization is in place... 2023-07-21T22:07:55.129 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:07:55.129 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:07:55.134 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:07:55.134 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:07:55.140 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-07-21T22:07:55.146 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-07-21T22:07:55.146 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:07:55.146 INFO:teuthology.orchestra.run.smithi093.stdout:Repeating the final host check... 2023-07-21T22:07:55.290 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:07:55.290 INFO:teuthology.orchestra.run.smithi093.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-21T22:07:55.290 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl is present 2023-07-21T22:07:55.291 INFO:teuthology.orchestra.run.smithi093.stdout:lvcreate is present 2023-07-21T22:07:55.296 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:07:55.296 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:07:55.301 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:07:55.302 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:07:55.307 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-07-21T22:07:55.313 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-07-21T22:07:55.313 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:07:55.313 INFO:teuthology.orchestra.run.smithi093.stdout:Host looks OK 2023-07-21T22:07:55.313 INFO:teuthology.orchestra.run.smithi093.stdout:Cluster fsid: de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:07:55.313 INFO:teuthology.orchestra.run.smithi093.stdout:Acquiring lock 140286259243496 on /run/cephadm/de1cec7a-2812-11ee-9b34-001a4aab830c.lock 2023-07-21T22:07:55.314 INFO:teuthology.orchestra.run.smithi093.stdout:Lock 140286259243496 acquired on /run/cephadm/de1cec7a-2812-11ee-9b34-001a4aab830c.lock 2023-07-21T22:07:55.314 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 3300 ... 2023-07-21T22:07:55.314 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 6789 ... 2023-07-21T22:07:55.314 INFO:teuthology.orchestra.run.smithi093.stdout:Base mon IP(s) is [172.21.15.93:3300, 172.21.15.93:6789], mon addrv is [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-07-21T22:07:55.317 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-21T22:07:55.317 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.93 metric 100 2023-07-21T22:07:55.320 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-21T22:07:55.320 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-21T22:07:55.320 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-21T22:07:55.322 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-21T22:07:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-21T22:07:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:07:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-21T22:07:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 fe80::1d18:7f22:62a7:7e48/64 scope link noprefixroute 2023-07-21T22:07:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:07:55.324 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-07-21T22:07:55.324 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-07-21T22:07:55.324 INFO:teuthology.orchestra.run.smithi093.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-21T22:07:55.324 INFO:teuthology.orchestra.run.smithi093.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-21T22:07:55.325 INFO:teuthology.orchestra.run.smithi093.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Getting image source signatures 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:99ee8d5f86f7949582f207d2e584c4024ff3caa1cfdcb812f3da7de6a1ca5e74 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-21T22:07:55.772 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying config sha256:fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:07:55.773 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-21T22:07:55.773 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Storing signatures 2023-07-21T22:07:56.629 INFO:teuthology.orchestra.run.smithi093.stdout:ceph: stdout ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:07:56.629 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph version: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:07:56.629 INFO:teuthology.orchestra.run.smithi093.stdout:Extracting ceph user uid/gid from container image... 2023-07-21T22:07:57.379 INFO:teuthology.orchestra.run.smithi093.stdout:stat: stdout 167 167 2023-07-21T22:07:57.379 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial keys... 2023-07-21T22:07:58.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQC9AbtkkVLbJxAAS5tojM6KU1PnKE3h5Qp8Tw== 2023-07-21T22:07:58.772 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQC+AbtkQot0GRAAkeGBHQoHJSEqcqKkbOYBnw== 2023-07-21T22:07:59.419 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQC/AbtkmHYWBBAA49R8BSDe/2zWNRppFDkfCQ== 2023-07-21T22:07:59.420 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial monmap... 2023-07-21T22:08:00.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:08:00.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-21T22:08:00.167 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:monmaptool for a [v2:172.21.15.93:3300,v1:172.21.15.93:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:setting min_mon_release = pacific 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: set fsid to de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:00.168 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mon... 2023-07-21T22:08:00.919 INFO:teuthology.orchestra.run.smithi093.stdout:create mon.a on 2023-07-21T22:08:01.036 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-21T22:08:01.122 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-21T22:08:01.239 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target. 2023-07-21T22:08:01.239 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target. 2023-07-21T22:08:01.432 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:08:01.432 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-21T22:08:01.517 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target.wants/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:08:01.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:01 smithi093 systemd[1]: Starting Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:08:02.022 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:08:02.023 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-07-21T22:08:02.028 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:08:02.028 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:08:02.028 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-07-21T22:08:02.028 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:08:02.029 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon to start... 2023-07-21T22:08:02.029 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon... 2023-07-21T22:08:02.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:01 smithi093 podman[96042]: 2023-07-21T22:08:02.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:01 smithi093 bash[96042]: b6b5b418cadcd36e2e0240426eb47e59816796b7c61a0ef2e32722fd70e21c77 2023-07-21T22:08:02.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:01 smithi093 systemd[1]: Started Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:08:02.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:02 smithi093 ceph-mon[96081]: mkfs de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:02.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:02 smithi093 ceph-mon[96081]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:08:02.880 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout cluster: 2023-07-21T22:08:02.881 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout id: de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:02.881 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-21T22:08:02.881 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:02.881 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout services: 2023-07-21T22:08:02.881 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.520171s) 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout data: 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-21T22:08:02.882 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pgs: 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.447+0000 7f3c793e8700 1 Processor -- start 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.447+0000 7f3c793e8700 1 -- start start 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.447+0000 7f3c793e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.447+0000 7f3c793e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74079700 con 0x7f3c7407acc0 2023-07-21T22:08:02.883 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:02.884 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40758/0 (socket says 172.21.15.93:40758) 2023-07-21T22:08:02.884 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c72ffd700 1 -- 172.21.15.93:0/2766425993 learned_addr learned my addr 172.21.15.93:0/2766425993 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:02.884 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c72ffd700 1 -- 172.21.15.93:0/2766425993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74079840 con 0x7f3c7407acc0 2023-07-21T22:08:02.885 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c72ffd700 1 --2- 172.21.15.93:0/2766425993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098a0 tx=0x7f3c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899570b6bde250ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:02.885 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c71ffb700 1 -- 172.21.15.93:0/2766425993 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68004160 con 0x7f3c7407acc0 2023-07-21T22:08:02.886 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c71ffb700 1 -- 172.21.15.93:0/2766425993 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680042c0 con 0x7f3c7407acc0 2023-07-21T22:08:02.886 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.448+0000 7f3c71ffb700 1 -- 172.21.15.93:0/2766425993 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c680045a0 con 0x7f3c7407acc0 2023-07-21T22:08:02.886 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- 172.21.15.93:0/2766425993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 msgr2=0x7f3c740791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:02.887 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 --2- 172.21.15.93:0/2766425993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c680098a0 tx=0x7f3c68009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:02.887 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- 172.21.15.93:0/2766425993 shutdown_connections 2023-07-21T22:08:02.887 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 --2- 172.21.15.93:0/2766425993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c740791c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:02.887 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- 172.21.15.93:0/2766425993 >> 172.21.15.93:0/2766425993 conn(0x7f3c74100b70 msgr2=0x7f3c74102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:02.888 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- 172.21.15.93:0/2766425993 shutdown_connections 2023-07-21T22:08:02.888 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- 172.21.15.93:0/2766425993 wait complete. 2023-07-21T22:08:02.888 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 Processor -- start 2023-07-21T22:08:02.889 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.449+0000 7f3c793e8700 1 -- start start 2023-07-21T22:08:02.889 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c793e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:02.889 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c793e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7419b1d0 con 0x7f3c7407acc0 2023-07-21T22:08:02.889 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:02.890 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40760/0 (socket says 172.21.15.93:40760) 2023-07-21T22:08:02.890 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c72ffd700 1 -- 172.21.15.93:0/2109048853 learned_addr learned my addr 172.21.15.93:0/2109048853 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:02.890 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c72ffd700 1 -- 172.21.15.93:0/2109048853 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009550 con 0x7f3c7407acc0 2023-07-21T22:08:02.891 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c72ffd700 1 --2- 172.21.15.93:0/2109048853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c68004000 tx=0x7f3c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:02.891 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68011630 con 0x7f3c7407acc0 2023-07-21T22:08:02.891 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68011790 con 0x7f3c7407acc0 2023-07-21T22:08:02.891 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7419b3d0 con 0x7f3c7407acc0 2023-07-21T22:08:02.892 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c68011a30 con 0x7f3c7407acc0 2023-07-21T22:08:02.892 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7419b7f0 con 0x7f3c7407acc0 2023-07-21T22:08:02.892 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3c68022070 con 0x7f3c7407acc0 2023-07-21T22:08:02.892 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.450+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3c6801c450 con 0x7f3c7407acc0 2023-07-21T22:08:02.892 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.451+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c74194a50 con 0x7f3c7407acc0 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.453+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3c6801b450 con 0x7f3c7407acc0 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.522+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3c7402cab0 con 0x7f3c7407acc0 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.522+0000 7f3c5bfff700 1 -- 172.21.15.93:0/2109048853 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f3c6801b5f0 con 0x7f3c7407acc0 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 msgr2=0x7f3c7419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 --2- 172.21.15.93:0/2109048853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c68004000 tx=0x7f3c68004750 comp rx=0 tx=0).stop 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 shutdown_connections 2023-07-21T22:08:02.893 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 --2- 172.21.15.93:0/2109048853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7407acc0 0x7f3c7419ac90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:02.894 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 >> 172.21.15.93:0/2109048853 conn(0x7f3c74100b70 msgr2=0x7f3c74102480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:02.894 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 shutdown_connections 2023-07-21T22:08:02.894 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:02.523+0000 7f3c793e8700 1 -- 172.21.15.93:0/2109048853 wait complete. 2023-07-21T22:08:02.894 INFO:teuthology.orchestra.run.smithi093.stdout:mon is available 2023-07-21T22:08:02.894 INFO:teuthology.orchestra.run.smithi093.stdout:Assimilating anything we can from ceph.conf... 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: fsmap 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: mgrmap e1: no daemons active 2023-07-21T22:08:03.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:03 smithi093 ceph-mon[96081]: from='client.? 172.21.15.93:0/2109048853' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-21T22:08:04.225 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:04.225 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:08:04.225 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:04.226 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:08:04.227 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.325+0000 7fbb22766700 1 Processor -- start 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.325+0000 7fbb22766700 1 -- start start 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.325+0000 7fbb22766700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:04.228 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.325+0000 7fbb22766700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c073be0 con 0x7fbb1c106050 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.326+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.326+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40762/0 (socket says 172.21.15.93:40762) 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.326+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1584749830 learned_addr learned my addr 172.21.15.93:0/1584749830 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.326+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1584749830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c073d20 con 0x7fbb1c106050 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.326+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1584749830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0098a0 tx=0x7fbb0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62238ded8fe93c10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb1affd700 1 -- 172.21.15.93:0/1584749830 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c004160 con 0x7fbb1c106050 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb1affd700 1 -- 172.21.15.93:0/1584749830 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0042c0 con 0x7fbb1c106050 2023-07-21T22:08:04.229 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb1affd700 1 -- 172.21.15.93:0/1584749830 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c0045a0 con 0x7fbb1c106050 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 -- 172.21.15.93:0/1584749830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 msgr2=0x7fbb1c108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 --2- 172.21.15.93:0/1584749830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0098a0 tx=0x7fbb0c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 -- 172.21.15.93:0/1584749830 shutdown_connections 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 --2- 172.21.15.93:0/1584749830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c108430 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 -- 172.21.15.93:0/1584749830 >> 172.21.15.93:0/1584749830 conn(0x7fbb1c0ffa50 msgr2=0x7fbb1c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 -- 172.21.15.93:0/1584749830 shutdown_connections 2023-07-21T22:08:04.230 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.327+0000 7fbb22766700 1 -- 172.21.15.93:0/1584749830 wait complete. 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb22766700 1 Processor -- start 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb22766700 1 -- start start 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb22766700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb22766700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c105ef0 con 0x7fbb1c106050 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40764/0 (socket says 172.21.15.93:40764) 2023-07-21T22:08:04.231 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1302313293 learned_addr learned my addr 172.21.15.93:0/1302313293 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.328+0000 7fbb1bfff700 1 -- 172.21.15.93:0/1302313293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c009550 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb1bfff700 1 --2- 172.21.15.93:0/1302313293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004000 tx=0x7fbb0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c011630 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c011790 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb0c011a30 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c104060 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1044e0 con 0x7fbb1c106050 2023-07-21T22:08:04.232 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbb0c022070 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.329+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbb0c01c450 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.330+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c061b40 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.332+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbb0c01b450 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.398+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fbb1c192160 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.402+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb0c01b5f0 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.402+0000 7fbb197fa700 1 -- 172.21.15.93:0/1302313293 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fbb1c192160 con 0x7fbb1c106050 2023-07-21T22:08:04.233 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 msgr2=0x7fbb1c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 --2- 172.21.15.93:0/1302313293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004000 tx=0x7fbb0c004750 comp rx=0 tx=0).stop 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 shutdown_connections 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 --2- 172.21.15.93:0/1302313293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c106050 0x7fbb1c1059b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 >> 172.21.15.93:0/1302313293 conn(0x7fbb1c0ffa50 msgr2=0x7fbb1c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 shutdown_connections 2023-07-21T22:08:04.234 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:03.403+0000 7fbb22766700 1 -- 172.21.15.93:0/1302313293 wait complete. 2023-07-21T22:08:04.235 INFO:teuthology.orchestra.run.smithi093.stdout:Generating new minimal ceph.conf... 2023-07-21T22:08:04.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:04 smithi093 ceph-mon[96081]: from='client.? 172.21.15.93:0/1302313293' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:08:04.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:04 smithi093 ceph-mon[96081]: from='client.? 172.21.15.93:0/1302313293' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-21T22:08:05.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.620+0000 7fd76025f700 1 Processor -- start 2023-07-21T22:08:05.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.620+0000 7fd76025f700 1 -- start start 2023-07-21T22:08:05.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd76025f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:05.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd76025f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758105e50 con 0x7fd758105500 2023-07-21T22:08:05.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75dffb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:05.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75dffb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40766/0 (socket says 172.21.15.93:40766) 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75dffb700 1 -- 172.21.15.93:0/2137708120 learned_addr learned my addr 172.21.15.93:0/2137708120 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75dffb700 1 -- 172.21.15.93:0/2137708120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758105f90 con 0x7fd758105500 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75dffb700 1 --2- 172.21.15.93:0/2137708120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd7480098a0 tx=0x7fd748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1a9059618ee93f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75cff9700 1 -- 172.21.15.93:0/2137708120 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd748004160 con 0x7fd758105500 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75cff9700 1 -- 172.21.15.93:0/2137708120 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7480042c0 con 0x7fd758105500 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.621+0000 7fd75cff9700 1 -- 172.21.15.93:0/2137708120 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd748011a00 con 0x7fd758105500 2023-07-21T22:08:05.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 -- 172.21.15.93:0/2137708120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 msgr2=0x7fd758105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 --2- 172.21.15.93:0/2137708120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd7480098a0 tx=0x7fd748009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 -- 172.21.15.93:0/2137708120 shutdown_connections 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 --2- 172.21.15.93:0/2137708120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd758105910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 -- 172.21.15.93:0/2137708120 >> 172.21.15.93:0/2137708120 conn(0x7fd758100b70 msgr2=0x7fd758102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 -- 172.21.15.93:0/2137708120 shutdown_connections 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.622+0000 7fd76025f700 1 -- 172.21.15.93:0/2137708120 wait complete. 2023-07-21T22:08:05.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd76025f700 1 Processor -- start 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd76025f700 1 -- start start 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd76025f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd76025f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75807b030 con 0x7fd758105500 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd75dffb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd75dffb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40768/0 (socket says 172.21.15.93:40768) 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd75dffb700 1 -- 172.21.15.93:0/2706534007 learned_addr learned my addr 172.21.15.93:0/2706534007 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd75dffb700 1 -- 172.21.15.93:0/2706534007 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd748009550 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd75dffb700 1 --2- 172.21.15.93:0/2706534007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd748009fb0 tx=0x7fd748011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd748003a40 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd748003ba0 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758079140 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.623+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd74801a750 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.624+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758079560 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.624+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd74801acd0 con 0x7fd758105500 2023-07-21T22:08:05.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.624+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd748024430 con 0x7fd758105500 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.625+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758061b40 con 0x7fd758105500 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.627+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd748024b60 con 0x7fd758105500 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.692+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd7581073c0 con 0x7fd758105500 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.693+0000 7fd74effd700 1 -- 172.21.15.93:0/2706534007 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fd7581073c0 con 0x7fd758105500 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.693+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 msgr2=0x7fd75807aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.693+0000 7fd76025f700 1 --2- 172.21.15.93:0/2706534007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd748009fb0 tx=0x7fd748011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:05.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.694+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 shutdown_connections 2023-07-21T22:08:05.073 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.694+0000 7fd76025f700 1 --2- 172.21.15.93:0/2706534007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd758105500 0x7fd75807aaf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:05.073 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.694+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 >> 172.21.15.93:0/2706534007 conn(0x7fd758100b70 msgr2=0x7fd758102200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:05.073 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.694+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 shutdown_connections 2023-07-21T22:08:05.073 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:04.694+0000 7fd76025f700 1 -- 172.21.15.93:0/2706534007 wait complete. 2023-07-21T22:08:05.073 INFO:teuthology.orchestra.run.smithi093.stdout:Restarting the monitor... 2023-07-21T22:08:05.377 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 ceph-mon[96081]: from='client.? 172.21.15.93:0/2706534007' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:08:05.377 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 systemd[1]: Stopping Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:08:05.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96057]: 2023-07-21T22:08:05.336+0000 7fb350bba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:08:05.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96057]: 2023-07-21T22:08:05.336+0000 7fb350bba700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-21T22:08:05.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 bash[96417]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:08:06.299 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 systemd[1]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T22:08:06.299 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:05 smithi093 systemd[1]: Stopped Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:08:06.300 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 systemd[1]: Starting Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:08:06.518 INFO:teuthology.orchestra.run.smithi093.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-21T22:08:06.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 podman[96539]: 2023-07-21T22:08:06.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:08:06.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:08:06.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 bash[96539]: 5facc5ac05525cd36d0378be1970cf9196608b41229e0ac2dd5d177eca7f2f99 2023-07-21T22:08:06.578 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: pidfile_write: ignore empty --pid-file 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: load: jerasure load: lrc 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Git sha 0 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: DB SUMMARY 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: DB Session ID: W1IBWFLB5AUXZ00LPM22 2023-07-21T22:08:06.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100055 ; 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:08:06.580 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.env: 0x55b530f32720 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.info_log: 0x55b533753260 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:08:06.581 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.statistics: (nil) 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.use_fsync: 0 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:08:06.582 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.db_log_dir: 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.wal_dir: 2023-07-21T22:08:06.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:08:06.584 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.write_buffer_manager: 0x55b5336743c0 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:08:06.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.unordered_write: 0 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.row_cache: None 2023-07-21T22:08:06.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.wal_filter: None 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.wal_compression: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:08:06.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:08:06.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:08:06.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_open_files: -1 2023-07-21T22:08:06.590 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Compression algorithms supported: 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kZSTD supported: 0 2023-07-21T22:08:06.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:08:06.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.merge_operator: 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_filter: None 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:08:06.593 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b533753360) 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_type: 0 2023-07-21T22:08:06.594 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_index_type: 0 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_shortening: 1 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: checksum: 4 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: no_block_cache: 0 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache: 0x55b5337131f0 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_options: 2023-07-21T22:08:06.595 INFO:journalctl@ceph.mon.a.smithi093.stdout: capacity : 536870912 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_shard_bits : 4 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: strict_capacity_limit : 0 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_compressed: (nil) 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: persistent_cache: (nil) 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size: 4096 2023-07-21T22:08:06.596 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size_deviation: 10 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_restart_interval: 16 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_block_restart_interval: 1 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: metadata_block_size: 4096 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: partition_filters: 0 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: use_delta_encoding: 1 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: filter_policy: bloomfilter 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: whole_key_filtering: 1 2023-07-21T22:08:06.597 INFO:journalctl@ceph.mon.a.smithi093.stdout: verify_compression: 0 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: format_version: 5 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_index_compression: 1 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_align: 0 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: max_auto_readahead_size: 262144 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: prepopulate_block_cache: 0 2023-07-21T22:08:06.598 INFO:journalctl@ceph.mon.a.smithi093.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression: NoCompression 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:08:06.599 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.num_levels: 7 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:08:06.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:08:06.601 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:08:06.601 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:08:06.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:08:06.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:08:06.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:08:06.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:08:06.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:08:06.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:08:06.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.ttl: 2592000 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:08:06.609 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:08:06.610 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e92f21c2-3792-4f9c-a4dc-2b2d0a1e6883 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977286461975, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-21T22:08:06.611 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977286462972, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95167, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 93155, "index_size": 294, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13991, "raw_average_key_size": 50, "raw_value_size": 85545, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977286, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e92f21c2-3792-4f9c-a4dc-2b2d0a1e6883", "db_session_id": "W1IBWFLB5AUXZ00LPM22", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977286463042, "job": 1, "event": "recovery_finished"} 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b5337ac000 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: DB pointer 0x55b533798000 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:08:06.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** DB Stats ** 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:08:06.613 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: L0 2/0 94.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: Sum 2/0 94.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:08:06.614 INFO:journalctl@ceph.mon.a.smithi093.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:08:06.615 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative compaction: 0.00 GB write, 16.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:08:06.616 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval compaction: 0.00 GB write, 16.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache BinnedLRUCache@0x55b5337131f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: starting mon.a rank 0 at public addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] at bind addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:06.617 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???) e1 preinit fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).mds e1 new map 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).mds e1 print_map 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: e1 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: legacy client fscid: -1 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-07-21T22:08:06.618 INFO:journalctl@ceph.mon.a.smithi093.stdout: No filesystems configured 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expand map: {default=false} 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:08:06.619 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expand map: {default=info} 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:08:06.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:08:06.621 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:08:06.621 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:08:06.621 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 systemd[1]: Started Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:08:06.621 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:08:06.621 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-07-21T22:08:06.622 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: fsmap 2023-07-21T22:08:06.622 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:08:06.622 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:06 smithi093 ceph-mon[96579]: mgrmap e1: no daemons active 2023-07-21T22:08:07.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.975+0000 7f4f6df36700 1 Processor -- start 2023-07-21T22:08:07.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.975+0000 7f4f6df36700 1 -- start start 2023-07-21T22:08:07.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.975+0000 7f4f6df36700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:07.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.975+0000 7f4f6df36700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68079700 con 0x7f4f6807acc0 2023-07-21T22:08:07.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40770/0 (socket says 172.21.15.93:40770) 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f677fe700 1 -- 172.21.15.93:0/1092022292 learned_addr learned my addr 172.21.15.93:0/1092022292 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f677fe700 1 -- 172.21.15.93:0/1092022292 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68079840 con 0x7f4f6807acc0 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f677fe700 1 --2- 172.21.15.93:0/1092022292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0098a0 tx=0x7f4f5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7bd93605f52428 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f667fc700 1 -- 172.21.15.93:0/1092022292 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f5c004160 con 0x7f4f6807acc0 2023-07-21T22:08:07.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f667fc700 1 -- 172.21.15.93:0/1092022292 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f5c0042c0 con 0x7f4f6807acc0 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.976+0000 7f4f667fc700 1 -- 172.21.15.93:0/1092022292 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f5c011a00 con 0x7f4f6807acc0 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 -- 172.21.15.93:0/1092022292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 msgr2=0x7f4f680791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 --2- 172.21.15.93:0/1092022292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0098a0 tx=0x7f4f5c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 -- 172.21.15.93:0/1092022292 shutdown_connections 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 --2- 172.21.15.93:0/1092022292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f680791c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:07.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 -- 172.21.15.93:0/1092022292 >> 172.21.15.93:0/1092022292 conn(0x7f4f68100b70 msgr2=0x7f4f68102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:07.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 -- 172.21.15.93:0/1092022292 shutdown_connections 2023-07-21T22:08:07.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.977+0000 7f4f6df36700 1 -- 172.21.15.93:0/1092022292 wait complete. 2023-07-21T22:08:07.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f6df36700 1 Processor -- start 2023-07-21T22:08:07.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f6df36700 1 -- start start 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f6df36700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f6df36700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6819ba10 con 0x7f4f6807acc0 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40772/0 (socket says 172.21.15.93:40772) 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f677fe700 1 -- 172.21.15.93:0/1365393101 learned_addr learned my addr 172.21.15.93:0/1365393101 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:07.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.978+0000 7f4f677fe700 1 -- 172.21.15.93:0/1365393101 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c009550 con 0x7f4f6807acc0 2023-07-21T22:08:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f677fe700 1 --2- 172.21.15.93:0/1365393101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fb0 tx=0x7f4f5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f5c003a40 con 0x7f4f6807acc0 2023-07-21T22:08:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4f5c003ba0 con 0x7f4f6807acc0 2023-07-21T22:08:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f5c01a750 con 0x7f4f6807acc0 2023-07-21T22:08:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6819bc10 con 0x7f4f6807acc0 2023-07-21T22:08:07.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6819c030 con 0x7f4f6807acc0 2023-07-21T22:08:07.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4f5c01ac80 con 0x7f4f6807acc0 2023-07-21T22:08:07.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.979+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f5c024430 con 0x7f4f6807acc0 2023-07-21T22:08:07.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.980+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f68061b40 con 0x7f4f6807acc0 2023-07-21T22:08:07.475 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:06.982+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4f5c024b60 con 0x7f4f6807acc0 2023-07-21T22:08:07.475 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.047+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4f6819ea70 con 0x7f4f6807acc0 2023-07-21T22:08:07.475 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f64ff9700 1 -- 172.21.15.93:0/1365393101 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4f5c024d00 con 0x7f4f6807acc0 2023-07-21T22:08:07.475 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 msgr2=0x7f4f6819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:07.476 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f6df36700 1 --2- 172.21.15.93:0/1365393101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009fb0 tx=0x7f4f5c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:07.476 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 shutdown_connections 2023-07-21T22:08:07.476 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f6df36700 1 --2- 172.21.15.93:0/1365393101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f6807acc0 0x7f4f6819b4d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:07.476 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.050+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 >> 172.21.15.93:0/1365393101 conn(0x7f4f68100b70 msgr2=0x7f4f68102240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:07.477 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.051+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 shutdown_connections 2023-07-21T22:08:07.477 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:07.051+0000 7f4f6df36700 1 -- 172.21.15.93:0/1365393101 wait complete. 2023-07-21T22:08:07.477 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-21T22:08:07.479 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:08:07.479 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mgr... 2023-07-21T22:08:07.479 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 9283 ... 2023-07-21T22:08:07.480 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 8765 ... 2023-07-21T22:08:07.673 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:08:07.674 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-21T22:08:07.761 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c.target.wants/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:08:07.768 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:07 smithi093 systemd[1]: Starting Ceph mgr.x for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:08:08.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1365393101' entity='client.admin' 2023-07-21T22:08:08.321 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 podman[96812]: 2023-07-21T22:08:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:08:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-07-21T22:08:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:08:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:08:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-07-21T22:08:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:08:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:08:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-07-21T22:08:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:08:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:08:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-07-21T22:08:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-21T22:08:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr to start... 2023-07-21T22:08:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr... 2023-07-21T22:08:08.620 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 bash[96812]: 9bfff3ca84f6f133a0a9011e370578b10a0d5b8289baea8bf798334ec7f9b465 2023-07-21T22:08:08.620 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 systemd[1]: Started Ceph mgr.x for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:08:08.620 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:08.417+0000 7fc11a0cd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:08:08.620 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:08.496+0000 7fc11a0cd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:08.906 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:08.589+0000 7fc11a0cd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:08:08.906 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:08.688+0000 7fc11a0cd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:08:08.906 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:08.865+0000 7fc11a0cd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:08:09.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4172203586' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:08:09.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:09.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:09.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:08:09.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:08:09.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:08:09.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:08:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:08:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:08:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:08:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.355 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:08:09.356 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.357 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:09.358 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:08:02.004797+0000", 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:08:09.359 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.885+0000 7ffaebc01700 1 Processor -- start 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffaebc01700 1 -- start start 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffaebc01700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffaebc01700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4104d50 con 0x7ffae4104400 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffae999d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffae999d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40778/0 (socket says 172.21.15.93:40778) 2023-07-21T22:08:09.360 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.886+0000 7ffae999d700 1 -- 172.21.15.93:0/2599690384 learned_addr learned my addr 172.21.15.93:0/2599690384 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffae999d700 1 -- 172.21.15.93:0/2599690384 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae4104e90 con 0x7ffae4104400 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffae999d700 1 --2- 172.21.15.93:0/2599690384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffae00098a0 tx=0x7ffae0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916d2f7a68ddc9a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffae899b700 1 -- 172.21.15.93:0/2599690384 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae0004160 con 0x7ffae4104400 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffae899b700 1 -- 172.21.15.93:0/2599690384 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffae00042c0 con 0x7ffae4104400 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffae899b700 1 -- 172.21.15.93:0/2599690384 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae0011a00 con 0x7ffae4104400 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffaebc01700 1 -- 172.21.15.93:0/2599690384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 msgr2=0x7ffae4104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.887+0000 7ffaebc01700 1 --2- 172.21.15.93:0/2599690384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffae00098a0 tx=0x7ffae0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:09.361 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.888+0000 7ffaebc01700 1 -- 172.21.15.93:0/2599690384 shutdown_connections 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.888+0000 7ffaebc01700 1 --2- 172.21.15.93:0/2599690384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae4104810 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.888+0000 7ffaebc01700 1 -- 172.21.15.93:0/2599690384 >> 172.21.15.93:0/2599690384 conn(0x7ffae40ffa50 msgr2=0x7ffae4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.888+0000 7ffaebc01700 1 -- 172.21.15.93:0/2599690384 shutdown_connections 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.888+0000 7ffaebc01700 1 -- 172.21.15.93:0/2599690384 wait complete. 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffaebc01700 1 Processor -- start 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffaebc01700 1 -- start start 2023-07-21T22:08:09.362 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffaebc01700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffaebc01700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae419bfc0 con 0x7ffae4104400 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffae999d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffae999d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40780/0 (socket says 172.21.15.93:40780) 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffae999d700 1 -- 172.21.15.93:0/4172203586 learned_addr learned my addr 172.21.15.93:0/4172203586 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffae999d700 1 -- 172.21.15.93:0/4172203586 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0009550 con 0x7ffae4104400 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.889+0000 7ffae999d700 1 --2- 172.21.15.93:0/4172203586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009130 tx=0x7ffae0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae0003b70 con 0x7ffae4104400 2023-07-21T22:08:09.363 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffae0003cd0 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae419c100 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffae00197d0 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae419c520 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffae00199c0 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.890+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffae000b720 con 0x7ffae4104400 2023-07-21T22:08:09.364 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.891+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4108890 con 0x7ffae4104400 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.893+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ffae002a060 con 0x7ffae4104400 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.964+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffae41062c0 con 0x7ffae4104400 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.965+0000 7ffadaffd700 1 -- 172.21.15.93:0/4172203586 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ffae0019e00 con 0x7ffae4104400 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 msgr2=0x7ffae40739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 --2- 172.21.15.93:0/4172203586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009130 tx=0x7ffae0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 shutdown_connections 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 --2- 172.21.15.93:0/4172203586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffae4104400 0x7ffae40739f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:09.365 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 >> 172.21.15.93:0/4172203586 conn(0x7ffae40ffa50 msgr2=0x7ffae4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:09.366 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 shutdown_connections 2023-07-21T22:08:09.366 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:08.966+0000 7ffaebc01700 1 -- 172.21.15.93:0/4172203586 wait complete. 2023-07-21T22:08:09.366 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (1/15)... 2023-07-21T22:08:09.542 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:09.187+0000 7fc11a0cd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:08:09.542 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:09.276+0000 7fc11a0cd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:08:09.542 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:09.408+0000 7fc11a0cd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:08:09.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:09.501+0000 7fc11a0cd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:08:10.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:09.914+0000 7fc11a0cd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:08:10.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:10.479+0000 7fc11a0cd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:08:10.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:10.646+0000 7fc11a0cd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:08:11.717 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:11.409+0000 7fc11a0cd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:11.717 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:11.502+0000 7fc11a0cd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:08:11.991 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:11.677+0000 7fc11a0cd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:12.244 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:11.964+0000 7fc11a0cd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:08:12.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:12.298 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:12.298 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:08:12.298 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:08:12.299 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:08:12.299 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:08:12.299 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:08:12.299 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:08:12.300 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:08:12.301 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.302 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:08:12.302 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:12.302 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:08:12.302 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:08:12.302 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:08:12.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:08:12.314 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:08:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:08:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:08:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:08:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:08:12.316 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:08:12.317 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:08:12.318 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:08:02.004797+0000", 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:12.319 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:08:12.320 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:12.320 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.843+0000 7f423e254700 1 Processor -- start 2023-07-21T22:08:12.320 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.843+0000 7f423e254700 1 -- start start 2023-07-21T22:08:12.320 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.843+0000 7f423e254700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:12.320 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.843+0000 7f423e254700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238105e90 con 0x7f4238073ba0 2023-07-21T22:08:12.321 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:12.321 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40782/0 (socket says 172.21.15.93:40782) 2023-07-21T22:08:12.321 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42377fe700 1 -- 172.21.15.93:0/3405044493 learned_addr learned my addr 172.21.15.93:0/3405044493 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:12.321 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42377fe700 1 -- 172.21.15.93:0/3405044493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423810a9e0 con 0x7f4238073ba0 2023-07-21T22:08:12.322 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42377fe700 1 --2- 172.21.15.93:0/3405044493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42280098a0 tx=0x7f4228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f50536fa816ebb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:12.322 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42367fc700 1 -- 172.21.15.93:0/3405044493 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228004160 con 0x7f4238073ba0 2023-07-21T22:08:12.322 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42367fc700 1 -- 172.21.15.93:0/3405044493 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42280042c0 con 0x7f4238073ba0 2023-07-21T22:08:12.322 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.844+0000 7f42367fc700 1 -- 172.21.15.93:0/3405044493 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228011a00 con 0x7f4238073ba0 2023-07-21T22:08:12.323 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 -- 172.21.15.93:0/3405044493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 msgr2=0x7f423810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:12.323 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 --2- 172.21.15.93:0/3405044493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42280098a0 tx=0x7f4228009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:12.323 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 -- 172.21.15.93:0/3405044493 shutdown_connections 2023-07-21T22:08:12.323 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 --2- 172.21.15.93:0/3405044493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4238073ba0 0x7f423810a4a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:12.324 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 -- 172.21.15.93:0/3405044493 >> 172.21.15.93:0/3405044493 conn(0x7f42380ffa50 msgr2=0x7f4238101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:12.324 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 -- 172.21.15.93:0/3405044493 shutdown_connections 2023-07-21T22:08:12.324 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.845+0000 7f423e254700 1 -- 172.21.15.93:0/3405044493 wait complete. 2023-07-21T22:08:12.324 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f423e254700 1 Processor -- start 2023-07-21T22:08:12.325 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f423e254700 1 -- start start 2023-07-21T22:08:12.325 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f423e254700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:12.325 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f423e254700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423819bdd0 con 0x7f423819b480 2023-07-21T22:08:12.325 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f42377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:12.326 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f42377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40784/0 (socket says 172.21.15.93:40784) 2023-07-21T22:08:12.326 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f42377fe700 1 -- 172.21.15.93:0/4198056938 learned_addr learned my addr 172.21.15.93:0/4198056938 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:12.326 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.846+0000 7f42377fe700 1 -- 172.21.15.93:0/4198056938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228009550 con 0x7f423819b480 2023-07-21T22:08:12.326 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f42377fe700 1 --2- 172.21.15.93:0/4198056938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4228009130 tx=0x7f4228011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:12.327 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228003ab0 con 0x7f423819b480 2023-07-21T22:08:12.327 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4228003c10 con 0x7f423819b480 2023-07-21T22:08:12.327 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423819bfd0 con 0x7f423819b480 2023-07-21T22:08:12.327 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42280247d0 con 0x7f423819b480 2023-07-21T22:08:12.328 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423819ec30 con 0x7f423819b480 2023-07-21T22:08:12.328 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4228024d50 con 0x7f423819b480 2023-07-21T22:08:12.328 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.847+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42280193c0 con 0x7f423819b480 2023-07-21T22:08:12.329 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.848+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4238061b40 con 0x7f423819b480 2023-07-21T22:08:12.329 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.850+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f422801abb0 con 0x7f423819b480 2023-07-21T22:08:12.329 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.921+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4238073df0 con 0x7f423819b480 2023-07-21T22:08:12.329 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.921+0000 7f4234ff9700 1 -- 172.21.15.93:0/4198056938 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f422801ad50 con 0x7f423819b480 2023-07-21T22:08:12.330 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 msgr2=0x7f423819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:12.330 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 --2- 172.21.15.93:0/4198056938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4228009130 tx=0x7f4228011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:12.330 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 shutdown_connections 2023-07-21T22:08:12.330 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 --2- 172.21.15.93:0/4198056938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f423819b480 0x7f423819b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:12.331 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 >> 172.21.15.93:0/4198056938 conn(0x7f42380ffa50 msgr2=0x7f4238101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:12.331 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 shutdown_connections 2023-07-21T22:08:12.331 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:11.922+0000 7f423e254700 1 -- 172.21.15.93:0/4198056938 wait complete. 2023-07-21T22:08:12.331 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (2/15)... 2023-07-21T22:08:12.511 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:12.204+0000 7fc11a0cd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:08:12.511 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:12.353+0000 7fc11a0cd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:08:12.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:12 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4198056938' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:08:12.944 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:12.614+0000 7fc11a0cd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:08:13.658 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:13.301+0000 7fc11a0cd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:08:13.658 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:13.431+0000 7fc11a0cd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:08:13.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:13.618+0000 7fc11a0cd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:08:13.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:13.731+0000 7fc11a0cd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:14.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:13.942+0000 7fc11a0cd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:08:15.188 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:14.822+0000 7fc11a0cd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:08:15.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3323978903' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: Activating manager daemon x 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: mgrmap e2: x(active, starting, since 0.00308541s) 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:08:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:08:15.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: Manager daemon x is now available 2023-07-21T22:08:15.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:14 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:08:15.263 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:15.263 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:15.263 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:08:15.263 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:08:15.263 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:08:15.264 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:08:15.264 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:08:15.264 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.264 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-07-21T22:08:15.265 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:08:15.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:15.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:08:15.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:08:15.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.266 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:08:15.267 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:15.267 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:08:15.267 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:08:15.267 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:08:15.267 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:08:15.268 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:08:15.268 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:08:15.268 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.268 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:08:15.268 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:08:15.269 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.270 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:15.271 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:08:02.004797+0000", 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:15.272 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.741+0000 7fbc239ae700 1 Processor -- start 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.741+0000 7fbc239ae700 1 -- start start 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.741+0000 7fbc239ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.741+0000 7fbc239ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c073be0 con 0x7fbc1c103e40 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc2174a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:15.273 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc2174a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40786/0 (socket says 172.21.15.93:40786) 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc2174a700 1 -- 172.21.15.93:0/2611787400 learned_addr learned my addr 172.21.15.93:0/2611787400 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc2174a700 1 -- 172.21.15.93:0/2611787400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c073d20 con 0x7fbc1c103e40 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc2174a700 1 --2- 172.21.15.93:0/2611787400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098a0 tx=0x7fbc18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9191c420ab81b3c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc13fff700 1 -- 172.21.15.93:0/2611787400 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc18004160 con 0x7fbc1c103e40 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc13fff700 1 -- 172.21.15.93:0/2611787400 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc180042c0 con 0x7fbc1c103e40 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.742+0000 7fbc13fff700 1 -- 172.21.15.93:0/2611787400 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc18011a00 con 0x7fbc1c103e40 2023-07-21T22:08:15.274 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 -- 172.21.15.93:0/2611787400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 msgr2=0x7fbc1c104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 --2- 172.21.15.93:0/2611787400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc180098a0 tx=0x7fbc18009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 -- 172.21.15.93:0/2611787400 shutdown_connections 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 --2- 172.21.15.93:0/2611787400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c104250 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 -- 172.21.15.93:0/2611787400 >> 172.21.15.93:0/2611787400 conn(0x7fbc1c0ffa50 msgr2=0x7fbc1c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 -- 172.21.15.93:0/2611787400 shutdown_connections 2023-07-21T22:08:15.275 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.743+0000 7fbc239ae700 1 -- 172.21.15.93:0/2611787400 wait complete. 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc239ae700 1 Processor -- start 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc239ae700 1 -- start start 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc239ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc239ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c19b990 con 0x7fbc1c103e40 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc2174a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc2174a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40788/0 (socket says 172.21.15.93:40788) 2023-07-21T22:08:15.276 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc2174a700 1 -- 172.21.15.93:0/3323978903 learned_addr learned my addr 172.21.15.93:0/3323978903 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.744+0000 7fbc2174a700 1 -- 172.21.15.93:0/3323978903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18009550 con 0x7fbc1c103e40 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc2174a700 1 --2- 172.21.15.93:0/3323978903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009fb0 tx=0x7fbc18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc18003a40 con 0x7fbc1c103e40 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc18003ba0 con 0x7fbc1c103e40 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c19bbf0 con 0x7fbc1c103e40 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc1801a750 con 0x7fbc1c103e40 2023-07-21T22:08:15.277 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c19c070 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbc1801acd0 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.745+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbc1802de40 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.746+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c194c90 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.748+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbc18024b60 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.820+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbc1c02cac0 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.821+0000 7fbc127fc700 1 -- 172.21.15.93:0/3323978903 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbc18024d00 con 0x7fbc1c103e40 2023-07-21T22:08:15.278 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 msgr2=0x7fbc1c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 --2- 172.21.15.93:0/3323978903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009fb0 tx=0x7fbc18011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 shutdown_connections 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 --2- 172.21.15.93:0/3323978903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc1c103e40 0x7fbc1c19b450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 >> 172.21.15.93:0/3323978903 conn(0x7fbc1c0ffa50 msgr2=0x7fbc1c18fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 shutdown_connections 2023-07-21T22:08:15.279 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:14.822+0000 7fbc239ae700 1 -- 172.21.15.93:0/3323978903 wait complete. 2023-07-21T22:08:15.280 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (3/15)... 2023-07-21T22:08:16.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:15 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:08:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:15 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' 2023-07-21T22:08:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:15 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' 2023-07-21T22:08:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:15 smithi093 ceph-mon[96579]: from='mgr.14100 172.21.15.93:0/3206165379' entity='mgr.x' 2023-07-21T22:08:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:15 smithi093 ceph-mon[96579]: mgrmap e3: x(active, since 1.00719s) 2023-07-21T22:08:18.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:18.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:18.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:08:18.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:08:18.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:18.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:08:18.415 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:08:18.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:08:18.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:08:18.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:08:18.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:08:18.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:08:18.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:08:18.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:08:18.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:08:18.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:08:18.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:08:18.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:08:18.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:08:02.004797+0000", 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1fd33700 1 Processor -- start 2023-07-21T22:08:18.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1fd33700 1 -- start start 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1fd33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1fd33700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18105240 con 0x7f0c181048f0 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1dacf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1dacf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40814/0 (socket says 172.21.15.93:40814) 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.735+0000 7f0c1dacf700 1 -- 172.21.15.93:0/3270627615 learned_addr learned my addr 172.21.15.93:0/3270627615 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:18.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1dacf700 1 -- 172.21.15.93:0/3270627615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18105380 con 0x7f0c181048f0 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1dacf700 1 --2- 172.21.15.93:0/3270627615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c080098a0 tx=0x7f0c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6461d70716a428 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1cacd700 1 -- 172.21.15.93:0/3270627615 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c08004160 con 0x7f0c181048f0 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1cacd700 1 -- 172.21.15.93:0/3270627615 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c080042c0 con 0x7f0c181048f0 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1cacd700 1 -- 172.21.15.93:0/3270627615 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c08011a00 con 0x7f0c181048f0 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1fd33700 1 -- 172.21.15.93:0/3270627615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 msgr2=0x7f0c18104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:18.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.736+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/3270627615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c080098a0 tx=0x7f0c08009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- 172.21.15.93:0/3270627615 shutdown_connections 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/3270627615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18104d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- 172.21.15.93:0/3270627615 >> 172.21.15.93:0/3270627615 conn(0x7f0c180ff1c0 msgr2=0x7f0c181015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- 172.21.15.93:0/3270627615 shutdown_connections 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- 172.21.15.93:0/3270627615 wait complete. 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 Processor -- start 2023-07-21T22:08:18.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- start start 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.737+0000 7f0c1fd33700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181997c0 con 0x7f0c181048f0 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1dacf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1dacf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40816/0 (socket says 172.21.15.93:40816) 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1dacf700 1 -- 172.21.15.93:0/1827290340 learned_addr learned my addr 172.21.15.93:0/1827290340 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1dacf700 1 -- 172.21.15.93:0/1827290340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009550 con 0x7f0c181048f0 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1dacf700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fb0 tx=0x7f0c08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:18.434 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c08003a40 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c181999c0 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c08003ba0 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c0801a750 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.738+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18199de0 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.739+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f0c08024430 con 0x7f0c181048f0 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.739+0000 7f0c0effd700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 0x7f0c0403ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:18.435 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.739+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18061b40 con 0x7f0c181048f0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.739+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0c0804c2d0 con 0x7f0c181048f0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.739+0000 7f0c1d2ce700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 0x7f0c0403ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.740+0000 7f0c1d2ce700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 0x7f0c0403ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:17.744+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0c08018350 con 0x7f0c181048f0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.002+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0c181088b0 con 0x7f0c181048f0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.003+0000 7f0c0effd700 1 -- 172.21.15.93:0/1827290340 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0c08018350 con 0x7f0c181048f0 2023-07-21T22:08:18.436 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 msgr2=0x7f0c0403ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 0x7f0c0403ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 msgr2=0x7f0c18199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009fb0 tx=0x7f0c08011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 shutdown_connections 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f0c04038670 0x7f0c0403ab20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 --2- 172.21.15.93:0/1827290340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c181048f0 0x7f0c18199280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 >> 172.21.15.93:0/1827290340 conn(0x7f0c180ff1c0 msgr2=0x7f0c18103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:18.437 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 shutdown_connections 2023-07-21T22:08:18.438 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.005+0000 7f0c1fd33700 1 -- 172.21.15.93:0/1827290340 wait complete. 2023-07-21T22:08:18.438 INFO:teuthology.orchestra.run.smithi093.stdout:mgr is available 2023-07-21T22:08:18.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:18 smithi093 ceph-mon[96579]: mgrmap e4: x(active, since 2s) 2023-07-21T22:08:18.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:18 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1827290340' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:08:19.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:19 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2859915394' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:08:20.063 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:20.063 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:20.064 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:08:20.065 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33af310700 1 Processor -- start 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33af310700 1 -- start start 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33af310700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33af310700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8073be0 con 0x7f33a8103e40 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33ad0ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33ad0ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40820/0 (socket says 172.21.15.93:40820) 2023-07-21T22:08:20.066 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33ad0ac700 1 -- 172.21.15.93:0/2810653437 learned_addr learned my addr 172.21.15.93:0/2810653437 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.802+0000 7f33ad0ac700 1 -- 172.21.15.93:0/2810653437 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8073d20 con 0x7f33a8103e40 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33ad0ac700 1 --2- 172.21.15.93:0/2810653437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33980098a0 tx=0x7f3398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e90799de436d73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f339ffff700 1 -- 172.21.15.93:0/2810653437 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398004160 con 0x7f33a8103e40 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f339ffff700 1 -- 172.21.15.93:0/2810653437 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33980042c0 con 0x7f33a8103e40 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f339ffff700 1 -- 172.21.15.93:0/2810653437 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398011a00 con 0x7f33a8103e40 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 -- 172.21.15.93:0/2810653437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 msgr2=0x7f33a8104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:20.067 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 --2- 172.21.15.93:0/2810653437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33980098a0 tx=0x7f3398009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 -- 172.21.15.93:0/2810653437 shutdown_connections 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 --2- 172.21.15.93:0/2810653437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a8104250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 -- 172.21.15.93:0/2810653437 >> 172.21.15.93:0/2810653437 conn(0x7f33a80ffa50 msgr2=0x7f33a8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 -- 172.21.15.93:0/2810653437 shutdown_connections 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.803+0000 7f33af310700 1 -- 172.21.15.93:0/2810653437 wait complete. 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33af310700 1 Processor -- start 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33af310700 1 -- start start 2023-07-21T22:08:20.068 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33af310700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33af310700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819b990 con 0x7f33a8103e40 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33ad0ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33ad0ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40822/0 (socket says 172.21.15.93:40822) 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33ad0ac700 1 -- 172.21.15.93:0/2859915394 learned_addr learned my addr 172.21.15.93:0/2859915394 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.804+0000 7f33ad0ac700 1 -- 172.21.15.93:0/2859915394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398009550 con 0x7f33a8103e40 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f33ad0ac700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3398009fb0 tx=0x7f3398011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:20.069 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3398003a40 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3398003ba0 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a819bbf0 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f339801a750 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.805+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a819c070 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f339801a980 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8194c90 con 0x7f33a8103e40 2023-07-21T22:08:20.070 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f339e7fc700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 0x7f339403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f339804c4a0 con 0x7f33a8103e40 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f33ac8ab700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 0x7f339403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.806+0000 7f33ac8ab700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 0x7f339403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006fd0 tx=0x7f33a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.810+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f339801ac30 con 0x7f33a8103e40 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.995+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f33a802bea0 con 0x7f33a8103e40 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.996+0000 7f339e7fc700 1 -- 172.21.15.93:0/2859915394 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f3398018350 con 0x7f33a8103e40 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 msgr2=0x7f339403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:20.071 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 0x7f339403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006fd0 tx=0x7f33a4006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 msgr2=0x7f33a819b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3398009fb0 tx=0x7f3398011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 shutdown_connections 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f33940386c0 0x7f339403ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 --2- 172.21.15.93:0/2859915394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33a8103e40 0x7f33a819b450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 >> 172.21.15.93:0/2859915394 conn(0x7f33a80ffa50 msgr2=0x7f33a818fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:20.072 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 shutdown_connections 2023-07-21T22:08:20.073 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:18.998+0000 7f33af310700 1 -- 172.21.15.93:0/2859915394 wait complete. 2023-07-21T22:08:20.073 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling cephadm module... 2023-07-21T22:08:22.844 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1501143705' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-21T22:08:22.844 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setuser ceph since I am not root 2023-07-21T22:08:22.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setgroup ceph since I am not root 2023-07-21T22:08:22.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:22.637+0000 7fbac6d2b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:08:22.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:22.716+0000 7fbac6d2b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:22.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:22.804+0000 7fbac6d2b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:08:23.107 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:22.895+0000 7fbac6d2b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:08:23.410 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:23.067+0000 7fbac6d2b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:08:23.687 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1501143705' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-21T22:08:23.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-mon[96579]: mgrmap e5: x(active, since 7s) 2023-07-21T22:08:23.688 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:23.370+0000 7fbac6d2b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:08:23.688 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:23.460+0000 7fbac6d2b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:08:23.688 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:23.589+0000 7fbac6d2b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:08:24.136 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:23.683+0000 7fbac6d2b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:08:24.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:24.096+0000 7fbac6d2b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:08:25.187 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:24.651+0000 7fbac6d2b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:08:25.187 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:24.819+0000 7fbac6d2b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:08:25.422 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d9f5f700 1 Processor -- start 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d9f5f700 1 -- start start 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d9f5f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d9f5f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4073be0 con 0x7f55d4106040 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40826/0 (socket says 172.21.15.93:40826) 2023-07-21T22:08:25.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.251+0000 7f55d37fe700 1 -- 172.21.15.93:0/3173989723 learned_addr learned my addr 172.21.15.93:0/3173989723 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d37fe700 1 -- 172.21.15.93:0/3173989723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4073d20 con 0x7f55d4106040 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d37fe700 1 --2- 172.21.15.93:0/3173989723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55c40098a0 tx=0x7f55c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8be81e36391654 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d27fc700 1 -- 172.21.15.93:0/3173989723 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55c4004160 con 0x7f55d4106040 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d27fc700 1 -- 172.21.15.93:0/3173989723 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f55c40042c0 con 0x7f55d4106040 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d27fc700 1 -- 172.21.15.93:0/3173989723 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55c4011a00 con 0x7f55d4106040 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d9f5f700 1 -- 172.21.15.93:0/3173989723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 msgr2=0x7f55d4108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:25.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.252+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/3173989723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55c40098a0 tx=0x7f55c4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.253+0000 7f55d9f5f700 1 -- 172.21.15.93:0/3173989723 shutdown_connections 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.253+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/3173989723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d4108420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.253+0000 7f55d9f5f700 1 -- 172.21.15.93:0/3173989723 >> 172.21.15.93:0/3173989723 conn(0x7f55d40ffa40 msgr2=0x7f55d4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.253+0000 7f55d9f5f700 1 -- 172.21.15.93:0/3173989723 shutdown_connections 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.253+0000 7f55d9f5f700 1 -- 172.21.15.93:0/3173989723 wait complete. 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.254+0000 7f55d9f5f700 1 Processor -- start 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.254+0000 7f55d9f5f700 1 -- start start 2023-07-21T22:08:25.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.254+0000 7f55d9f5f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d9f5f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4199710 con 0x7f55d4106040 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40828/0 (socket says 172.21.15.93:40828) 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d37fe700 1 -- 172.21.15.93:0/1501143705 learned_addr learned my addr 172.21.15.93:0/1501143705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d37fe700 1 -- 172.21.15.93:0/1501143705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009550 con 0x7f55d4106040 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d37fe700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fb0 tx=0x7f55c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:25.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55c4003930 con 0x7f55d4106040 2023-07-21T22:08:25.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f55c4003a90 con 0x7f55d4106040 2023-07-21T22:08:25.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d4199970 con 0x7f55d4106040 2023-07-21T22:08:25.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55c40195e0 con 0x7f55d4106040 2023-07-21T22:08:25.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.255+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d4199df0 con 0x7f55d4106040 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.256+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d4109560 con 0x7f55d4106040 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.257+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f55c401a9d0 con 0x7f55d4106040 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.257+0000 7f55d0ff9700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 0x7f55bc03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.257+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f55c404c800 con 0x7f55d4106040 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.257+0000 7f55d2ffd700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 0x7f55bc03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.257+0000 7f55d2ffd700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 0x7f55bc03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:25.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.261+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55c400bbd0 con 0x7f55d4106040 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:21.501+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f55d419cb90 con 0x7f55d4106040 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.476+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f55c404b980 con 0x7f55d4106040 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.478+0000 7f55d0ff9700 1 -- 172.21.15.93:0/1501143705 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f55c404bd00 con 0x7f55d4106040 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 msgr2=0x7f55bc03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 0x7f55bc03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 msgr2=0x7f55d41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:25.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fb0 tx=0x7f55c4011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 shutdown_connections 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f55bc0386c0 0x7f55bc03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 --2- 172.21.15.93:0/1501143705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55d4106040 0x7f55d41991d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 >> 172.21.15.93:0/1501143705 conn(0x7f55d40ffa40 msgr2=0x7f55d41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 shutdown_connections 2023-07-21T22:08:25.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:22.481+0000 7f55d9f5f700 1 -- 172.21.15.93:0/1501143705 wait complete. 2023-07-21T22:08:25.687 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:25.588+0000 7fbac6d2b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:26.162 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:25.678+0000 7fbac6d2b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:08:26.162 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:25.846+0000 7fbac6d2b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:26.163 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:26.121+0000 7fbac6d2b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:08:26.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:26.346+0000 7fbac6d2b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:08:26.798 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:26.492+0000 7fbac6d2b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:08:27.187 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:26.758+0000 7fbac6d2b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:08:27.844 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:27 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/145743759' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:08:27.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:27.479+0000 7fbac6d2b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:08:27.845 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:27.614+0000 7fbac6d2b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:08:28.178 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:27.804+0000 7fbac6d2b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:08:28.178 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:27.921+0000 7fbac6d2b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:28.178 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:28.137+0000 7fbac6d2b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:08:28.579 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:28.579 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-21T22:08:28.579 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:08:28.579 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:08:28.579 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.293+0000 7f301f678700 1 Processor -- start 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.293+0000 7f301f678700 1 -- start start 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301f678700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301f678700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3018073bc0 con 0x7f3018106070 2023-07-21T22:08:28.580 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301d414700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301d414700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40846/0 (socket says 172.21.15.93:40846) 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301d414700 1 -- 172.21.15.93:0/26300562 learned_addr learned my addr 172.21.15.93:0/26300562 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301d414700 1 -- 172.21.15.93:0/26300562 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3018073d00 con 0x7f3018106070 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f301d414700 1 --2- 172.21.15.93:0/26300562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30140098a0 tx=0x7f3014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daffea7542f75798 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f300ffff700 1 -- 172.21.15.93:0/26300562 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014004160 con 0x7f3018106070 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f300ffff700 1 -- 172.21.15.93:0/26300562 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30140042c0 con 0x7f3018106070 2023-07-21T22:08:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.294+0000 7f300ffff700 1 -- 172.21.15.93:0/26300562 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014011a00 con 0x7f3018106070 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 -- 172.21.15.93:0/26300562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 msgr2=0x7f3018108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 --2- 172.21.15.93:0/26300562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30140098a0 tx=0x7f3014009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 -- 172.21.15.93:0/26300562 shutdown_connections 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 --2- 172.21.15.93:0/26300562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f3018108450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 -- 172.21.15.93:0/26300562 >> 172.21.15.93:0/26300562 conn(0x7f30180ffa50 msgr2=0x7f3018101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 -- 172.21.15.93:0/26300562 shutdown_connections 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.295+0000 7f301f678700 1 -- 172.21.15.93:0/26300562 wait complete. 2023-07-21T22:08:28.582 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301f678700 1 Processor -- start 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301f678700 1 -- start start 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301f678700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301f678700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30181997f0 con 0x7f3018106070 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301d414700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301d414700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40848/0 (socket says 172.21.15.93:40848) 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301d414700 1 -- 172.21.15.93:0/145743759 learned_addr learned my addr 172.21.15.93:0/145743759 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:28.583 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301d414700 1 -- 172.21.15.93:0/145743759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3014009550 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f301d414700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fb0 tx=0x7f3014011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.296+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3014003a40 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.297+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3014003ba0 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.297+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f301401a750 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.297+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30181999f0 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.297+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3018199e10 con 0x7f3018106070 2023-07-21T22:08:28.584 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.297+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f301401a980 con 0x7f3018106070 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.298+0000 7f300e7fc700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 0x7f300403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.298+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3018061b40 con 0x7f3018106070 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.298+0000 7f301cc13700 1 -- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 msgr2=0x7f300403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2048241181 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.298+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f301404c320 con 0x7f3018106070 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.298+0000 7f301cc13700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 0x7f300403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:08:28.585 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.302+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3014018db0 con 0x7f3018106070 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.499+0000 7f301cc13700 1 -- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 msgr2=0x7f300403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2048241181 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.499+0000 7f301cc13700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 0x7f300403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.550+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f301819cf30 con 0x7f3018106070 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.551+0000 7f300e7fc700 1 -- 172.21.15.93:0/145743759 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f301400b3e0 con 0x7f3018106070 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 msgr2=0x7f300403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 0x7f300403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.586 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 msgr2=0x7f30181992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fb0 tx=0x7f3014011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 shutdown_connections 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:6800/2048241181,v1:172.21.15.93:6801/2048241181] conn(0x7f3004038710 0x7f300403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 --2- 172.21.15.93:0/145743759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3018106070 0x7f30181992b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 >> 172.21.15.93:0/145743759 conn(0x7f30180ffa50 msgr2=0x7f3018101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 shutdown_connections 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:27.553+0000 7f301f678700 1 -- 172.21.15.93:0/145743759 wait complete. 2023-07-21T22:08:28.587 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-07-21T22:08:28.588 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 5... 2023-07-21T22:08:29.118 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:29.013+0000 7fbac6d2b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:08:29.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: Active manager daemon x restarted 2023-07-21T22:08:29.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: Activating manager daemon x 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: osdmap e2: 0 total, 0 up, 0 in 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: mgrmap e6: x(active, starting, since 0.00358932s) 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:08:29.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:08:29.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: Manager daemon x is now available 2023-07-21T22:08:29.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:29.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:29.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:29.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:29 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:30.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.034+0000 7fe483937700 1 Processor -- start 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.034+0000 7fe483937700 1 -- start start 2023-07-21T22:08:30.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.034+0000 7fe483937700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.034+0000 7fe483937700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c105240 con 0x7fe47c1048f0 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.034+0000 7fe4816d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe4816d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40860/0 (socket says 172.21.15.93:40860) 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe4816d3700 1 -- 172.21.15.93:0/2361164165 learned_addr learned my addr 172.21.15.93:0/2361164165 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe4816d3700 1 -- 172.21.15.93:0/2361164165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c105380 con 0x7fe47c1048f0 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe4816d3700 1 --2- 172.21.15.93:0/2361164165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf810bb9046d2c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:30.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe473fff700 1 -- 172.21.15.93:0/2361164165 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe478004160 con 0x7fe47c1048f0 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe473fff700 1 -- 172.21.15.93:0/2361164165 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4780042c0 con 0x7fe47c1048f0 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.035+0000 7fe473fff700 1 -- 172.21.15.93:0/2361164165 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe478011a00 con 0x7fe47c1048f0 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 -- 172.21.15.93:0/2361164165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 msgr2=0x7fe47c104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 --2- 172.21.15.93:0/2361164165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 -- 172.21.15.93:0/2361164165 shutdown_connections 2023-07-21T22:08:30.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 --2- 172.21.15.93:0/2361164165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c104d00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 -- 172.21.15.93:0/2361164165 >> 172.21.15.93:0/2361164165 conn(0x7fe47c0ff1e0 msgr2=0x7fe47c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 -- 172.21.15.93:0/2361164165 shutdown_connections 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.036+0000 7fe483937700 1 -- 172.21.15.93:0/2361164165 wait complete. 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe483937700 1 Processor -- start 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe483937700 1 -- start start 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe483937700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe483937700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c18faa0 con 0x7fe47c1048f0 2023-07-21T22:08:30.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe4816d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe4816d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40864/0 (socket says 172.21.15.93:40864) 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe4816d3700 1 -- 172.21.15.93:0/726994201 learned_addr learned my addr 172.21.15.93:0/726994201 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.037+0000 7fe4816d3700 1 -- 172.21.15.93:0/726994201 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009550 con 0x7fe47c1048f0 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe4816d3700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fb0 tx=0x7fe478011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe478003a40 con 0x7fe47c1048f0 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe478003ba0 con 0x7fe47c1048f0 2023-07-21T22:08:30.429 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c18dbb0 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe47801a750 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c18dfd0 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.039+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7fe478024b60 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:29.039+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe47804c340 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.030+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fe47804b3e0 con 0x7fe47c1048f0 2023-07-21T22:08:30.430 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.031+0000 7fe4727fc700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 0x7fe46803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.031+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe46803bb00 con 0x7fe4680391e0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.031+0000 7fe480ed2700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 0x7fe46803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.031+0000 7fe480ed2700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 0x7fe46803b5c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.032+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fe46803bb00 con 0x7fe4680391e0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe47c18f100 con 0x7fe4680391e0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe4727fc700 1 -- 172.21.15.93:0/726994201 <== mgr.14120 v2:172.21.15.93:6800/3590233301 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe47c18f100 con 0x7fe4680391e0 2023-07-21T22:08:30.431 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 msgr2=0x7fe46803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 0x7fe46803b5c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 msgr2=0x7fe47c18f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fb0 tx=0x7fe478011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 shutdown_connections 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe4680391e0 0x7fe46803b5c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 --2- 172.21.15.93:0/726994201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe47c1048f0 0x7fe47c18f560 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:30.432 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 >> 172.21.15.93:0/726994201 conn(0x7fe47c0ff1e0 msgr2=0x7fe47c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:30.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 shutdown_connections 2023-07-21T22:08:30.433 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.038+0000 7fe483937700 1 -- 172.21.15.93:0/726994201 wait complete. 2023-07-21T22:08:30.433 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 5 is available 2023-07-21T22:08:30.433 INFO:teuthology.orchestra.run.smithi093.stdout:Setting orchestrator backend to cephadm... 2023-07-21T22:08:30.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: Found migration_current of "None". Setting to last migration. 2023-07-21T22:08:30.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:30.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:08:30.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:08:30.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:30.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:30.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:30 smithi093 ceph-mon[96579]: mgrmap e7: x(active, since 1.00563s) 2023-07-21T22:08:31.396 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: from='client.14125 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T22:08:31.396 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: from='client.14125 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T22:08:31.397 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:30] ENGINE Bus STARTING 2023-07-21T22:08:31.397 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:30] ENGINE Serving on http://172.21.15.93:8765 2023-07-21T22:08:31.397 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:30] ENGINE Serving on https://172.21.15.93:7150 2023-07-21T22:08:31.397 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:30] ENGINE Bus STARTED 2023-07-21T22:08:31.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:31.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:31.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:31 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:31.466 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff9570700 1 Processor -- start 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff9570700 1 -- start start 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff9570700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff9570700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4104d40 con 0x7fbff41043f0 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40896/0 (socket says 172.21.15.93:40896) 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.839+0000 7fbff2ffd700 1 -- 172.21.15.93:0/3290037965 learned_addr learned my addr 172.21.15.93:0/3290037965 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:31.467 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff2ffd700 1 -- 172.21.15.93:0/3290037965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4104e80 con 0x7fbff41043f0 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff2ffd700 1 --2- 172.21.15.93:0/3290037965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800bb40 tx=0x7fbfe800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc90bf84b3db8c1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff1ffb700 1 -- 172.21.15.93:0/3290037965 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe800dbe0 con 0x7fbff41043f0 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff1ffb700 1 -- 172.21.15.93:0/3290037965 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfe80044d0 con 0x7fbff41043f0 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff1ffb700 1 -- 172.21.15.93:0/3290037965 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8010470 con 0x7fbff41043f0 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 -- 172.21.15.93:0/3290037965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 msgr2=0x7fbff4104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 --2- 172.21.15.93:0/3290037965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800bb40 tx=0x7fbfe800be50 comp rx=0 tx=0).stop 2023-07-21T22:08:31.468 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 -- 172.21.15.93:0/3290037965 shutdown_connections 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 --2- 172.21.15.93:0/3290037965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff4104800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 -- 172.21.15.93:0/3290037965 >> 172.21.15.93:0/3290037965 conn(0x7fbff40ffa40 msgr2=0x7fbff4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.840+0000 7fbff9570700 1 -- 172.21.15.93:0/3290037965 shutdown_connections 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff9570700 1 -- 172.21.15.93:0/3290037965 wait complete. 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff9570700 1 Processor -- start 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff9570700 1 -- start start 2023-07-21T22:08:31.469 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff9570700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:31.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff9570700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff419b940 con 0x7fbff41043f0 2023-07-21T22:08:31.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.841+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:31.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40898/0 (socket says 172.21.15.93:40898) 2023-07-21T22:08:31.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff2ffd700 1 -- 172.21.15.93:0/667653723 learned_addr learned my addr 172.21.15.93:0/667653723 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:31.470 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff2ffd700 1 -- 172.21.15.93:0/667653723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe800b7f0 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff2ffd700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8003a80 tx=0x7fbfe8003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe80106e0 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfe8010ce0 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff419bb40 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe8021c60 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.842+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff419bf00 con 0x7fbff41043f0 2023-07-21T22:08:31.471 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.843+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fbfe80193f0 con 0x7fbff41043f0 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.843+0000 7fbfdbfff700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 0x7fbfdc03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.843+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbfe804c9f0 con 0x7fbff41043f0 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.843+0000 7fbff27fc700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 0x7fbfdc03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.843+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff4195130 con 0x7fbff41043f0 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.844+0000 7fbff27fc700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 0x7fbfdc03aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4006fd0 tx=0x7fbfe4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:31.472 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:30.848+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbfe800e950 con 0x7fbff41043f0 2023-07-21T22:08:31.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.047+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbff41063c0 con 0x7fbfdc038630 2023-07-21T22:08:31.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.052+0000 7fbfdbfff700 1 -- 172.21.15.93:0/667653723 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbff41063c0 con 0x7fbfdc038630 2023-07-21T22:08:31.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.054+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 msgr2=0x7fbfdc03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:31.473 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.054+0000 7fbff9570700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 0x7fbfdc03aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4006fd0 tx=0x7fbfe4006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.054+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 msgr2=0x7fbff419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.054+0000 7fbff9570700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8003a80 tx=0x7fbfe8003df0 comp rx=0 tx=0).stop 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 shutdown_connections 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fbfdc038630 0x7fbfdc03aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 --2- 172.21.15.93:0/667653723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbff41043f0 0x7fbff419b400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 >> 172.21.15.93:0/667653723 conn(0x7fbff40ffa40 msgr2=0x7fbff4192270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 shutdown_connections 2023-07-21T22:08:31.474 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.055+0000 7fbff9570700 1 -- 172.21.15.93:0/667653723 wait complete. 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f1801266700 1 Processor -- start 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f1801266700 1 -- start start 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f1801266700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f1801266700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc104d50 con 0x7f17fc104400 2023-07-21T22:08:32.407 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f17faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f17faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40902/0 (socket says 172.21.15.93:40902) 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.843+0000 7f17faffd700 1 -- 172.21.15.93:0/3000865797 learned_addr learned my addr 172.21.15.93:0/3000865797 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f17faffd700 1 -- 172.21.15.93:0/3000865797 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc104e90 con 0x7f17fc104400 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f17faffd700 1 --2- 172.21.15.93:0/3000865797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17f00098a0 tx=0x7f17f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e6bd6f29a43cc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f17f9ffb700 1 -- 172.21.15.93:0/3000865797 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17f0004160 con 0x7f17fc104400 2023-07-21T22:08:32.408 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f17f9ffb700 1 -- 172.21.15.93:0/3000865797 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17f00042c0 con 0x7f17fc104400 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f17f9ffb700 1 -- 172.21.15.93:0/3000865797 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17f0011a00 con 0x7f17fc104400 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 -- 172.21.15.93:0/3000865797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 msgr2=0x7f17fc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 --2- 172.21.15.93:0/3000865797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17f00098a0 tx=0x7f17f0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 -- 172.21.15.93:0/3000865797 shutdown_connections 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 --2- 172.21.15.93:0/3000865797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc104810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 -- 172.21.15.93:0/3000865797 >> 172.21.15.93:0/3000865797 conn(0x7f17fc0ffa50 msgr2=0x7f17fc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:32.409 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.844+0000 7f1801266700 1 -- 172.21.15.93:0/3000865797 shutdown_connections 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f1801266700 1 -- 172.21.15.93:0/3000865797 wait complete. 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f1801266700 1 Processor -- start 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f1801266700 1 -- start start 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f1801266700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f1801266700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc19bfe0 con 0x7f17fc104400 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f17faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f17faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40904/0 (socket says 172.21.15.93:40904) 2023-07-21T22:08:32.410 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.845+0000 7f17faffd700 1 -- 172.21.15.93:0/2398745514 learned_addr learned my addr 172.21.15.93:0/2398745514 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f17faffd700 1 -- 172.21.15.93:0/2398745514 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f0009550 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f17faffd700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009fb0 tx=0x7f17f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17f0003a40 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17f0003ba0 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc19c120 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17f001a750 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.846+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc19c540 con 0x7f17fc104400 2023-07-21T22:08:32.411 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.847+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f17f001a8b0 con 0x7f17fc104400 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.847+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc061b40 con 0x7f17fc104400 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.847+0000 7f17e3fff700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 0x7f17e403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.847+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f17f004c690 con 0x7f17fc104400 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.847+0000 7f17fa7fc700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 0x7f17e403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.848+0000 7f17fa7fc700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 0x7f17e403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006fd0 tx=0x7f17ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:32.412 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:31.852+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f17f001ab60 con 0x7f17fc104400 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.038+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f17fc19cba0 con 0x7f17e4038760 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.039+0000 7f17e3fff700 1 -- 172.21.15.93:0/2398745514 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f17fc19cba0 con 0x7f17e4038760 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 msgr2=0x7f17e403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 0x7f17e403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f17ec006fd0 tx=0x7f17ec006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 msgr2=0x7f17fc0739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009fb0 tx=0x7f17f0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 shutdown_connections 2023-07-21T22:08:32.413 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f17e4038760 0x7f17e403ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:32.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 --2- 172.21.15.93:0/2398745514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17fc104400 0x7f17fc0739d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:32.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 >> 172.21.15.93:0/2398745514 conn(0x7f17fc0ffa50 msgr2=0x7f17fc101100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:32.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.041+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 shutdown_connections 2023-07-21T22:08:32.414 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.042+0000 7f1801266700 1 -- 172.21.15.93:0/2398745514 wait complete. 2023-07-21T22:08:32.414 INFO:teuthology.orchestra.run.smithi093.stdout:Generating ssh key... 2023-07-21T22:08:32.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:32 smithi093 ceph-mon[96579]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:32.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:32 smithi093 ceph-mon[96579]: mgrmap e8: x(active, since 2s) 2023-07-21T22:08:33.426 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-mon[96579]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:33.427 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: Generating public/private rsa key pair. 2023-07-21T22:08:33.427 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: Your identification has been saved in /tmp/tmpuuywiv6y/key. 2023-07-21T22:08:33.427 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: Your public key has been saved in /tmp/tmpuuywiv6y/key.pub. 2023-07-21T22:08:33.427 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: The key fingerprint is: 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: SHA256:T1ECnt3iP83822uZIJNuV1v9yaH657o/qEAlCtYFaXs ceph-de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: The key's randomart image is: 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: +---[RSA 3072]----+ 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .+o. . | 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .+.o + | 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | o..+.+.. | 2023-07-21T22:08:33.428 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | . ...Eoo | 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .S.o . .| 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .o = = oo| 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .o = X O| 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .o = @o| 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: | .o=+B=*| 2023-07-21T22:08:33.429 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: +----[SHA256]-----+ 2023-07-21T22:08:33.650 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f46c6700 1 Processor -- start 2023-07-21T22:08:33.651 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f46c6700 1 -- start start 2023-07-21T22:08:33.651 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f46c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:33.651 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f46c6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec073c20 con 0x7fd8ec106010 2023-07-21T22:08:33.651 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f2462700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:33.651 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f2462700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40908/0 (socket says 172.21.15.93:40908) 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f2462700 1 -- 172.21.15.93:0/3788933539 learned_addr learned my addr 172.21.15.93:0/3788933539 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.812+0000 7fd8f2462700 1 -- 172.21.15.93:0/3788933539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec073d60 con 0x7fd8ec106010 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f2462700 1 --2- 172.21.15.93:0/3788933539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098a0 tx=0x7fd8e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a6d6263fd15ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f1460700 1 -- 172.21.15.93:0/3788933539 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0004160 con 0x7fd8ec106010 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f1460700 1 -- 172.21.15.93:0/3788933539 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8e00042c0 con 0x7fd8ec106010 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f1460700 1 -- 172.21.15.93:0/3788933539 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0011a00 con 0x7fd8ec106010 2023-07-21T22:08:33.652 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 -- 172.21.15.93:0/3788933539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 msgr2=0x7fd8ec1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/3788933539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098a0 tx=0x7fd8e0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 -- 172.21.15.93:0/3788933539 shutdown_connections 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/3788933539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec1083f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 -- 172.21.15.93:0/3788933539 >> 172.21.15.93:0/3788933539 conn(0x7fd8ec0ffa50 msgr2=0x7fd8ec101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 -- 172.21.15.93:0/3788933539 shutdown_connections 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.813+0000 7fd8f46c6700 1 -- 172.21.15.93:0/3788933539 wait complete. 2023-07-21T22:08:33.653 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f46c6700 1 Processor -- start 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f46c6700 1 -- start start 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f46c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f46c6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec1997c0 con 0x7fd8ec106010 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f2462700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f2462700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40910/0 (socket says 172.21.15.93:40910) 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.814+0000 7fd8f2462700 1 -- 172.21.15.93:0/2673287571 learned_addr learned my addr 172.21.15.93:0/2673287571 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:33.654 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8f2462700 1 -- 172.21.15.93:0/2673287571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0009550 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8f2462700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fb0 tx=0x7fd8e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0003a40 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8e0003ba0 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec1999c0 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e001a750 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.815+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec199de0 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.816+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec061b40 con 0x7fd8ec106010 2023-07-21T22:08:33.655 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.816+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd8e0024b60 con 0x7fd8ec106010 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.816+0000 7fd8df7fe700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 0x7fd8d803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.816+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd8e004cb50 con 0x7fd8ec106010 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.817+0000 7fd8f1c61700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 0x7fd8d803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.817+0000 7fd8f1c61700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 0x7fd8d803ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:32.821+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd8e00237a0 con 0x7fd8ec106010 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.005+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8ec19c8e0 con 0x7fd8d80386c0 2023-07-21T22:08:33.656 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.244+0000 7fd8df7fe700 1 -- 172.21.15.93:0/2673287571 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd8ec19c8e0 con 0x7fd8d80386c0 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 msgr2=0x7fd8d803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 0x7fd8d803ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 msgr2=0x7fd8ec199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009fb0 tx=0x7fd8e0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 shutdown_connections 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd8d80386c0 0x7fd8d803ab70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:33.657 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 --2- 172.21.15.93:0/2673287571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8ec106010 0x7fd8ec199280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:33.658 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 >> 172.21.15.93:0/2673287571 conn(0x7fd8ec0ffa50 msgr2=0x7fd8ec101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:33.658 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 shutdown_connections 2023-07-21T22:08:33.658 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:33.246+0000 7fd8f46c6700 1 -- 172.21.15.93:0/2673287571 wait complete. 2023-07-21T22:08:34.676 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:34 smithi093 ceph-mon[96579]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:34.676 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:34 smithi093 ceph-mon[96579]: Generating ssh key... 2023-07-21T22:08:34.677 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:34 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:34.677 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:34 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:34.694 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:34.694 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.058+0000 7fe9b0cc8700 1 Processor -- start 2023-07-21T22:08:34.694 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.058+0000 7fe9b0cc8700 1 -- start start 2023-07-21T22:08:34.694 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.058+0000 7fe9b0cc8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.058+0000 7fe9b0cc8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac073c20 con 0x7fe9ac106010 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40914/0 (socket says 172.21.15.93:40914) 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9aa59c700 1 -- 172.21.15.93:0/3430632400 learned_addr learned my addr 172.21.15.93:0/3430632400 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9aa59c700 1 -- 172.21.15.93:0/3430632400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac073d60 con 0x7fe9ac106010 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9aa59c700 1 --2- 172.21.15.93:0/3430632400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00098a0 tx=0x7fe9a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ab47b5c13636fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:34.695 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9a959a700 1 -- 172.21.15.93:0/3430632400 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9a0004160 con 0x7fe9ac106010 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9a959a700 1 -- 172.21.15.93:0/3430632400 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9a00042c0 con 0x7fe9ac106010 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.059+0000 7fe9a959a700 1 -- 172.21.15.93:0/3430632400 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9a0011a00 con 0x7fe9ac106010 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/3430632400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 msgr2=0x7fe9ac1083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/3430632400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00098a0 tx=0x7fe9a0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/3430632400 shutdown_connections 2023-07-21T22:08:34.696 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/3430632400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac1083f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/3430632400 >> 172.21.15.93:0/3430632400 conn(0x7fe9ac0ffa50 msgr2=0x7fe9ac101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/3430632400 shutdown_connections 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.060+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/3430632400 wait complete. 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9b0cc8700 1 Processor -- start 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9b0cc8700 1 -- start start 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9b0cc8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:34.697 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9b0cc8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac1997c0 con 0x7fe9ac106010 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40916/0 (socket says 172.21.15.93:40916) 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9aa59c700 1 -- 172.21.15.93:0/1965106385 learned_addr learned my addr 172.21.15.93:0/1965106385 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.061+0000 7fe9aa59c700 1 -- 172.21.15.93:0/1965106385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0009550 con 0x7fe9ac106010 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe9aa59c700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009130 tx=0x7fe9a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9a0003ab0 con 0x7fe9ac106010 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9a0003c10 con 0x7fe9ac106010 2023-07-21T22:08:34.698 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac1999c0 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9a00247d0 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.062+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac199de0 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.063+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe9a002d3f0 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.063+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac061b40 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.063+0000 7fe99b7fe700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 0x7fe99403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.063+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe9a004c9b0 con 0x7fe9ac106010 2023-07-21T22:08:34.699 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.063+0000 7fe9a9d9b700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 0x7fe99403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.064+0000 7fe9a9d9b700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 0x7fe99403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006fd0 tx=0x7fe99c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.068+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe9a0010970 con 0x7fe9ac106010 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.256+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9ac101d60 con 0x7fe994038760 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.257+0000 7fe99b7fe700 1 -- 172.21.15.93:0/1965106385 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe9ac101d60 con 0x7fe994038760 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 msgr2=0x7fe99403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 0x7fe99403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006fd0 tx=0x7fe99c006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:34.700 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 msgr2=0x7fe9ac199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009130 tx=0x7fe9a0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 shutdown_connections 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe994038760 0x7fe99403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 --2- 172.21.15.93:0/1965106385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9ac106010 0x7fe9ac199280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 >> 172.21.15.93:0/1965106385 conn(0x7fe9ac0ffa50 msgr2=0x7fe9ac101140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 shutdown_connections 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:34.259+0000 7fe9b0cc8700 1 -- 172.21.15.93:0/1965106385 wait complete. 2023-07-21T22:08:34.701 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:08:34.702 INFO:teuthology.orchestra.run.smithi093.stdout:Adding key to root@localhost authorized_keys... 2023-07-21T22:08:34.702 INFO:teuthology.orchestra.run.smithi093.stdout:Adding host smithi093... 2023-07-21T22:08:35.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:35 smithi093 ceph-mon[96579]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:36.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:36 smithi093 ceph-mon[96579]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:37.886 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:37 smithi093 ceph-mon[96579]: Deploying cephadm binary to smithi093 2023-07-21T22:08:37.944 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Added host 'smithi093' with addr '172.21.15.93' 2023-07-21T22:08:37.944 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a7e732700 1 Processor -- start 2023-07-21T22:08:37.944 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a7e732700 1 -- start start 2023-07-21T22:08:37.944 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a7e732700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:37.944 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a7e732700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78104130 con 0x7f2a781037e0 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40920/0 (socket says 172.21.15.93:40920) 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a77fff700 1 -- 172.21.15.93:0/1845111423 learned_addr learned my addr 172.21.15.93:0/1845111423 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a77fff700 1 -- 172.21.15.93:0/1845111423 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78104270 con 0x7f2a781037e0 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.088+0000 7f2a77fff700 1 --2- 172.21.15.93:0/1845111423 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2a680098a0 tx=0x7f2a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57a0c506997881c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a76ffd700 1 -- 172.21.15.93:0/1845111423 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a68004160 con 0x7f2a781037e0 2023-07-21T22:08:37.945 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a76ffd700 1 -- 172.21.15.93:0/1845111423 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a680042c0 con 0x7f2a781037e0 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a76ffd700 1 -- 172.21.15.93:0/1845111423 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a68011a00 con 0x7f2a781037e0 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 -- 172.21.15.93:0/1845111423 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 msgr2=0x7f2a78103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1845111423 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2a680098a0 tx=0x7f2a68009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 -- 172.21.15.93:0/1845111423 shutdown_connections 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1845111423 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78103bf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 -- 172.21.15.93:0/1845111423 >> 172.21.15.93:0/1845111423 conn(0x7f2a780ff1c0 msgr2=0x7f2a781015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:37.946 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 -- 172.21.15.93:0/1845111423 shutdown_connections 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.089+0000 7f2a7e732700 1 -- 172.21.15.93:0/1845111423 wait complete. 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a7e732700 1 Processor -- start 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a7e732700 1 -- start start 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a7e732700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a7e732700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781066b0 con 0x7f2a781037e0 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40922/0 (socket says 172.21.15.93:40922) 2023-07-21T22:08:37.947 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a77fff700 1 -- 172.21.15.93:0/1197947625 learned_addr learned my addr 172.21.15.93:0/1197947625 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.090+0000 7f2a77fff700 1 -- 172.21.15.93:0/1197947625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68009550 con 0x7f2a781037e0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a77fff700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009130 tx=0x7f2a68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a68003ab0 con 0x7f2a781037e0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a68003c10 con 0x7f2a781037e0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a781047c0 con 0x7f2a781037e0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a680247d0 con 0x7f2a781037e0 2023-07-21T22:08:37.948 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.091+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a78104be0 con 0x7f2a781037e0 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.092+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a78061b40 con 0x7f2a781037e0 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.092+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2a6801abb0 con 0x7f2a781037e0 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.093+0000 7f2a757fa700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 0x7f2a6003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.093+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2a6804cb60 con 0x7f2a781037e0 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.093+0000 7f2a777fe700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 0x7f2a6003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:37.949 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.093+0000 7f2a777fe700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 0x7f2a6003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006fd0 tx=0x7f2a6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.097+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2a68018760 con 0x7f2a781037e0 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:35.285+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a78101520 con 0x7f2a60038760 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.554+0000 7f2a757fa700 1 -- 172.21.15.93:0/1197947625 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f2a78101520 con 0x7f2a60038760 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 msgr2=0x7f2a6003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 0x7f2a6003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c006fd0 tx=0x7f2a6c006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 msgr2=0x7f2a78106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:37.950 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009130 tx=0x7f2a68011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 shutdown_connections 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f2a60038760 0x7f2a6003ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 --2- 172.21.15.93:0/1197947625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a781037e0 0x7f2a78106170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 >> 172.21.15.93:0/1197947625 conn(0x7f2a780ff1c0 msgr2=0x7f2a78100900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 shutdown_connections 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:37.557+0000 7f2a7e732700 1 -- 172.21.15.93:0/1197947625 wait complete. 2023-07-21T22:08:37.951 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mon service... 2023-07-21T22:08:38.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:38 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:38.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:38 smithi093 ceph-mon[96579]: Added host smithi093 2023-07-21T22:08:38.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:38 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:39.488 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.356+0000 7fc722c3c700 1 Processor -- start 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.356+0000 7fc722c3c700 1 -- start start 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.356+0000 7fc722c3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.356+0000 7fc722c3c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c104d40 con 0x7fc71c1043f0 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc7209d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:39.489 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc7209d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40956/0 (socket says 172.21.15.93:40956) 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc7209d8700 1 -- 172.21.15.93:0/2491644072 learned_addr learned my addr 172.21.15.93:0/2491644072 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc7209d8700 1 -- 172.21.15.93:0/2491644072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c104e80 con 0x7fc71c1043f0 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc7209d8700 1 --2- 172.21.15.93:0/2491644072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0098a0 tx=0x7fc70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3afc9e36523925c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc71b7fe700 1 -- 172.21.15.93:0/2491644072 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc70c004160 con 0x7fc71c1043f0 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc71b7fe700 1 -- 172.21.15.93:0/2491644072 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc70c0042c0 con 0x7fc71c1043f0 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.357+0000 7fc71b7fe700 1 -- 172.21.15.93:0/2491644072 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc70c011a00 con 0x7fc71c1043f0 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 -- 172.21.15.93:0/2491644072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 msgr2=0x7fc71c104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:39.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 --2- 172.21.15.93:0/2491644072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0098a0 tx=0x7fc70c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 -- 172.21.15.93:0/2491644072 shutdown_connections 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 --2- 172.21.15.93:0/2491644072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c104800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 -- 172.21.15.93:0/2491644072 >> 172.21.15.93:0/2491644072 conn(0x7fc71c0ffa40 msgr2=0x7fc71c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 -- 172.21.15.93:0/2491644072 shutdown_connections 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.358+0000 7fc722c3c700 1 -- 172.21.15.93:0/2491644072 wait complete. 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc722c3c700 1 Processor -- start 2023-07-21T22:08:39.491 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc722c3c700 1 -- start start 2023-07-21T22:08:39.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc722c3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:39.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc722c3c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c19bf90 con 0x7fc71c1043f0 2023-07-21T22:08:39.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc7209d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:39.492 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc7209d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40958/0 (socket says 172.21.15.93:40958) 2023-07-21T22:08:39.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc7209d8700 1 -- 172.21.15.93:0/516319622 learned_addr learned my addr 172.21.15.93:0/516319622 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:39.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc7209d8700 1 -- 172.21.15.93:0/516319622 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009550 con 0x7fc71c1043f0 2023-07-21T22:08:39.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc7209d8700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009fb0 tx=0x7fc70c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:39.493 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc70c003a40 con 0x7fc71c1043f0 2023-07-21T22:08:39.494 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c19c0d0 con 0x7fc71c1043f0 2023-07-21T22:08:39.494 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc70c003ba0 con 0x7fc71c1043f0 2023-07-21T22:08:39.494 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.359+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc70c01a750 con 0x7fc71c1043f0 2023-07-21T22:08:39.494 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.360+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c19c4f0 con 0x7fc71c1043f0 2023-07-21T22:08:39.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.360+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc70c01a9d0 con 0x7fc71c1043f0 2023-07-21T22:08:39.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.361+0000 7fc719ffb700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 0x7fc70803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:39.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.361+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc70c04c780 con 0x7fc71c1043f0 2023-07-21T22:08:39.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.361+0000 7fc71bfff700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 0x7fc70803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:39.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.361+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c061b40 con 0x7fc71c1043f0 2023-07-21T22:08:39.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.361+0000 7fc71bfff700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 0x7fc70803ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc704006fd0 tx=0x7fc704006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:39.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.366+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc70c023390 con 0x7fc71c1043f0 2023-07-21T22:08:39.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.569+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc71c101d30 con 0x7fc7080386c0 2023-07-21T22:08:39.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.574+0000 7fc719ffb700 1 -- 172.21.15.93:0/516319622 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc71c101d30 con 0x7fc7080386c0 2023-07-21T22:08:39.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 msgr2=0x7fc70803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:39.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 0x7fc70803ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc704006fd0 tx=0x7fc704006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:39.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 msgr2=0x7fc71c0739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:39.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009fb0 tx=0x7fc70c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:39.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 shutdown_connections 2023-07-21T22:08:39.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fc7080386c0 0x7fc70803ab70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:39.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 --2- 172.21.15.93:0/516319622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc71c1043f0 0x7fc71c0739f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:39.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 >> 172.21.15.93:0/516319622 conn(0x7fc71c0ffa40 msgr2=0x7fc71c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:39.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 shutdown_connections 2023-07-21T22:08:39.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:38.577+0000 7fc722c3c700 1 -- 172.21.15.93:0/516319622 wait complete. 2023-07-21T22:08:39.499 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mgr service... 2023-07-21T22:08:39.756 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:39 smithi093 ceph-mon[96579]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:39.756 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:39 smithi093 ceph-mon[96579]: Saving service mon spec with placement count:5 2023-07-21T22:08:39.756 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:39 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:40.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe1189d7700 1 Processor -- start 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe1189d7700 1 -- start start 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe1189d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe1189d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114073be0 con 0x7fe114106050 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe11259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe11259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40962/0 (socket says 172.21.15.93:40962) 2023-07-21T22:08:40.633 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe11259c700 1 -- 172.21.15.93:0/505043134 learned_addr learned my addr 172.21.15.93:0/505043134 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.994+0000 7fe11259c700 1 -- 172.21.15.93:0/505043134 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114073d20 con 0x7fe114106050 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe11259c700 1 --2- 172.21.15.93:0/505043134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1040098a0 tx=0x7fe104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201e1d7a31ca9782 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe11159a700 1 -- 172.21.15.93:0/505043134 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe104004160 con 0x7fe114106050 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe11159a700 1 -- 172.21.15.93:0/505043134 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1040042c0 con 0x7fe114106050 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe11159a700 1 -- 172.21.15.93:0/505043134 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe104011a00 con 0x7fe114106050 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 -- 172.21.15.93:0/505043134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 msgr2=0x7fe114108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 --2- 172.21.15.93:0/505043134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1040098a0 tx=0x7fe104009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:40.634 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 -- 172.21.15.93:0/505043134 shutdown_connections 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 --2- 172.21.15.93:0/505043134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114108430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 -- 172.21.15.93:0/505043134 >> 172.21.15.93:0/505043134 conn(0x7fe1140ffa50 msgr2=0x7fe114101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 -- 172.21.15.93:0/505043134 shutdown_connections 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.995+0000 7fe1189d7700 1 -- 172.21.15.93:0/505043134 wait complete. 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe1189d7700 1 Processor -- start 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe1189d7700 1 -- start start 2023-07-21T22:08:40.635 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe1189d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe1189d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114199790 con 0x7fe114106050 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe11259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe11259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40964/0 (socket says 172.21.15.93:40964) 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.996+0000 7fe11259c700 1 -- 172.21.15.93:0/3124218081 learned_addr learned my addr 172.21.15.93:0/3124218081 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe11259c700 1 -- 172.21.15.93:0/3124218081 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104009550 con 0x7fe114106050 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe11259c700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe104009fb0 tx=0x7fe104011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe104003a40 con 0x7fe114106050 2023-07-21T22:08:40.636 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1141999f0 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe104003ba0 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe10401a750 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.997+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe114199e70 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe10401a9d0 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe114109570 con 0x7fe114106050 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe1037fe700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 0x7fe0fc03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:40.637 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe10404c660 con 0x7fe114106050 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe111d9b700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 0x7fe0fc03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:39.998+0000 7fe111d9b700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 0x7fe0fc03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe108006fd0 tx=0x7fe108006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.003+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe104018b80 con 0x7fe114106050 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.050+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mon.0 v2:172.21.15.93:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe104020f00 con 0x7fe114106050 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.215+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe114101d90 con 0x7fe0fc0386c0 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.223+0000 7fe1037fe700 1 -- 172.21.15.93:0/3124218081 <== mgr.14120 v2:172.21.15.93:6800/3590233301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe114101d90 con 0x7fe0fc0386c0 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 msgr2=0x7fe0fc03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:40.638 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 0x7fe0fc03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe108006fd0 tx=0x7fe108006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 msgr2=0x7fe114199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe104009fb0 tx=0x7fe104011ad0 comp rx=0 tx=0).stop 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 shutdown_connections 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe0fc0386c0 0x7fe0fc03ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 --2- 172.21.15.93:0/3124218081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe114106050 0x7fe114199250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.225+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 >> 172.21.15.93:0/3124218081 conn(0x7fe1140ffa50 msgr2=0x7fe114101130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:40.639 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.226+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 shutdown_connections 2023-07-21T22:08:40.640 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:40.226+0000 7fe1189d7700 1 -- 172.21.15.93:0/3124218081 wait complete. 2023-07-21T22:08:41.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:41 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:41 smithi093 ceph-mon[96579]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:41 smithi093 ceph-mon[96579]: Saving service mgr spec with placement count:2 2023-07-21T22:08:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:41 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:41 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:41.730 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.089+0000 7fd3df9f5700 1 Processor -- start 2023-07-21T22:08:41.730 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3df9f5700 1 -- start start 2023-07-21T22:08:41.731 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3df9f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:41.731 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3df9f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d8104d50 con 0x7fd3d8104400 2023-07-21T22:08:41.731 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3dd791700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:41.731 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3dd791700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40968/0 (socket says 172.21.15.93:40968) 2023-07-21T22:08:41.732 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3dd791700 1 -- 172.21.15.93:0/2024979204 learned_addr learned my addr 172.21.15.93:0/2024979204 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:41.732 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3dd791700 1 -- 172.21.15.93:0/2024979204 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8104e90 con 0x7fd3d8104400 2023-07-21T22:08:41.732 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.090+0000 7fd3dd791700 1 --2- 172.21.15.93:0/2024979204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40098a0 tx=0x7fd3d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cb08f134463a5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:41.733 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3cffff700 1 -- 172.21.15.93:0/2024979204 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d4004160 con 0x7fd3d8104400 2023-07-21T22:08:41.733 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3cffff700 1 -- 172.21.15.93:0/2024979204 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d40042c0 con 0x7fd3d8104400 2023-07-21T22:08:41.733 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3cffff700 1 -- 172.21.15.93:0/2024979204 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d40119b0 con 0x7fd3d8104400 2023-07-21T22:08:41.734 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2024979204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 msgr2=0x7fd3d8104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:41.734 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2024979204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40098a0 tx=0x7fd3d4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:41.734 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2024979204 shutdown_connections 2023-07-21T22:08:41.734 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2024979204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d8104810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:41.735 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2024979204 >> 172.21.15.93:0/2024979204 conn(0x7fd3d80ffa50 msgr2=0x7fd3d8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:41.735 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2024979204 shutdown_connections 2023-07-21T22:08:41.735 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.091+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2024979204 wait complete. 2023-07-21T22:08:41.736 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3df9f5700 1 Processor -- start 2023-07-21T22:08:41.736 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3df9f5700 1 -- start start 2023-07-21T22:08:41.736 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3df9f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:41.737 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3df9f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d819bfe0 con 0x7fd3d8104400 2023-07-21T22:08:41.737 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3dd791700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:41.737 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3dd791700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40970/0 (socket says 172.21.15.93:40970) 2023-07-21T22:08:41.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3dd791700 1 -- 172.21.15.93:0/2157149632 learned_addr learned my addr 172.21.15.93:0/2157149632 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:41.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3dd791700 1 -- 172.21.15.93:0/2157149632 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009550 con 0x7fd3d8104400 2023-07-21T22:08:41.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.092+0000 7fd3dd791700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4003f00 tx=0x7fd3d4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:41.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d4003b20 con 0x7fd3d8104400 2023-07-21T22:08:41.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d4003c80 con 0x7fd3d8104400 2023-07-21T22:08:41.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3d4019770 con 0x7fd3d8104400 2023-07-21T22:08:41.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d819c120 con 0x7fd3d8104400 2023-07-21T22:08:41.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d819c540 con 0x7fd3d8104400 2023-07-21T22:08:41.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.093+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd3d40199e0 con 0x7fd3d8104400 2023-07-21T22:08:41.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.094+0000 7fd3ce7fc700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 0x7fd3c403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:41.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.094+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd3d404c910 con 0x7fd3d8104400 2023-07-21T22:08:41.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.094+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d8061b40 con 0x7fd3d8104400 2023-07-21T22:08:41.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.094+0000 7fd3dcf90700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 0x7fd3c403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:41.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.094+0000 7fd3dcf90700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 0x7fd3c403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8006fd0 tx=0x7fd3c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:41.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.098+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd3d4018db0 con 0x7fd3d8104400 2023-07-21T22:08:41.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.304+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd3d819cd70 con 0x7fd3d8104400 2023-07-21T22:08:41.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.310+0000 7fd3ce7fc700 1 -- 172.21.15.93:0/2157149632 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd3d4018350 con 0x7fd3d8104400 2023-07-21T22:08:41.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 msgr2=0x7fd3c403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:41.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 0x7fd3c403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8006fd0 tx=0x7fd3c8006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:41.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 msgr2=0x7fd3d80739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:41.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4003f00 tx=0x7fd3d4011a30 comp rx=0 tx=0).stop 2023-07-21T22:08:41.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 shutdown_connections 2023-07-21T22:08:41.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd3c4038760 0x7fd3c403ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:41.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 --2- 172.21.15.93:0/2157149632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d8104400 0x7fd3d80739d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:41.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 >> 172.21.15.93:0/2157149632 conn(0x7fd3d80ffa50 msgr2=0x7fd3d8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:41.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 shutdown_connections 2023-07-21T22:08:41.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:41.312+0000 7fd3df9f5700 1 -- 172.21.15.93:0/2157149632 wait complete. 2023-07-21T22:08:42.614 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2157149632' entity='client.admin' 2023-07-21T22:08:42.615 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:42 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:42.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.334+0000 7fd9cb19f700 1 Processor -- start 2023-07-21T22:08:42.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.334+0000 7fd9cb19f700 1 -- start start 2023-07-21T22:08:42.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9cb19f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:42.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9cb19f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4073be0 con 0x7fd9c4106050 2023-07-21T22:08:42.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c8f3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:42.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c8f3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40974/0 (socket says 172.21.15.93:40974) 2023-07-21T22:08:42.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c8f3b700 1 -- 172.21.15.93:0/2801202589 learned_addr learned my addr 172.21.15.93:0/2801202589 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:42.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c8f3b700 1 -- 172.21.15.93:0/2801202589 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4073d20 con 0x7fd9c4106050 2023-07-21T22:08:42.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c8f3b700 1 --2- 172.21.15.93:0/2801202589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80098a0 tx=0x7fd9b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f736e8fa8fc498 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:42.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c37fe700 1 -- 172.21.15.93:0/2801202589 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9b8004160 con 0x7fd9c4106050 2023-07-21T22:08:42.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c37fe700 1 -- 172.21.15.93:0/2801202589 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b80042c0 con 0x7fd9c4106050 2023-07-21T22:08:42.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.335+0000 7fd9c37fe700 1 -- 172.21.15.93:0/2801202589 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9b80119b0 con 0x7fd9c4106050 2023-07-21T22:08:42.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 -- 172.21.15.93:0/2801202589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 msgr2=0x7fd9c4108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:42.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/2801202589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80098a0 tx=0x7fd9b8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 -- 172.21.15.93:0/2801202589 shutdown_connections 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/2801202589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4108430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 -- 172.21.15.93:0/2801202589 >> 172.21.15.93:0/2801202589 conn(0x7fd9c40ffa50 msgr2=0x7fd9c4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 -- 172.21.15.93:0/2801202589 shutdown_connections 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.336+0000 7fd9cb19f700 1 -- 172.21.15.93:0/2801202589 wait complete. 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9cb19f700 1 Processor -- start 2023-07-21T22:08:42.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9cb19f700 1 -- start start 2023-07-21T22:08:42.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9cb19f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:42.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9cb19f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41997c0 con 0x7fd9c4106050 2023-07-21T22:08:42.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c8f3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:42.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c8f3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40976/0 (socket says 172.21.15.93:40976) 2023-07-21T22:08:42.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c8f3b700 1 -- 172.21.15.93:0/3137100829 learned_addr learned my addr 172.21.15.93:0/3137100829 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:42.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c8f3b700 1 -- 172.21.15.93:0/3137100829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009550 con 0x7fd9c4106050 2023-07-21T22:08:42.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c8f3b700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8003e60 tx=0x7fd9b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:42.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.337+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9b8003a60 con 0x7fd9c4106050 2023-07-21T22:08:42.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.338+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b8003bc0 con 0x7fd9c4106050 2023-07-21T22:08:42.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.338+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c4199a20 con 0x7fd9c4106050 2023-07-21T22:08:42.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.338+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9b80196c0 con 0x7fd9c4106050 2023-07-21T22:08:42.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.338+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c4199ea0 con 0x7fd9c4106050 2023-07-21T22:08:42.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd9b801ab60 con 0x7fd9c4106050 2023-07-21T22:08:42.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c41095c0 con 0x7fd9c4106050 2023-07-21T22:08:42.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9c1ffb700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 0x7fd9b403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:42.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd9b804c8c0 con 0x7fd9c4106050 2023-07-21T22:08:42.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9c3fff700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 0x7fd9b403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:42.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.339+0000 7fd9c3fff700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 0x7fd9b403abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0006fd0 tx=0x7fd9b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:42.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.344+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd9b8010790 con 0x7fd9c4106050 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.535+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fd9c41097b0 con 0x7fd9c4106050 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.541+0000 7fd9c1ffb700 1 -- 172.21.15.93:0/3137100829 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd9b8010380 con 0x7fd9c4106050 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 msgr2=0x7fd9b403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 0x7fd9b403abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0006fd0 tx=0x7fd9b0006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 msgr2=0x7fd9c4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8003e60 tx=0x7fd9b8011a30 comp rx=0 tx=0).stop 2023-07-21T22:08:42.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 shutdown_connections 2023-07-21T22:08:42.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd9b4038740 0x7fd9b403abf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:42.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 --2- 172.21.15.93:0/3137100829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9c4106050 0x7fd9c4199280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:42.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 >> 172.21.15.93:0/3137100829 conn(0x7fd9c40ffa50 msgr2=0x7fd9c4101130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:42.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 shutdown_connections 2023-07-21T22:08:42.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:42.543+0000 7fd9cb19f700 1 -- 172.21.15.93:0/3137100829 wait complete. 2023-07-21T22:08:42.934 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling the dashboard module... 2023-07-21T22:08:43.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:43 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3137100829' entity='client.admin' 2023-07-21T22:08:43.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:43 smithi093 ceph-mon[96579]: from='mgr.14120 172.21.15.93:0/1365336018' entity='mgr.x' 2023-07-21T22:08:44.620 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1846223391' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-21T22:08:44.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setuser ceph since I am not root 2023-07-21T22:08:44.938 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setgroup ceph since I am not root 2023-07-21T22:08:44.938 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:44.730+0000 7fed90b5c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:08:44.938 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:44.810+0000 7fed90b5c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:44.969 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea101a0700 1 Processor -- start 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea101a0700 1 -- start start 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea101a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea101a0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea08104d50 con 0x7fea08104400 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea0df3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea0df3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40980/0 (socket says 172.21.15.93:40980) 2023-07-21T22:08:44.970 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.409+0000 7fea0df3c700 1 -- 172.21.15.93:0/3612657800 learned_addr learned my addr 172.21.15.93:0/3612657800 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea0df3c700 1 -- 172.21.15.93:0/3612657800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08104e90 con 0x7fea08104400 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea0df3c700 1 --2- 172.21.15.93:0/3612657800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea040098a0 tx=0x7fea04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b02569b012883e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea0cf3a700 1 -- 172.21.15.93:0/3612657800 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea04004160 con 0x7fea08104400 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea0cf3a700 1 -- 172.21.15.93:0/3612657800 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea040042c0 con 0x7fea08104400 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea0cf3a700 1 -- 172.21.15.93:0/3612657800 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea040119b0 con 0x7fea08104400 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea101a0700 1 -- 172.21.15.93:0/3612657800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 msgr2=0x7fea08104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.410+0000 7fea101a0700 1 --2- 172.21.15.93:0/3612657800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea040098a0 tx=0x7fea04009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:44.971 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- 172.21.15.93:0/3612657800 shutdown_connections 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 --2- 172.21.15.93:0/3612657800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea08104810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- 172.21.15.93:0/3612657800 >> 172.21.15.93:0/3612657800 conn(0x7fea080ffa50 msgr2=0x7fea08101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- 172.21.15.93:0/3612657800 shutdown_connections 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- 172.21.15.93:0/3612657800 wait complete. 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 Processor -- start 2023-07-21T22:08:44.972 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- start start 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea101a0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0818fae0 con 0x7fea08104400 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea0df3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea0df3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40982/0 (socket says 172.21.15.93:40982) 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.411+0000 7fea0df3c700 1 -- 172.21.15.93:0/1846223391 learned_addr learned my addr 172.21.15.93:0/1846223391 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:44.973 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fea0df3c700 1 -- 172.21.15.93:0/1846223391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009550 con 0x7fea08104400 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fea0df3c700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea04006af0 tx=0x7fea04011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea04003a40 con 0x7fea08104400 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea04003ba0 con 0x7fea08104400 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea040196c0 con 0x7fea08104400 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0818dbf0 con 0x7fea08104400 2023-07-21T22:08:44.974 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.412+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0818e010 con 0x7fea08104400 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fea04019930 con 0x7fea08104400 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea08061b40 con 0x7fea08104400 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fe9feffd700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 0x7fe9f403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fea0404c810 con 0x7fea08104400 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fea0d73b700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 0x7fe9f403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.413+0000 7fea0d73b700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 0x7fe9f403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8006fd0 tx=0x7fe9f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:44.975 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.418+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fea04019e60 con 0x7fea08104400 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:43.678+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fea08106300 con 0x7fea08104400 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.548+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fea0401aa80 con 0x7fea08104400 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.550+0000 7fe9feffd700 1 -- 172.21.15.93:0/1846223391 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fea04024e40 con 0x7fea08104400 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 msgr2=0x7fe9f403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 0x7fe9f403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8006fd0 tx=0x7fe9f8006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:44.976 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 msgr2=0x7fea0818f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea04006af0 tx=0x7fea04011a10 comp rx=0 tx=0).stop 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 shutdown_connections 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fe9f4038710 0x7fe9f403abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 --2- 172.21.15.93:0/1846223391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08104400 0x7fea0818f5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 >> 172.21.15.93:0/1846223391 conn(0x7fea080ffa50 msgr2=0x7fea08073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 shutdown_connections 2023-07-21T22:08:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:44.553+0000 7fea101a0700 1 -- 172.21.15.93:0/1846223391 wait complete. 2023-07-21T22:08:45.205 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:44.901+0000 7fed90b5c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:08:45.205 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.007+0000 7fed90b5c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:08:45.546 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.186+0000 7fed90b5c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:08:45.546 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.506+0000 7fed90b5c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:08:45.866 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.596+0000 7fed90b5c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:08:45.866 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.730+0000 7fed90b5c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:08:45.866 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1846223391' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-21T22:08:45.867 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-mon[96579]: mgrmap e9: x(active, since 15s) 2023-07-21T22:08:46.120 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.396+0000 7fd2163df700 1 Processor -- start 2023-07-21T22:08:46.121 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd2163df700 1 -- start start 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd2163df700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd2163df700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210073be0 con 0x7fd210103e30 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40996/0 (socket says 172.21.15.93:40996) 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd20ffff700 1 -- 172.21.15.93:0/654289577 learned_addr learned my addr 172.21.15.93:0/654289577 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:46.122 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd20ffff700 1 -- 172.21.15.93:0/654289577 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210073d20 con 0x7fd210103e30 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.397+0000 7fd20ffff700 1 --2- 172.21.15.93:0/654289577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd2000098a0 tx=0x7fd200009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffd2fcbaea75290 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd20effd700 1 -- 172.21.15.93:0/654289577 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd200004160 con 0x7fd210103e30 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd20effd700 1 -- 172.21.15.93:0/654289577 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2000042c0 con 0x7fd210103e30 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd20effd700 1 -- 172.21.15.93:0/654289577 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2000119b0 con 0x7fd210103e30 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 -- 172.21.15.93:0/654289577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 msgr2=0x7fd210104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 --2- 172.21.15.93:0/654289577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd2000098a0 tx=0x7fd200009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.123 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 -- 172.21.15.93:0/654289577 shutdown_connections 2023-07-21T22:08:46.124 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 --2- 172.21.15.93:0/654289577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd210104240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.124 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 -- 172.21.15.93:0/654289577 >> 172.21.15.93:0/654289577 conn(0x7fd2100ffa40 msgr2=0x7fd210101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:46.124 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 -- 172.21.15.93:0/654289577 shutdown_connections 2023-07-21T22:08:46.124 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.398+0000 7fd2163df700 1 -- 172.21.15.93:0/654289577 wait complete. 2023-07-21T22:08:46.124 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.399+0000 7fd2163df700 1 Processor -- start 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.399+0000 7fd2163df700 1 -- start start 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.399+0000 7fd2163df700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.399+0000 7fd2163df700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21019b960 con 0x7fd210103e30 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40998/0 (socket says 172.21.15.93:40998) 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20ffff700 1 -- 172.21.15.93:0/730858053 learned_addr learned my addr 172.21.15.93:0/730858053 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:46.125 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20ffff700 1 -- 172.21.15.93:0/730858053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200009550 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20ffff700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd200003e60 tx=0x7fd200011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd200003970 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd200003ad0 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21019bbc0 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd200019590 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.400+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21019c040 con 0x7fd210103e30 2023-07-21T22:08:46.126 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.401+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd200019850 con 0x7fd210103e30 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.401+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210194c30 con 0x7fd210103e30 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.401+0000 7fd20d7fa700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 0x7fd1f803aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.402+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd20004c860 con 0x7fd210103e30 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.402+0000 7fd20f7fe700 1 -- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 msgr2=0x7fd1f803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.402+0000 7fd20f7fe700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 0x7fd1f803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.406+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd20002b340 con 0x7fd210103e30 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.603+0000 7fd20f7fe700 1 -- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 msgr2=0x7fd1f803aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:46.127 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.603+0000 7fd20f7fe700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 0x7fd1f803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:08:46.128 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.654+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd21002c850 con 0x7fd210103e30 2023-07-21T22:08:46.128 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.655+0000 7fd20d7fa700 1 -- 172.21.15.93:0/730858053 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd200018b40 con 0x7fd210103e30 2023-07-21T22:08:46.128 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 msgr2=0x7fd1f803aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:08:46.128 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 0x7fd1f803aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.128 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 msgr2=0x7fd21019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:46.131 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd200003e60 tx=0x7fd200011aa0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.132 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 shutdown_connections 2023-07-21T22:08:46.132 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7fd1f80386f0 0x7fd1f803aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.132 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 --2- 172.21.15.93:0/730858053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210103e30 0x7fd21019b420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:46.133 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.657+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 >> 172.21.15.93:0/730858053 conn(0x7fd2100ffa40 msgr2=0x7fd21018f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:46.133 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.658+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 shutdown_connections 2023-07-21T22:08:46.133 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:45.658+0000 7fd2163df700 1 -- 172.21.15.93:0/730858053 wait complete. 2023-07-21T22:08:46.134 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-07-21T22:08:46.134 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 9... 2023-07-21T22:08:46.187 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:45 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:45.832+0000 7fed90b5c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:08:46.588 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:46 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:46.269+0000 7fed90b5c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:08:46.884 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:46 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/730858053' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:08:47.187 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:46 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:46.844+0000 7fed90b5c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:08:47.188 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:47 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:47.012+0000 7fed90b5c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:08:48.065 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:47 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:47.765+0000 7fed90b5c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:48.065 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:47 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:47.855+0000 7fed90b5c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:08:48.065 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:48.024+0000 7fed90b5c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:08:48.687 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:48.299+0000 7fed90b5c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:08:48.688 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:48.520+0000 7fed90b5c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:08:48.967 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:48.667+0000 7fed90b5c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:08:49.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:48.927+0000 7fed90b5c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:08:49.937 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:49 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:49.615+0000 7fed90b5c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:08:49.938 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:49 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:49.745+0000 7fed90b5c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:08:50.308 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:49 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:49.938+0000 7fed90b5c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:08:50.309 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:50 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:50.051+0000 7fed90b5c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:08:51.437 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:50 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:50.268+0000 7fed90b5c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:08:51.438 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:08:51.116+0000 7fed90b5c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:08:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: Active manager daemon x restarted 2023-07-21T22:08:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: Activating manager daemon x 2023-07-21T22:08:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: osdmap e3: 0 total, 0 up, 0 in 2023-07-21T22:08:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: mgrmap e10: x(active, starting, since 0.00332666s) 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: Manager daemon x is now available 2023-07-21T22:08:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:51.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:08:52.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:08:52.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:08:52.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:52 smithi093 ceph-mon[96579]: mgrmap e11: x(active, since 1.00551s) 2023-07-21T22:08:52.781 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:08:52.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-21T22:08:52.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:08:52.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:08:52.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cdb934700 1 Processor -- start 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cdb934700 1 -- start start 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cdb934700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cdb934700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4105e90 con 0x7f5cd4073ba0 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cd96d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cd96d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41006/0 (socket says 172.21.15.93:41006) 2023-07-21T22:08:52.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.531+0000 7f5cd96d0700 1 -- 172.21.15.93:0/1939647736 learned_addr learned my addr 172.21.15.93:0/1939647736 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5cd96d0700 1 -- 172.21.15.93:0/1939647736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd410a9e0 con 0x7f5cd4073ba0 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5cd96d0700 1 --2- 172.21.15.93:0/1939647736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098a0 tx=0x7f5cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287cb0d602894cb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5ccbfff700 1 -- 172.21.15.93:0/1939647736 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cd0004160 con 0x7f5cd4073ba0 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5ccbfff700 1 -- 172.21.15.93:0/1939647736 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cd00042c0 con 0x7f5cd4073ba0 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5ccbfff700 1 -- 172.21.15.93:0/1939647736 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cd00119b0 con 0x7f5cd4073ba0 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.532+0000 7f5cdb934700 1 -- 172.21.15.93:0/1939647736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 msgr2=0x7f5cd410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:52.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 --2- 172.21.15.93:0/1939647736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098a0 tx=0x7f5cd0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- 172.21.15.93:0/1939647736 shutdown_connections 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 --2- 172.21.15.93:0/1939647736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd4073ba0 0x7f5cd410a4a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- 172.21.15.93:0/1939647736 >> 172.21.15.93:0/1939647736 conn(0x7f5cd40ffa50 msgr2=0x7f5cd4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- 172.21.15.93:0/1939647736 shutdown_connections 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- 172.21.15.93:0/1939647736 wait complete. 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 Processor -- start 2023-07-21T22:08:52.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- start start 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.533+0000 7f5cdb934700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd419bdf0 con 0x7f5cd419b4a0 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cd96d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cd96d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41008/0 (socket says 172.21.15.93:41008) 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cd96d0700 1 -- 172.21.15.93:0/367363419 learned_addr learned my addr 172.21.15.93:0/367363419 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cd96d0700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009550 con 0x7f5cd419b4a0 2023-07-21T22:08:52.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cd96d0700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006af0 tx=0x7f5cd0011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cd0003a40 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd419bff0 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cd0003ba0 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cd0019690 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.534+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd419ec50 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f5cd001aa60 con 0x7f5cd419b4a0 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cca7fc700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:52.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cc003b2d0 con 0x7f5cc0038710 2023-07-21T22:08:52.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5cd004c800 con 0x7f5cd419b4a0 2023-07-21T22:08:52.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cd8ecf700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:52.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.535+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:08:52.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.736+0000 7f5cd8ecf700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:52.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:46.736+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:08:52.789 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:47.137+0000 7f5cd8ecf700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:52.789 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:47.137+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:08:52.789 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:47.938+0000 7f5cd8ecf700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:52.789 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:47.938+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:49.539+0000 7f5cd8ecf700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3590233301 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:49.539+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:51.132+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f5cd002d430 con 0x7f5cd419b4a0 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:51.132+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 msgr2=0x7f5cc003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:51.132+0000 7f5cca7fc700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.790 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.133+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f5cd004e630 con 0x7f5cd419b4a0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.133+0000 7f5cca7fc700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 0x7f5cc003e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.133+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cc003b2d0 con 0x7f5cc003bff0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.134+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 0x7f5cc003e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.135+0000 7f5cd8ecf700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 0x7f5cc003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003a10 tx=0x7f5cc40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.135+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f5cc003b2d0 con 0x7f5cc003bff0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.141+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5cd419c820 con 0x7f5cc003bff0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cca7fc700 1 -- 172.21.15.93:0/367363419 <== mgr.14152 v2:172.21.15.93:6800/2721150511 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5cd419c820 con 0x7f5cc003bff0 2023-07-21T22:08:52.791 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 msgr2=0x7f5cc003e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 0x7f5cc003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003a10 tx=0x7f5cc40092b0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 msgr2=0x7f5cd419b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006af0 tx=0x7f5cd0011a10 comp rx=0 tx=0).stop 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 shutdown_connections 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5cc003bff0 0x7f5cc003e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cd419b4a0 0x7f5cd419b8b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.792 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 --2- 172.21.15.93:0/367363419 >> [v2:172.21.15.93:6800/3590233301,v1:172.21.15.93:6801/3590233301] conn(0x7f5cc0038710 0x7f5cc003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:52.793 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 >> 172.21.15.93:0/367363419 conn(0x7f5cd40ffa50 msgr2=0x7f5cd4101120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:52.793 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 shutdown_connections 2023-07-21T22:08:52.793 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:52.142+0000 7f5cdb934700 1 -- 172.21.15.93:0/367363419 wait complete. 2023-07-21T22:08:52.793 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 9 is available 2023-07-21T22:08:52.794 INFO:teuthology.orchestra.run.smithi093.stdout:Generating a dashboard self-signed certificate... 2023-07-21T22:08:53.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:52] ENGINE Bus STARTING 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:52] ENGINE Serving on https://172.21.15.93:7150 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:52] ENGINE Serving on http://172.21.15.93:8765 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: [21/Jul/2023:22:08:52] ENGINE Bus STARTED 2023-07-21T22:08:53.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:08:54.026 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-21T22:08:54.026 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc360736700 1 Processor -- start 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc360736700 1 -- start start 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc360736700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc360736700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358073bc0 con 0x7fc358103e60 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc35e4d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc35e4d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41062/0 (socket says 172.21.15.93:41062) 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.224+0000 7fc35e4d2700 1 -- 172.21.15.93:0/2411038817 learned_addr learned my addr 172.21.15.93:0/2411038817 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:54.027 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc35e4d2700 1 -- 172.21.15.93:0/2411038817 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358073d00 con 0x7fc358103e60 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc35e4d2700 1 --2- 172.21.15.93:0/2411038817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbf566f028c3670 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc35d4d0700 1 -- 172.21.15.93:0/2411038817 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc348004160 con 0x7fc358103e60 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc35d4d0700 1 -- 172.21.15.93:0/2411038817 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3480042c0 con 0x7fc358103e60 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc35d4d0700 1 -- 172.21.15.93:0/2411038817 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3480119b0 con 0x7fc358103e60 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 -- 172.21.15.93:0/2411038817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 msgr2=0x7fc358104270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 --2- 172.21.15.93:0/2411038817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3480098a0 tx=0x7fc348009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 -- 172.21.15.93:0/2411038817 shutdown_connections 2023-07-21T22:08:54.028 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 --2- 172.21.15.93:0/2411038817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc358104270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 -- 172.21.15.93:0/2411038817 >> 172.21.15.93:0/2411038817 conn(0x7fc3580ffa50 msgr2=0x7fc358101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 -- 172.21.15.93:0/2411038817 shutdown_connections 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.225+0000 7fc360736700 1 -- 172.21.15.93:0/2411038817 wait complete. 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc360736700 1 Processor -- start 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc360736700 1 -- start start 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc360736700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:54.029 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc360736700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35819ba00 con 0x7fc358103e60 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc35e4d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc35e4d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41064/0 (socket says 172.21.15.93:41064) 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.226+0000 7fc35e4d2700 1 -- 172.21.15.93:0/1739326306 learned_addr learned my addr 172.21.15.93:0/1739326306 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc35e4d2700 1 -- 172.21.15.93:0/1739326306 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348009550 con 0x7fc358103e60 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc35e4d2700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc348003e60 tx=0x7fc348011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc348003a60 con 0x7fc358103e60 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35819bc00 con 0x7fc358103e60 2023-07-21T22:08:54.030 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc348003bc0 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3480196c0 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.227+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35819c020 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fc348019980 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc358194d00 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc34f7fe700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 0x7fc34403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc34804c8b0 con 0x7fc358103e60 2023-07-21T22:08:54.031 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc35dcd1700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 0x7fc34403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.228+0000 7fc35dcd1700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 0x7fc34403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc354006fd0 tx=0x7fc354006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.233+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc348023370 con 0x7fc358103e60 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.432+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc358060950 con 0x7fc344038630 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.644+0000 7fc34f7fe700 1 -- 172.21.15.93:0/1739326306 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc358060950 con 0x7fc344038630 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 msgr2=0x7fc34403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 0x7fc34403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc354006fd0 tx=0x7fc354006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:54.032 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 msgr2=0x7fc35819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc348003e60 tx=0x7fc348011a30 comp rx=0 tx=0).stop 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 shutdown_connections 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc344038630 0x7fc34403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 --2- 172.21.15.93:0/1739326306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc358103e60 0x7fc35819b4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 >> 172.21.15.93:0/1739326306 conn(0x7fc3580ffa50 msgr2=0x7fc35818fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 shutdown_connections 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:53.647+0000 7fc360736700 1 -- 172.21.15.93:0/1739326306 wait complete. 2023-07-21T22:08:54.033 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial admin user... 2023-07-21T22:08:54.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:54 smithi093 ceph-mon[96579]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:54.306 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:08:54.307 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:08:54.307 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:08:54.307 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:54 smithi093 ceph-mon[96579]: mgrmap e12: x(active, since 2s) 2023-07-21T22:08:55.674 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KMkWObC4E61l0X.yf3eWbO2ZXlBoIXz/mBLSNnjEmyaFu6a39F0/i", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689977334, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-21T22:08:55.674 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.489+0000 7fb6f6c22700 1 Processor -- start 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.489+0000 7fb6f6c22700 1 -- start start 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.489+0000 7fb6f6c22700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.489+0000 7fb6f6c22700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f0073be0 con 0x7fb6f0106050 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6f49be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6f49be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41068/0 (socket says 172.21.15.93:41068) 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6f49be700 1 -- 172.21.15.93:0/470984082 learned_addr learned my addr 172.21.15.93:0/470984082 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:55.675 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6f49be700 1 -- 172.21.15.93:0/470984082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0073d20 con 0x7fb6f0106050 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6f49be700 1 --2- 172.21.15.93:0/470984082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40098a0 tx=0x7fb6e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97950f5117ef4771 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6ef7fe700 1 -- 172.21.15.93:0/470984082 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6e4004160 con 0x7fb6f0106050 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6ef7fe700 1 -- 172.21.15.93:0/470984082 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e40042c0 con 0x7fb6f0106050 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.490+0000 7fb6ef7fe700 1 -- 172.21.15.93:0/470984082 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6e40119b0 con 0x7fb6f0106050 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 -- 172.21.15.93:0/470984082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 msgr2=0x7fb6f0108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/470984082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40098a0 tx=0x7fb6e4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:55.676 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 -- 172.21.15.93:0/470984082 shutdown_connections 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/470984082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0108430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 -- 172.21.15.93:0/470984082 >> 172.21.15.93:0/470984082 conn(0x7fb6f00ffa50 msgr2=0x7fb6f0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 -- 172.21.15.93:0/470984082 shutdown_connections 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.491+0000 7fb6f6c22700 1 -- 172.21.15.93:0/470984082 wait complete. 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f6c22700 1 Processor -- start 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f6c22700 1 -- start start 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f6c22700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:55.677 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f6c22700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f0199790 con 0x7fb6f0106050 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f49be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f49be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41070/0 (socket says 172.21.15.93:41070) 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f49be700 1 -- 172.21.15.93:0/2685031750 learned_addr learned my addr 172.21.15.93:0/2685031750 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f49be700 1 -- 172.21.15.93:0/2685031750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4009550 con 0x7fb6f0106050 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6f49be700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006af0 tx=0x7fb6e4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.492+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6e4003a40 con 0x7fb6f0106050 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.493+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6e4003ba0 con 0x7fb6f0106050 2023-07-21T22:08:55.678 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.493+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f01999f0 con 0x7fb6f0106050 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.493+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6e4019690 con 0x7fb6f0106050 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.493+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f0199e70 con 0x7fb6f0106050 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fb6e4019950 con 0x7fb6f0106050 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6edffb700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 0x7fb6e003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6e404d9b0 con 0x7fb6f0106050 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6effff700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 0x7fb6e003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:55.679 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f01095c0 con 0x7fb6f0106050 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.494+0000 7fb6effff700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 0x7fb6e003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006fd0 tx=0x7fb6dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.499+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb6e40243f0 con 0x7fb6f0106050 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.707+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb6f0109780 con 0x7fb6e0038710 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.932+0000 7fb6edffb700 1 -- 172.21.15.93:0/2685031750 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fb6f0109780 con 0x7fb6e0038710 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 msgr2=0x7fb6e003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 0x7fb6e003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc006fd0 tx=0x7fb6dc006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:55.680 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 msgr2=0x7fb6f0199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4006af0 tx=0x7fb6e4011a10 comp rx=0 tx=0).stop 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 shutdown_connections 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e0038710 0x7fb6e003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 --2- 172.21.15.93:0/2685031750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f0106050 0x7fb6f0199250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 >> 172.21.15.93:0/2685031750 conn(0x7fb6f00ffa50 msgr2=0x7fb6f0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 shutdown_connections 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:54.934+0000 7fb6f6c22700 1 -- 172.21.15.93:0/2685031750 wait complete. 2023-07-21T22:08:55.681 INFO:teuthology.orchestra.run.smithi093.stdout:Fetching dashboard port number... 2023-07-21T22:08:56.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:55 smithi093 ceph-mon[96579]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:08:56.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:08:56.716 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 8443 2023-07-21T22:08:56.717 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f28a620d700 1 Processor -- start 2023-07-21T22:08:56.717 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f28a620d700 1 -- start start 2023-07-21T22:08:56.717 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f28a620d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:56.717 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f28a620d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0073c20 con 0x7f28a0106010 2023-07-21T22:08:56.717 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41074/0 (socket says 172.21.15.93:41074) 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f289f7fe700 1 -- 172.21.15.93:0/3578886085 learned_addr learned my addr 172.21.15.93:0/3578886085 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.127+0000 7f289f7fe700 1 -- 172.21.15.93:0/3578886085 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0073d60 con 0x7f28a0106010 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f289f7fe700 1 --2- 172.21.15.93:0/3578886085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f28900098a0 tx=0x7f2890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a22b0dc2e89063c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f289e7fc700 1 -- 172.21.15.93:0/3578886085 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890004160 con 0x7f28a0106010 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f289e7fc700 1 -- 172.21.15.93:0/3578886085 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28900042c0 con 0x7f28a0106010 2023-07-21T22:08:56.718 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f289e7fc700 1 -- 172.21.15.93:0/3578886085 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28900119b0 con 0x7f28a0106010 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 -- 172.21.15.93:0/3578886085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 msgr2=0x7f28a01083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 --2- 172.21.15.93:0/3578886085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f28900098a0 tx=0x7f2890009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 -- 172.21.15.93:0/3578886085 shutdown_connections 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 --2- 172.21.15.93:0/3578886085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01083f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 -- 172.21.15.93:0/3578886085 >> 172.21.15.93:0/3578886085 conn(0x7f28a00ffa50 msgr2=0x7f28a0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 -- 172.21.15.93:0/3578886085 shutdown_connections 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.128+0000 7f28a620d700 1 -- 172.21.15.93:0/3578886085 wait complete. 2023-07-21T22:08:56.719 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f28a620d700 1 Processor -- start 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f28a620d700 1 -- start start 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f28a620d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f28a620d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0199810 con 0x7f28a0106010 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f289f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41076/0 (socket says 172.21.15.93:41076) 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.129+0000 7f289f7fe700 1 -- 172.21.15.93:0/94677256 learned_addr learned my addr 172.21.15.93:0/94677256 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:56.720 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f289f7fe700 1 -- 172.21.15.93:0/94677256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890009550 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f289f7fe700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2890006af0 tx=0x7f2890011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890003a40 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2890003ba0 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a0199a10 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2890019690 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.130+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a0199e30 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.131+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f2890019900 con 0x7f28a0106010 2023-07-21T22:08:56.721 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.131+0000 7f289cff9700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 0x7f288803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.131+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f289004cb60 con 0x7f28a0106010 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.131+0000 7f289effd700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 0x7f288803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.131+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a0061b40 con 0x7f28a0106010 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.135+0000 7f289effd700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 0x7f288803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.136+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f28900243f0 con 0x7f28a0106010 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.331+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f28a019c870 con 0x7f28a0106010 2023-07-21T22:08:56.722 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.331+0000 7f289cff9700 1 -- 172.21.15.93:0/94677256 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f2890018350 con 0x7f28a0106010 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 msgr2=0x7f288803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 0x7f288803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 msgr2=0x7f28a01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2890006af0 tx=0x7f2890011a10 comp rx=0 tx=0).stop 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 shutdown_connections 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2888038760 0x7f288803ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 --2- 172.21.15.93:0/94677256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28a0106010 0x7f28a01992d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:56.723 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 >> 172.21.15.93:0/94677256 conn(0x7f28a00ffa50 msgr2=0x7f28a0101130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:56.724 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 shutdown_connections 2023-07-21T22:08:56.724 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:56.333+0000 7f28a620d700 1 -- 172.21.15.93:0/94677256 wait complete. 2023-07-21T22:08:56.725 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:08:56.725 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-07-21T22:08:56.732 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:08:56.732 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-07-21T22:08:56.733 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-07-21T22:08:56.733 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph Dashboard is now available at: 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout: URL: https://smithi093.front.sepia.ceph.com:8443/ 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout: User: admin 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout: Password: ofy32rrpql 2023-07-21T22:08:56.734 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:56.735 INFO:teuthology.orchestra.run.smithi093.stdout:Saving cluster configuration to /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config directory 2023-07-21T22:08:56.735 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling autotune for osd_memory_target 2023-07-21T22:08:57.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:56 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/94677256' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-21T22:08:57.781 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f0b075700 1 Processor -- start 2023-07-21T22:08:57.781 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f0b075700 1 -- start start 2023-07-21T22:08:57.781 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f0b075700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:57.781 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f0b075700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04104d40 con 0x7f2f041043f0 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f08e11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f08e11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41084/0 (socket says 172.21.15.93:41084) 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f08e11700 1 -- 172.21.15.93:0/987861566 learned_addr learned my addr 172.21.15.93:0/987861566 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f08e11700 1 -- 172.21.15.93:0/987861566 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04104e80 con 0x7f2f041043f0 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f08e11700 1 --2- 172.21.15.93:0/987861566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800bb40 tx=0x7f2ef800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1a93c7374f46bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f037fe700 1 -- 172.21.15.93:0/987861566 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ef800dbe0 con 0x7f2f041043f0 2023-07-21T22:08:57.782 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f037fe700 1 -- 172.21.15.93:0/987861566 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef80044f0 con 0x7f2f041043f0 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.151+0000 7f2f037fe700 1 -- 172.21.15.93:0/987861566 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ef8003e60 con 0x7f2f041043f0 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 -- 172.21.15.93:0/987861566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 msgr2=0x7f2f04104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 --2- 172.21.15.93:0/987861566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800bb40 tx=0x7f2ef800be50 comp rx=0 tx=0).stop 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 -- 172.21.15.93:0/987861566 shutdown_connections 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 --2- 172.21.15.93:0/987861566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f04104800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 -- 172.21.15.93:0/987861566 >> 172.21.15.93:0/987861566 conn(0x7f2f040ffa40 msgr2=0x7f2f04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 -- 172.21.15.93:0/987861566 shutdown_connections 2023-07-21T22:08:57.783 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.152+0000 7f2f0b075700 1 -- 172.21.15.93:0/987861566 wait complete. 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f0b075700 1 Processor -- start 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f0b075700 1 -- start start 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f0b075700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f0b075700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0419b9b0 con 0x7f2f041043f0 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f08e11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f08e11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41086/0 (socket says 172.21.15.93:41086) 2023-07-21T22:08:57.784 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f08e11700 1 -- 172.21.15.93:0/607835741 learned_addr learned my addr 172.21.15.93:0/607835741 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f08e11700 1 -- 172.21.15.93:0/607835741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef800b7f0 con 0x7f2f041043f0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f08e11700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80104d0 tx=0x7f2ef8004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ef8010760 con 0x7f2f041043f0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef8010d80 con 0x7f2f041043f0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ef8019ce0 con 0x7f2f041043f0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.153+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0419bbb0 con 0x7f2f041043f0 2023-07-21T22:08:57.785 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.154+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0419bf70 con 0x7f2f041043f0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.154+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f2ef80194f0 con 0x7f2f041043f0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.154+0000 7f2f01ffb700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 0x7f2ef403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.154+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f04190d20 con 0x7f2f041043f0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.155+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ef804cbb0 con 0x7f2f041043f0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.156+0000 7f2f03fff700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 0x7f2ef403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.156+0000 7f2f03fff700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 0x7f2ef403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006fd0 tx=0x7f2ef0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.159+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2ef802a970 con 0x7f2f041043f0 2023-07-21T22:08:57.786 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.343+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2f04061120 con 0x7f2f041043f0 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.343+0000 7f2f01ffb700 1 -- 172.21.15.93:0/607835741 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2ef800fbc0 con 0x7f2f041043f0 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 msgr2=0x7f2ef403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 0x7f2ef403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0006fd0 tx=0x7f2ef0006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 msgr2=0x7f2f0419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80104d0 tx=0x7f2ef8004020 comp rx=0 tx=0).stop 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 shutdown_connections 2023-07-21T22:08:57.787 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ef4038760 0x7f2ef403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:57.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 --2- 172.21.15.93:0/607835741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f041043f0 0x7f2f0419b470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:57.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 >> 172.21.15.93:0/607835741 conn(0x7f2f040ffa40 msgr2=0x7f2f0418de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:57.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 shutdown_connections 2023-07-21T22:08:57.788 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:57.346+0000 7f2f0b075700 1 -- 172.21.15.93:0/607835741 wait complete. 2023-07-21T22:08:58.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.227+0000 7f48bf8a1700 1 Processor -- start 2023-07-21T22:08:58.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bf8a1700 1 -- start start 2023-07-21T22:08:58.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bf8a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:58.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bf8a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8107460 con 0x7f48b8106b10 2023-07-21T22:08:58.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bd63d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:58.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bd63d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41090/0 (socket says 172.21.15.93:41090) 2023-07-21T22:08:58.928 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bd63d700 1 -- 172.21.15.93:0/3869197174 learned_addr learned my addr 172.21.15.93:0/3869197174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:58.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bd63d700 1 -- 172.21.15.93:0/3869197174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b81075a0 con 0x7f48b8106b10 2023-07-21T22:08:58.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.228+0000 7f48bd63d700 1 --2- 172.21.15.93:0/3869197174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48a80098a0 tx=0x7f48a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8125b569a3219b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:58.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48affff700 1 -- 172.21.15.93:0/3869197174 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48a8004160 con 0x7f48b8106b10 2023-07-21T22:08:58.929 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48affff700 1 -- 172.21.15.93:0/3869197174 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48a80042c0 con 0x7f48b8106b10 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48affff700 1 -- 172.21.15.93:0/3869197174 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48a80119b0 con 0x7f48b8106b10 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 -- 172.21.15.93:0/3869197174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 msgr2=0x7f48b8106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/3869197174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48a80098a0 tx=0x7f48a8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 -- 172.21.15.93:0/3869197174 shutdown_connections 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/3869197174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8106f20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 -- 172.21.15.93:0/3869197174 >> 172.21.15.93:0/3869197174 conn(0x7f48b80ff1c0 msgr2=0x7f48b81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:58.930 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 -- 172.21.15.93:0/3869197174 shutdown_connections 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.229+0000 7f48bf8a1700 1 -- 172.21.15.93:0/3869197174 wait complete. 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bf8a1700 1 Processor -- start 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bf8a1700 1 -- start start 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bf8a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bf8a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81997d0 con 0x7f48b8106b10 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bd63d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:58.931 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bd63d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41092/0 (socket says 172.21.15.93:41092) 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bd63d700 1 -- 172.21.15.93:0/2735809882 learned_addr learned my addr 172.21.15.93:0/2735809882 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bd63d700 1 -- 172.21.15.93:0/2735809882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009550 con 0x7f48b8106b10 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.230+0000 7f48bd63d700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48a8006af0 tx=0x7f48a8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48a8003a40 con 0x7f48b8106b10 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48a8003ba0 con 0x7f48b8106b10 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b81999d0 con 0x7f48b8106b10 2023-07-21T22:08:58.932 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b8199df0 con 0x7f48b8106b10 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48a80196c0 con 0x7f48b8106b10 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f48a8019930 con 0x7f48b8106b10 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.231+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c005320 con 0x7f48b8106b10 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.232+0000 7f48ae7fc700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 0x7f48a403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.232+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48a804cba0 con 0x7f48b8106b10 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.232+0000 7f48bce3c700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 0x7f48a403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:08:58.933 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.232+0000 7f48bce3c700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 0x7f48a403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.236+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f48a8018760 con 0x7f48b8106b10 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.524+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f489c005f70 con 0x7f48b8106b10 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.526+0000 7f48ae7fc700 1 -- 172.21.15.93:0/2735809882 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f48a8018350 con 0x7f48b8106b10 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 msgr2=0x7f48a403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 0x7f48a403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4006e40 comp rx=0 tx=0).stop 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 msgr2=0x7f48b8199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48a8006af0 tx=0x7f48a8011a10 comp rx=0 tx=0).stop 2023-07-21T22:08:58.934 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 shutdown_connections 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f48a4038800 0x7f48a403acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 --2- 172.21.15.93:0/2735809882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48b8106b10 0x7f48b8199290 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 >> 172.21.15.93:0/2735809882 conn(0x7f48b80ff1c0 msgr2=0x7f48b81008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.528+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 shutdown_connections 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-07-21T22:08:58.529+0000 7f48bf8a1700 1 -- 172.21.15.93:0/2735809882 wait complete. 2023-07-21T22:08:58.935 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout:Or, if you are only running a single cluster on this host: 2023-07-21T22:08:58.936 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout: ceph telemetry on 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.937 INFO:teuthology.orchestra.run.smithi093.stdout:For more information see: 2023-07-21T22:08:58.938 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.938 INFO:teuthology.orchestra.run.smithi093.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-21T22:08:58.938 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:08:58.938 INFO:teuthology.orchestra.run.smithi093.stdout:Bootstrap complete. 2023-07-21T22:08:58.959 INFO:tasks.cephadm:Fetching config... 2023-07-21T22:08:58.960 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:08:58.960 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-21T22:08:58.979 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-21T22:08:58.980 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:08:58.980 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-21T22:08:59.037 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-21T22:08:59.037 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:08:59.037 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-21T22:08:59.107 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-21T22:08:59.108 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:08:59.108 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-21T22:08:59.164 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-21T22:08:59.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:08:59.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:58 smithi093 ceph-mon[96579]: mgrmap e13: x(active, since 6s) 2023-07-21T22:08:59.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:08:58 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2735809882' entity='client.admin' 2023-07-21T22:08:59.225 INFO:teuthology.orchestra.run.smithi093.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:59.236 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:08:59.283 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:59.295 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:08:59.341 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDR828hKHfprhicBSFapl/o4rvhS+dz9xv75zFFsmWTiytE9ABx3+2YfQMkCQx1UMEZhzEXOFCD2st8Rf4hilKj4HdxNCCzpVOu8sxhBBNArdMJghTKq95Bv7/K/sj420ILEuB38GQ7yxrUYd3vt0nnvM5gHEJzdpi02gKYRzyNa50+PpNhx7mN+Rbq7olrphpgjXmHSJuznLzD+ab7UOQAC3tjgAboQWAuYRcjMCP9lPV7GtUVYukNUQzl5J2A/VcYIRF3ObC66Z/iyM+vzK0h4x0jY38TacBDxbO9Q/MybSDyWrmcITjPJQ/lfMIDvQ6oZn54+klMYn/WINnbxRN0ngy+RU2xUFYxDUyyi0lQM6R5Aa6m983gM0Q3wgd6Cl/ebnoUIoHFr/c5KXpNTMthGmGa/KcbxnNKTKXu86ybPtqjD+pUxeaTxftQTFHW8DU9tWFisMU4KscWFyhksRhLSsQlYMbSXqPQdNwtetPmt8goDd7+T1hUS2ad9pHr1jU= ceph-de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:08:59.354 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-21T22:08:59.910 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:01.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 -- 172.21.15.93:0/2515140792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 msgr2=0x7f113c0fdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:01.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 --2- 172.21.15.93:0/2515140792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c0fdcd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1138009910 tx=0x7f1138009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 -- 172.21.15.93:0/2515140792 shutdown_connections 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 --2- 172.21.15.93:0/2515140792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c0fdcd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 -- 172.21.15.93:0/2515140792 >> 172.21.15.93:0/2515140792 conn(0x7f113c0f9210 msgr2=0x7f113c0fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 -- 172.21.15.93:0/2515140792 shutdown_connections 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.363+0000 7f11446a6700 1 -- 172.21.15.93:0/2515140792 wait complete. 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f11446a6700 1 Processor -- start 2023-07-21T22:09:01.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f11446a6700 1 -- start start 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f11446a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f11446a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c1007e0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f1142442700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f1142442700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41098/0 (socket says 172.21.15.93:41098) 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.364+0000 7f1142442700 1 -- 172.21.15.93:0/4137248669 learned_addr learned my addr 172.21.15.93:0/4137248669 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f1142442700 1 -- 172.21.15.93:0/4137248669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380095f0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f1142442700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1138006010 tx=0x7f11380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:01.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f113801d070 con 0x7f113c0fd8c0 2023-07-21T22:09:01.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c0fe8f0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11380044e0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11380039b0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.365+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c0fed30 con 0x7f113c0fd8c0 2023-07-21T22:09:01.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1138003c70 con 0x7f113c0fd8c0 2023-07-21T22:09:01.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c061b40 con 0x7f113c0fd8c0 2023-07-21T22:09:01.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f11337fe700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 0x7f112803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:01.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f113804c950 con 0x7f113c0fd8c0 2023-07-21T22:09:01.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f1141c41700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 0x7f112803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:01.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.366+0000 7f1141c41700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 0x7f112803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:01.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.371+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f113802a950 con 0x7f113c0fd8c0 2023-07-21T22:09:01.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.563+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f113c18dba0 con 0x7f113c0fd8c0 2023-07-21T22:09:01.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.566+0000 7f11337fe700 1 -- 172.21.15.93:0/4137248669 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f113802a360 con 0x7f113c0fd8c0 2023-07-21T22:09:01.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.568+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 msgr2=0x7f112803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:01.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.568+0000 7f11446a6700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 0x7f112803ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:01.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.568+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 msgr2=0x7f113c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:01.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.568+0000 7f11446a6700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1138006010 tx=0x7f11380057c0 comp rx=0 tx=0).stop 2023-07-21T22:09:01.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 shutdown_connections 2023-07-21T22:09:01.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f11280387b0 0x7f112803ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:01.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 --2- 172.21.15.93:0/4137248669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f113c0fd8c0 0x7f113c1002a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 >> 172.21.15.93:0/4137248669 conn(0x7f113c0f9210 msgr2=0x7f113c0f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 shutdown_connections 2023-07-21T22:09:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:01.569+0000 7f11446a6700 1 -- 172.21.15.93:0/4137248669 wait complete. 2023-07-21T22:09:01.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi093", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4137248669' entity='client.admin' 2023-07-21T22:09:01.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:01.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:02.234 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-21T22:09:02.235 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-21T22:09:02.785 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:04.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.048+0000 7facde5ea700 1 -- 172.21.15.93:0/1557381977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 msgr2=0x7facd80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:04.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.048+0000 7facde5ea700 1 --2- 172.21.15.93:0/1557381977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80ffed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7facc8009910 tx=0x7facc8009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:04.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 -- 172.21.15.93:0/1557381977 shutdown_connections 2023-07-21T22:09:04.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 --2- 172.21.15.93:0/1557381977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80ffed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:04.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 -- 172.21.15.93:0/1557381977 >> 172.21.15.93:0/1557381977 conn(0x7facd80f9210 msgr2=0x7facd80fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 -- 172.21.15.93:0/1557381977 shutdown_connections 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 -- 172.21.15.93:0/1557381977 wait complete. 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 Processor -- start 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.049+0000 7facde5ea700 1 -- start start 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facde5ea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:04.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facde5ea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8197480 con 0x7facd80ffac0 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41104/0 (socket says 172.21.15.93:41104) 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd7fff700 1 -- 172.21.15.93:0/526148232 learned_addr learned my addr 172.21.15.93:0/526148232 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd7fff700 1 -- 172.21.15.93:0/526148232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80095f0 con 0x7facd80ffac0 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd7fff700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facc8006010 tx=0x7facc80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:04.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc801d070 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facc80044e0 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd81975c0 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facc80039b0 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.050+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd8197a00 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.051+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7facc8003c70 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.051+0000 7facd57fa700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 0x7facc003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.051+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facc804c960 con 0x7facd80ffac0 2023-07-21T22:09:04.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.051+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd818d0b0 con 0x7facd80ffac0 2023-07-21T22:09:04.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.051+0000 7facd77fe700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 0x7facc003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:04.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.052+0000 7facd77fe700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 0x7facc003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faccc006fd0 tx=0x7faccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:04.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.056+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7facc800f460 con 0x7facd80ffac0 2023-07-21T22:09:04.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.244+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7facd80609e0 con 0x7facc00387b0 2023-07-21T22:09:04.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.248+0000 7facd57fa700 1 -- 172.21.15.93:0/526148232 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7facd80609e0 con 0x7facc00387b0 2023-07-21T22:09:04.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 msgr2=0x7facc003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:04.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 0x7facc003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faccc006fd0 tx=0x7faccc006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 msgr2=0x7facd80fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facc8006010 tx=0x7facc80057c0 comp rx=0 tx=0).stop 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 shutdown_connections 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7facc00387b0 0x7facc003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 --2- 172.21.15.93:0/526148232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facd80ffac0 0x7facd80fe1e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 >> 172.21.15.93:0/526148232 conn(0x7facd80f9210 msgr2=0x7facd80f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:04.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 shutdown_connections 2023-07-21T22:09:04.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:04.250+0000 7facde5ea700 1 -- 172.21.15.93:0/526148232 wait complete. 2023-07-21T22:09:04.739 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2023-07-21T22:09:04.740 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:09:04.740 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:09:04.760 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:09:04.761 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:04.819 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2023-07-21T22:09:04.820 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch host add smithi134 2023-07-21T22:09:05.381 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:05.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:05.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:05.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:05.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:05.586 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:05.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:09:05.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:05.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:05.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:05.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:06.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.558+0000 7f63b518d700 1 -- 172.21.15.93:0/2390320214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 msgr2=0x7f63b0105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:06.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.558+0000 7f63b518d700 1 --2- 172.21.15.93:0/2390320214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b0105a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009910 tx=0x7f63a4009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:06.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 -- 172.21.15.93:0/2390320214 shutdown_connections 2023-07-21T22:09:06.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 --2- 172.21.15.93:0/2390320214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b0105a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:06.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 -- 172.21.15.93:0/2390320214 >> 172.21.15.93:0/2390320214 conn(0x7f63b00fb130 msgr2=0x7f63b00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:06.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 -- 172.21.15.93:0/2390320214 shutdown_connections 2023-07-21T22:09:06.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 -- 172.21.15.93:0/2390320214 wait complete. 2023-07-21T22:09:06.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.559+0000 7f63b518d700 1 Processor -- start 2023-07-21T22:09:06.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63b518d700 1 -- start start 2023-07-21T22:09:06.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63b518d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:06.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63b518d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b019ce70 con 0x7f63b0105630 2023-07-21T22:09:06.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63aed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41110/0 (socket says 172.21.15.93:41110) 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63aed9d700 1 -- 172.21.15.93:0/4064579994 learned_addr learned my addr 172.21.15.93:0/4064579994 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63aed9d700 1 -- 172.21.15.93:0/4064579994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40095f0 con 0x7f63b0105630 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f63aed9d700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.560+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63a401d070 con 0x7f63b0105630 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b019d0d0 con 0x7f63b0105630 2023-07-21T22:09:06.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a400bc10 con 0x7f63b0105630 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63a4003910 con 0x7f63b0105630 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b01969b0 con 0x7f63b0105630 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f63a4003bd0 con 0x7f63b0105630 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.561+0000 7f6397fff700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 0x7f639803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.562+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f63a404d430 con 0x7f63b0105630 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.562+0000 7f63ae59c700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 0x7f639803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:06.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.562+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b0061b40 con 0x7f63b0105630 2023-07-21T22:09:06.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.562+0000 7f63ae59c700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 0x7f639803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006fd0 tx=0x7f63a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:06.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.566+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f63a4026020 con 0x7f63b0105630 2023-07-21T22:09:06.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:06 smithi093 ceph-mon[96579]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:06.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:06 smithi093 ceph-mon[96579]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:09:06.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:06.757+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b0196c00 con 0x7f63980387b0 2023-07-21T22:09:08.187 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:07 smithi093 ceph-mon[96579]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:08.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:07 smithi093 ceph-mon[96579]: Deploying cephadm binary to smithi134 2023-07-21T22:09:09.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.020+0000 7f6397fff700 1 -- 172.21.15.93:0/4064579994 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f63b0196c00 con 0x7f63980387b0 2023-07-21T22:09:09.061 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi134' with addr '172.21.15.134' 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 msgr2=0x7f639803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 0x7f639803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63a0006fd0 tx=0x7f63a0006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 msgr2=0x7f63b019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f63a4006010 tx=0x7f63a4004ff0 comp rx=0 tx=0).stop 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 shutdown_connections 2023-07-21T22:09:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f63980387b0 0x7f639803ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:09.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 --2- 172.21.15.93:0/4064579994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63b0105630 0x7f63b019c930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:09.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 >> 172.21.15.93:0/4064579994 conn(0x7f63b00fb130 msgr2=0x7f63b0074200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:09.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 shutdown_connections 2023-07-21T22:09:09.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:09.022+0000 7f63b518d700 1 -- 172.21.15.93:0/4064579994 wait complete. 2023-07-21T22:09:09.442 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:09:09.995 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:10.351 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:10.352 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:10 smithi093 ceph-mon[96579]: Added host smithi134 2023-07-21T22:09:10.352 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:10.352 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:11.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- 172.21.15.93:0/3249737842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 msgr2=0x7f27780fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:11.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 --2- 172.21.15.93:0/3249737842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780fe820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2768009910 tx=0x7f2768009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:11.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- 172.21.15.93:0/3249737842 shutdown_connections 2023-07-21T22:09:11.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 --2- 172.21.15.93:0/3249737842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780fe820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:11.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- 172.21.15.93:0/3249737842 >> 172.21.15.93:0/3249737842 conn(0x7f2778074f10 msgr2=0x7f2778075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- 172.21.15.93:0/3249737842 shutdown_connections 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- 172.21.15.93:0/3249737842 wait complete. 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 Processor -- start 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.270+0000 7f27804c5700 1 -- start start 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f27804c5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:11.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f27804c5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778101350 con 0x7f27780fe450 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f277e261700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f277e261700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41136/0 (socket says 172.21.15.93:41136) 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f277e261700 1 -- 172.21.15.93:0/1904787980 learned_addr learned my addr 172.21.15.93:0/1904787980 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f277e261700 1 -- 172.21.15.93:0/1904787980 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27680095f0 con 0x7f27780fe450 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f277e261700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2768006010 tx=0x7f2768004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f276801d070 con 0x7f27780fe450 2023-07-21T22:09:11.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276800bc10 con 0x7f27780fe450 2023-07-21T22:09:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780ff910 con 0x7f27780fe450 2023-07-21T22:09:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2768003910 con 0x7f27780fe450 2023-07-21T22:09:11.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.271+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780ffe10 con 0x7f27780fe450 2023-07-21T22:09:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.272+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f276800bd80 con 0x7f27780fe450 2023-07-21T22:09:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.272+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2778106c70 con 0x7f27780fe450 2023-07-21T22:09:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.272+0000 7f276f7fe700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 0x7f276403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.272+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f276804c480 con 0x7f27780fe450 2023-07-21T22:09:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.272+0000 7f277da60700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 0x7f276403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.273+0000 7f277da60700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 0x7f276403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.277+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2768029360 con 0x7f27780fe450 2023-07-21T22:09:11.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:11 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:11.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.468+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2778100900 con 0x7f2764038800 2023-07-21T22:09:11.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.470+0000 7f276f7fe700 1 -- 172.21.15.93:0/1904787980 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f2778100900 con 0x7f2764038800 2023-07-21T22:09:11.510 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:09:11.511 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2023-07-21T22:09:11.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 msgr2=0x7f276403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:11.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 0x7f276403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:11.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 msgr2=0x7f27780ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2768006010 tx=0x7f2768004ff0 comp rx=0 tx=0).stop 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 shutdown_connections 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2764038800 0x7f276403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 --2- 172.21.15.93:0/1904787980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27780fe450 0x7f27780ff3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 >> 172.21.15.93:0/1904787980 conn(0x7f2778074f10 msgr2=0x7f2778103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 shutdown_connections 2023-07-21T22:09:11.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:11.472+0000 7f27804c5700 1 -- 172.21.15.93:0/1904787980 wait complete. 2023-07-21T22:09:11.949 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2023-07-21T22:09:11.950 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:09:11.950 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:09:11.970 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:09:11.971 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:12.030 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2023-07-21T22:09:12.030 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch host add smithi194 2023-07-21T22:09:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:12.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:12 smithi093 ceph-mon[96579]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:12.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:12 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:12.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:12 smithi093 ceph-mon[96579]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:09:13.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 -- 172.21.15.93:0/3594778499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 msgr2=0x7f37ec0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:13.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/3594778499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec0fe820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009910 tx=0x7f37dc009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:13.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 -- 172.21.15.93:0/3594778499 shutdown_connections 2023-07-21T22:09:13.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/3594778499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec0fe820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:13.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 -- 172.21.15.93:0/3594778499 >> 172.21.15.93:0/3594778499 conn(0x7f37ec074f10 msgr2=0x7f37ec075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 -- 172.21.15.93:0/3594778499 shutdown_connections 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.724+0000 7f37f1e9e700 1 -- 172.21.15.93:0/3594778499 wait complete. 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37f1e9e700 1 Processor -- start 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37f1e9e700 1 -- start start 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37f1e9e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37f1e9e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0fc060 con 0x7f37ec0fe450 2023-07-21T22:09:13.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37eb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37eb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41142/0 (socket says 172.21.15.93:41142) 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.725+0000 7f37eb7fe700 1 -- 172.21.15.93:0/1909991421 learned_addr learned my addr 172.21.15.93:0/1909991421 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37eb7fe700 1 -- 172.21.15.93:0/1909991421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc0095f0 con 0x7f37ec0fe450 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37eb7fe700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37dc006010 tx=0x7f37dc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37dc01d070 con 0x7f37ec0fe450 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37dc00bc10 con 0x7f37ec0fe450 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec193d20 con 0x7f37ec0fe450 2023-07-21T22:09:13.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37dc003910 con 0x7f37ec0fe450 2023-07-21T22:09:13.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.726+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec194140 con 0x7f37ec0fe450 2023-07-21T22:09:13.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f37dc003bd0 con 0x7f37ec0fe450 2023-07-21T22:09:13.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec104400 con 0x7f37ec0fe450 2023-07-21T22:09:13.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37e8ff9700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 0x7f37d403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:13.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f37dc04c6f0 con 0x7f37ec0fe450 2023-07-21T22:09:13.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37eaffd700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 0x7f37d403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:13.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.727+0000 7f37eaffd700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 0x7f37d403ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37e0006fd0 tx=0x7f37e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:13.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.732+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f37dc029950 con 0x7f37ec0fe450 2023-07-21T22:09:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:13.918+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) v1 -- 0x7f37ec1943a0 con 0x7f37d40387b0 2023-07-21T22:09:14.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:14.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:14.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:14.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:14.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:14.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:09:14.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:14.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:15.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:15.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:09:15.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:15.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:15.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: Deploying cephadm binary to smithi194 2023-07-21T22:09:15.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:15.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:15.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:16.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.179+0000 7f37e8ff9700 1 -- 172.21.15.93:0/1909991421 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f37ec1943a0 con 0x7f37d40387b0 2023-07-21T22:09:16.220 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi194' with addr '172.21.15.194' 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 msgr2=0x7f37d403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 0x7f37d403ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f37e0006fd0 tx=0x7f37e0006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 msgr2=0x7f37ec1937e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37dc006010 tx=0x7f37dc004ff0 comp rx=0 tx=0).stop 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 shutdown_connections 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f37d40387b0 0x7f37d403ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 --2- 172.21.15.93:0/1909991421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37ec0fe450 0x7f37ec1937e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 >> 172.21.15.93:0/1909991421 conn(0x7f37ec074f10 msgr2=0x7f37ec101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 shutdown_connections 2023-07-21T22:09:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:16.181+0000 7f37f1e9e700 1 -- 172.21.15.93:0/1909991421 wait complete. 2023-07-21T22:09:16.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:16 smithi093 ceph-mon[96579]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:09:16.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:16 smithi093 ceph-mon[96579]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:16.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:16.587 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:16.641 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:09:17.190 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:17.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:17 smithi093 ceph-mon[96579]: Added host smithi194 2023-07-21T22:09:17.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:17 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:17.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:17 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 -- 172.21.15.93:0/4112068815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 msgr2=0x7fd0700ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 --2- 172.21.15.93:0/4112068815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd0700ff930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd064009910 tx=0x7fd064009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 -- 172.21.15.93:0/4112068815 shutdown_connections 2023-07-21T22:09:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 --2- 172.21.15.93:0/4112068815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd0700ff930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 -- 172.21.15.93:0/4112068815 >> 172.21.15.93:0/4112068815 conn(0x7fd070074f10 msgr2=0x7fd070075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 -- 172.21.15.93:0/4112068815 shutdown_connections 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.318+0000 7fd075965700 1 -- 172.21.15.93:0/4112068815 wait complete. 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd075965700 1 Processor -- start 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd075965700 1 -- start start 2023-07-21T22:09:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd075965700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd075965700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07019ae00 con 0x7fd0700ff560 2023-07-21T22:09:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd06effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd06effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41168/0 (socket says 172.21.15.93:41168) 2023-07-21T22:09:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd06effd700 1 -- 172.21.15.93:0/3247303761 learned_addr learned my addr 172.21.15.93:0/3247303761 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd06effd700 1 -- 172.21.15.93:0/3247303761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640095f0 con 0x7fd0700ff560 2023-07-21T22:09:18.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.319+0000 7fd06effd700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd064004f40 tx=0x7fd064005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.320+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd06401d070 con 0x7fd0700ff560 2023-07-21T22:09:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.320+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07019b000 con 0x7fd0700ff560 2023-07-21T22:09:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.320+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd06400be20 con 0x7fd0700ff560 2023-07-21T22:09:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.320+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd064022d00 con 0x7fd0700ff560 2023-07-21T22:09:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.320+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07019b420 con 0x7fd0700ff560 2023-07-21T22:09:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd064022500 con 0x7fd0700ff560 2023-07-21T22:09:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd070061b40 con 0x7fd0700ff560 2023-07-21T22:09:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd074963700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 0x7fd05803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd06404c7d0 con 0x7fd0700ff560 2023-07-21T22:09:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd06e7fc700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 0x7fd05803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.321+0000 7fd06e7fc700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 0x7fd05803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd060006fd0 tx=0x7fd060006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.325+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd06402a540 con 0x7fd0700ff560 2023-07-21T22:09:18.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.515+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd070103a40 con 0x7fd0580387b0 2023-07-21T22:09:18.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.516+0000 7fd074963700 1 -- 172.21.15.93:0/3247303761 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fd070103a40 con 0x7fd0580387b0 2023-07-21T22:09:18.557 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:09:18.557 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2023-07-21T22:09:18.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 msgr2=0x7fd05803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:18.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 0x7fd05803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd060006fd0 tx=0x7fd060006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 msgr2=0x7fd07019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd064004f40 tx=0x7fd064005020 comp rx=0 tx=0).stop 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 shutdown_connections 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd0580387b0 0x7fd05803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 --2- 172.21.15.93:0/3247303761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0700ff560 0x7fd07019a8c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 >> 172.21.15.93:0/3247303761 conn(0x7fd070074f10 msgr2=0x7fd070102320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:18.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 shutdown_connections 2023-07-21T22:09:18.561 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:18.518+0000 7fd075965700 1 -- 172.21.15.93:0/3247303761 wait complete. 2023-07-21T22:09:18.933 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:18 smithi093 ceph-mon[96579]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:18.934 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:18.934 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-21T22:09:18.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-21T22:09:19.485 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:19.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:19 smithi093 ceph-mon[96579]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:09:20.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/2620768827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 msgr2=0x7fd1a01048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:20.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/2620768827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a01048c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd190009910 tx=0x7fd190009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/2620768827 shutdown_connections 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/2620768827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a01048c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/2620768827 >> 172.21.15.93:0/2620768827 conn(0x7fd1a00742b0 msgr2=0x7fd1a00746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/2620768827 shutdown_connections 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.640+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/2620768827 wait complete. 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd1a5fa6700 1 Processor -- start 2023-07-21T22:09:20.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd1a5fa6700 1 -- start start 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd1a5fa6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd1a5fa6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0195ea0 con 0x7fd1a01044f0 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd19f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd19f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41174/0 (socket says 172.21.15.93:41174) 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.641+0000 7fd19f7fe700 1 -- 172.21.15.93:0/3558849273 learned_addr learned my addr 172.21.15.93:0/3558849273 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd19f7fe700 1 -- 172.21.15.93:0/3558849273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900095f0 con 0x7fd1a01044f0 2023-07-21T22:09:20.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd19f7fe700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd190004f40 tx=0x7fd190005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd19001d070 con 0x7fd1a01044f0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a01960a0 con 0x7fd1a01044f0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd190004370 con 0x7fd1a01044f0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1900039b0 con 0x7fd1a01044f0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.642+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01964c0 con 0x7fd1a01044f0 2023-07-21T22:09:20.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd190003c70 con 0x7fd1a01044f0 2023-07-21T22:09:20.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a01027a0 con 0x7fd1a01044f0 2023-07-21T22:09:20.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd19cff9700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 0x7fd18803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:20.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd19004c6e0 con 0x7fd1a01044f0 2023-07-21T22:09:20.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd19effd700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 0x7fd18803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:20.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.643+0000 7fd19effd700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 0x7fd18803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd194006fd0 tx=0x7fd194006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.648+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd190029770 con 0x7fd1a01044f0 2023-07-21T22:09:20.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:20.839+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd1a0067d00 con 0x7fd1a01044f0 2023-07-21T22:09:20.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:20 smithi093 ceph-mon[96579]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:21.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.613+0000 7fd19cff9700 1 -- 172.21.15.93:0/3558849273 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd190026070 con 0x7fd1a01044f0 2023-07-21T22:09:21.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 msgr2=0x7fd18803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:21.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 0x7fd18803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd194006fd0 tx=0x7fd194006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:21.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 msgr2=0x7fd1a0195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:21.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd190004f40 tx=0x7fd190005020 comp rx=0 tx=0).stop 2023-07-21T22:09:21.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 shutdown_connections 2023-07-21T22:09:21.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd1880387b0 0x7fd18803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:21.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 --2- 172.21.15.93:0/3558849273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1a01044f0 0x7fd1a0195960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:21.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 >> 172.21.15.93:0/3558849273 conn(0x7fd1a00742b0 msgr2=0x7fd1a00fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:21.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 shutdown_connections 2023-07-21T22:09:21.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:21.615+0000 7fd1a5fa6700 1 -- 172.21.15.93:0/3558849273 wait complete. 2023-07-21T22:09:21.657 INFO:teuthology.orchestra.run.smithi093.stderr:adjusted tunables profile to default 2023-07-21T22:09:21.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:09:21.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:21.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:21.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:21.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:21.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:09:21.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:21.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:22.030 INFO:tasks.cephadm:Adding mon.a on smithi093 2023-07-21T22:09:22.031 INFO:tasks.cephadm:Adding mon.b on smithi134 2023-07-21T22:09:22.031 INFO:tasks.cephadm:Adding mon.c on smithi194 2023-07-21T22:09:22.031 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c' 2023-07-21T22:09:22.605 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:09:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:22.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:22.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:22.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:23.829 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.786+0000 7fb5b271e700 1 -- 172.21.15.194:0/1087699102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 msgr2=0x7fb5ac0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.786+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1087699102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac0fe820 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb594009910 tx=0x7fb594009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 -- 172.21.15.194:0/1087699102 shutdown_connections 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1087699102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac0fe820 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 -- 172.21.15.194:0/1087699102 >> 172.21.15.194:0/1087699102 conn(0x7fb5ac074f10 msgr2=0x7fb5ac075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 -- 172.21.15.194:0/1087699102 shutdown_connections 2023-07-21T22:09:23.830 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 -- 172.21.15.194:0/1087699102 wait complete. 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 Processor -- start 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.787+0000 7fb5b271e700 1 -- start start 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5b271e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5b271e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac19ad10 con 0x7fb5ac0fe450 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.194:51116/0 (socket says 172.21.15.194:51116) 2023-07-21T22:09:23.831 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5abfff700 1 -- 172.21.15.194:0/1899076440 learned_addr learned my addr 172.21.15.194:0/1899076440 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:09:23.832 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5abfff700 1 -- 172.21.15.194:0/1899076440 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940095f0 con 0x7fb5ac0fe450 2023-07-21T22:09:23.832 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.788+0000 7fb5abfff700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb594006010 tx=0x7fb594004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:23.833 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb59401d070 con 0x7fb5ac0fe450 2023-07-21T22:09:23.833 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac19af10 con 0x7fb5ac0fe450 2023-07-21T22:09:23.833 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb59400bc10 con 0x7fb5ac0fe450 2023-07-21T22:09:23.833 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb594003910 con 0x7fb5ac0fe450 2023-07-21T22:09:23.833 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac194850 con 0x7fb5ac0fe450 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.789+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb59400bd80 con 0x7fb5ac0fe450 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.790+0000 7fb5a97fa700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 0x7fb59803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.790+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac061b40 con 0x7fb5ac0fe450 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.790+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb59404c5c0 con 0x7fb5ac0fe450 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.790+0000 7fb5ab7fe700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 0x7fb59803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:23.834 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.790+0000 7fb5ab7fe700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 0x7fb59803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb59c006fd0 tx=0x7fb59c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:23.836 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.796+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb594026020 con 0x7fb5ac0fe450 2023-07-21T22:09:23.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:23 smithi093 ceph-mon[96579]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:23.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:23 smithi093 ceph-mon[96579]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:09:24.029 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.988+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5ac194a40 con 0x7fb5980387b0 2023-07-21T22:09:24.038 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.998+0000 7fb5a97fa700 1 -- 172.21.15.194:0/1899076440 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb5ac194a40 con 0x7fb5980387b0 2023-07-21T22:09:24.038 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2023-07-21T22:09:24.040 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 msgr2=0x7fb59803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:24.040 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 0x7fb59803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb59c006fd0 tx=0x7fb59c006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:24.040 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 msgr2=0x7fb5ac19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:24.040 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb594006010 tx=0x7fb594004ff0 comp rx=0 tx=0).stop 2023-07-21T22:09:24.040 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 shutdown_connections 2023-07-21T22:09:24.041 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb5980387b0 0x7fb59803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:24.041 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 --2- 172.21.15.194:0/1899076440 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe450 0x7fb5ac19a7d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:24.041 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 >> 172.21.15.194:0/1899076440 conn(0x7fb5ac074f10 msgr2=0x7fb5ac103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:24.041 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 shutdown_connections 2023-07-21T22:09:24.041 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:23.999+0000 7fb5b271e700 1 -- 172.21.15.194:0/1899076440 wait complete. 2023-07-21T22:09:24.470 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.b.service 2023-07-21T22:09:24.472 DEBUG:teuthology.orchestra.run.smithi194:mon.c> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.c.service 2023-07-21T22:09:24.474 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:09:24.475 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:09:24.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:09:24.507 INFO:journalctl@ceph.mon.b.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:09:24.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:24.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:24.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c;count:3 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:24 smithi093 ceph-mon[96579]: Deploying daemon mon.c on smithi194 2023-07-21T22:09:25.148 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:25.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:25 smithi093 ceph-mon[96579]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.161 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 -- 172.21.15.194:0/2726894929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 msgr2=0x7fabe81048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:27.161 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 --2- 172.21.15.194:0/2726894929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe81048c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009910 tx=0x7fabd8009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:27.161 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 -- 172.21.15.194:0/2726894929 shutdown_connections 2023-07-21T22:09:27.161 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 --2- 172.21.15.194:0/2726894929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe81048c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 -- 172.21.15.194:0/2726894929 >> 172.21.15.194:0/2726894929 conn(0x7fabe80fa010 msgr2=0x7fabe80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 -- 172.21.15.194:0/2726894929 shutdown_connections 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 -- 172.21.15.194:0/2726894929 wait complete. 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.120+0000 7fabed682700 1 Processor -- start 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabed682700 1 -- start start 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabed682700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:27.162 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabed682700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe8195e80 con 0x7fabe81044f0 2023-07-21T22:09:27.163 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabe6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabe6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.194:51122/0 (socket says 172.21.15.194:51122) 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.121+0000 7fabe6ffd700 1 -- 172.21.15.194:0/278764859 learned_addr learned my addr 172.21.15.194:0/278764859 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabe6ffd700 1 -- 172.21.15.194:0/278764859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80095f0 con 0x7fabe81044f0 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabe6ffd700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd8004f40 tx=0x7fabd8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabd801d070 con 0x7fabe81044f0 2023-07-21T22:09:27.164 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe8196080 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabd8004370 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fabd80039b0 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.122+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe81964a0 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.123+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fabd8003c70 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.123+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe81027a0 con 0x7fabe81044f0 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.123+0000 7fabcffff700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 0x7fabd003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:27.165 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.123+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fabd804c770 con 0x7fabe81044f0 2023-07-21T22:09:27.166 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.124+0000 7fabe67fc700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 0x7fabd003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:27.166 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.124+0000 7fabe67fc700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 0x7fabd003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:27.168 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.128+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fabd8029770 con 0x7fabe81044f0 2023-07-21T22:09:27.449 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.408+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fabe80685b0 con 0x7fabe81044f0 2023-07-21T22:09:27.449 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.409+0000 7fabcffff700 1 -- 172.21.15.194:0/278764859 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fabd8026070 con 0x7fabe81044f0 2023-07-21T22:09:27.450 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:09:27.450 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:07:59.807368Z","created":"2023-07-21T22:07:59.807368Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-21T22:09:27.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 msgr2=0x7fabd003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:27.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 0x7fabd003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006fd0 tx=0x7fabdc006e40 comp rx=0 tx=0).stop 2023-07-21T22:09:27.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 msgr2=0x7fabe8195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fabd8004f40 tx=0x7fabd8005020 comp rx=0 tx=0).stop 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 shutdown_connections 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fabd00387b0 0x7fabd003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 --2- 172.21.15.194:0/278764859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabe81044f0 0x7fabe8195940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 >> 172.21.15.194:0/278764859 conn(0x7fabe80fa010 msgr2=0x7fabe8106850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 shutdown_connections 2023-07-21T22:09:27.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:27.411+0000 7fabed682700 1 -- 172.21.15.194:0/278764859 wait complete. 2023-07-21T22:09:27.453 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2023-07-21T22:09:27.638 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 podman[98226]: 2023-07-21T22:09:27.912 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:09:27.912 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:09:27.912 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 bash[98226]: a441c3d0057d1228d6e9e0c512aa76e047f624c941b482b193a1e0c5bba42b2e 2023-07-21T22:09:27.912 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pidfile_write: ignore empty --pid-file 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: load: jerasure load: lrc 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Git sha 0 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: DB SUMMARY 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: DB Session ID: B7K6O60TCDPIJY6GVWIK 2023-07-21T22:09:27.913 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:09:27.914 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.env: 0x5631290c0720 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.info_log: 0x56312b939260 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:09:27.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.statistics: (nil) 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.use_fsync: 0 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:09:27.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.db_log_dir: 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.wal_dir: 2023-07-21T22:09:27.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:09:27.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.write_buffer_manager: 0x56312b85a3c0 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:09:27.919 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.unordered_write: 0 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.row_cache: None 2023-07-21T22:09:27.920 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.wal_filter: None 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.wal_compression: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:09:27.921 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:09:27.922 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:09:27.923 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:09:27.923 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:09:27.923 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:09:27.924 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_open_files: -1 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:09:27.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Compression algorithms supported: 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kZSTD supported: 0 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:09:27.926 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:09:27.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.merge_operator: 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_filter: None 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:09:27.928 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56312b939360) 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_type: 0 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: data_block_index_type: 0 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_shortening: 1 2023-07-21T22:09:27.929 INFO:journalctl@ceph.mon.c.smithi194.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: checksum: 4 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: no_block_cache: 0 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache: 0x56312b8f91f0 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_options: 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: capacity : 536870912 2023-07-21T22:09:27.930 INFO:journalctl@ceph.mon.c.smithi194.stdout: num_shard_bits : 4 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: strict_capacity_limit : 0 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_compressed: (nil) 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: persistent_cache: (nil) 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_size: 4096 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_size_deviation: 10 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_restart_interval: 16 2023-07-21T22:09:27.931 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_block_restart_interval: 1 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: metadata_block_size: 4096 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: partition_filters: 0 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: use_delta_encoding: 1 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: filter_policy: bloomfilter 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: whole_key_filtering: 1 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: verify_compression: 0 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:09:27.932 INFO:journalctl@ceph.mon.c.smithi194.stdout: format_version: 5 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: enable_index_compression: 1 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_align: 0 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: max_auto_readahead_size: 262144 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: prepopulate_block_cache: 0 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:09:27.933 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:09:27.934 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression: NoCompression 2023-07-21T22:09:27.936 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:09:27.936 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:09:27.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:09:27.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.num_levels: 7 2023-07-21T22:09:27.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:09:27.938 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:09:27.938 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:09:27.938 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:09:27.939 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:09:27.939 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:09:27.939 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:09:27.940 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:09:27.941 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:09:27.941 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:09:27.942 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:09:27.942 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:09:27.942 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:09:27.942 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:09:27.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:09:27.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:09:27.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:09:27.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:09:27.943 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:09:27.944 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:09:27.944 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:09:27.945 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:09:27.945 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:09:27.945 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:09:27.945 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:09:27.946 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:09:27.946 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:09:27.946 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:09:27.946 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:09:27.947 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:09:27.947 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:09:27.947 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:09:27.948 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:09:27.948 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:09:27.948 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:09:27.948 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:09:27.949 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:09:27.949 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:09:27.949 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:09:27.950 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:09:27.950 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:09:27.950 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:09:27.950 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:09:27.951 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:09:27.951 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:09:27.952 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:09:27.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:09:27.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:09:27.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:09:27.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:09:27.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:09:27.954 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:09:27.954 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:09:27.954 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:09:27.954 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:09:27.954 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:09:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:09:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:09:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:09:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:09:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.ttl: 2592000 2023-07-21T22:09:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:09:27.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:09:27.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:09:27.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:09:27.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:09:27.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c3d1264f-ecd3-44ca-be49-4da51de94fbc 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977367604968, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:09:27.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977367605385, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977367, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c3d1264f-ecd3-44ca-be49-4da51de94fbc", "db_session_id": "B7K6O60TCDPIJY6GVWIK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977367605451, "job": 1, "event": "recovery_finished"} 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56312b992000 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: DB pointer 0x56312b97e000 2023-07-21T22:09:27.960 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** DB Stats ** 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:09:27.961 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** Compaction Stats [default] ** 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:09:27.962 INFO:journalctl@ceph.mon.c.smithi194.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** Compaction Stats [default] ** 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:09:27.963 INFO:journalctl@ceph.mon.c.smithi194.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:09:27.964 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: Block cache BinnedLRUCache@0x56312b8f91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T22:09:27.965 INFO:journalctl@ceph.mon.c.smithi194.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: using public_addr v2:172.21.15.194:0/0 -> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: starting mon.c rank -1 at public addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] at bind addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(???) e0 preinit fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:27.966 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).mds e1 new map 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: e1 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: legacy client fscid: -1 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2023-07-21T22:09:27.967 INFO:journalctl@ceph.mon.c.smithi194.stdout: No filesystems configured 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:09:27.968 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:09:27.969 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:09:27.970 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:27.971 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.972 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c;count:3 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:27.973 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: Deploying daemon mon.c on smithi194 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/278764859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:09:27.974 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expand map: {default=false} 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expand map: {default=info} 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:09:27.975 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 ceph-mon[98288]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:09:27.976 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:27 smithi194 systemd[1]: Started Ceph mon.c for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:09:27.977 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:27 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/278764859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:28.952 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:09:28.953 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:09:29.514 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:09:33.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:33.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: mon.a calling monitor election 2023-07-21T22:09:33.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:33.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: mon.c calling monitor election 2023-07-21T22:09:33.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:33.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:33.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:33.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: fsmap 2023-07-21T22:09:33.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:33.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: mgrmap e13: x(active, since 41s) 2023-07-21T22:09:33.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: overall HEALTH_OK 2023-07-21T22:09:33.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:33.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:33.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:33.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: mon.a calling monitor election 2023-07-21T22:09:33.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:33.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: mon.c calling monitor election 2023-07-21T22:09:33.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:33.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:33.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:33.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:33.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: fsmap 2023-07-21T22:09:33.193 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:33.193 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: mgrmap e13: x(active, since 41s) 2023-07-21T22:09:33.193 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: overall HEALTH_OK 2023-07-21T22:09:33.193 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.194 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.194 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:33.194 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:33.194 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:33.783 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/1849006249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c104660 msgr2=0x7f0b8c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:33.784 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/1849006249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c104660 0x7f0b8c005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005fa0 tx=0x7f0b9000ffc0 comp rx=0 tx=0).stop 2023-07-21T22:09:33.784 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/1849006249 shutdown_connections 2023-07-21T22:09:33.784 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/1849006249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c104660 0x7f0b8c005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:33.784 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/1849006249 >> 172.21.15.194:0/1849006249 conn(0x7f0b9c0fa140 msgr2=0x7f0b9c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:33.784 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/1849006249 shutdown_connections 2023-07-21T22:09:33.785 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.743+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/1849006249 wait complete. 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.744+0000 7f0ba2c4a700 1 Processor -- start 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba2c4a700 1 -- start start 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba2c4a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba2c4a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 0x7f0b9c196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba2c4a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c19e930 con 0x7f0b9c195e00 2023-07-21T22:09:33.786 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba2c4a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c19ea70 con 0x7f0b9c104660 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:46962/0 (socket says 172.21.15.194:46962) 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 -- 172.21.15.194:0/535708841 learned_addr learned my addr 172.21.15.194:0/535708841 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 -- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 msgr2=0x7f0b9c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 -- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 msgr2=0x7f0b9c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0ba09e6700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:09:33.787 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.745+0000 7f0b9bfff700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 0x7f0b9c196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:33.788 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b9bfff700 1 -- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 msgr2=0x7f0b9c1958c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:09:33.788 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b9bfff700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:33.788 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b9bfff700 1 -- 172.21.15.194:0/535708841 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9000f9c0 con 0x7f0b9c195e00 2023-07-21T22:09:33.788 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b9bfff700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 0x7f0b9c196270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b4d0 tx=0x7f0b8800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:33.789 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0b88011630 con 0x7f0b9c195e00 2023-07-21T22:09:33.789 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b88011c50 con 0x7f0b9c195e00 2023-07-21T22:09:33.789 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0b88010c60 con 0x7f0b9c195e00 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.746+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c19ec70 con 0x7f0b9c195e00 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.747+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c19f210 con 0x7f0b9c195e00 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.747+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0b8801a410 con 0x7f0b9c195e00 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.747+0000 7f0b99ffb700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 0x7f0b8003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.747+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b8804a7a0 con 0x7f0b9c195e00 2023-07-21T22:09:33.790 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.748+0000 7f0ba09e6700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 0x7f0b8003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:33.791 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.748+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c061b40 con 0x7f0b9c195e00 2023-07-21T22:09:33.791 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.748+0000 7f0ba09e6700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 0x7f0b8003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b900059d0 tx=0x7f0b90005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:33.793 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:33.753+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0b88018720 con 0x7f0b9c195e00 2023-07-21T22:09:34.054 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.014+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b9c19f5c0 con 0x7f0b9c195e00 2023-07-21T22:09:34.054 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.014+0000 7f0b99ffb700 1 -- 172.21.15.194:0/535708841 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f0b8803f090 con 0x7f0b9c195e00 2023-07-21T22:09:34.055 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:09:34.055 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:09:27.630964Z","created":"2023-07-21T22:07:59.807368Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T22:09:34.056 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 msgr2=0x7f0b8003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:34.056 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 0x7f0b8003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b900059d0 tx=0x7f0b90005940 comp rx=0 tx=0).stop 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 msgr2=0x7f0b9c196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 0x7f0b9c196270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b4d0 tx=0x7f0b8800b890 comp rx=0 tx=0).stop 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 shutdown_connections 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b800385f0 0x7f0b8003aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b9c104660 0x7f0b9c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 --2- 172.21.15.194:0/535708841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b9c195e00 0x7f0b9c196270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:34.057 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 >> 172.21.15.194:0/535708841 conn(0x7f0b9c0fa140 msgr2=0x7f0b9c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:34.058 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 shutdown_connections 2023-07-21T22:09:34.058 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:34.016+0000 7f0ba2c4a700 1 -- 172.21.15.194:0/535708841 wait complete. 2023-07-21T22:09:34.058 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2023-07-21T22:09:34.075 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:33 smithi194 ceph-mon[98288]: Deploying daemon mon.b on smithi134 2023-07-21T22:09:34.076 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:33 smithi194 ceph-mon[98288]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:34.076 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:33 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:34.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:33 smithi093 ceph-mon[96579]: Deploying daemon mon.b on smithi134 2023-07-21T22:09:34.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:33 smithi093 ceph-mon[96579]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:34.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:33 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:34.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:34 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/535708841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:34.939 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:09:34 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:09:34.630+0000 7fed4a4e6700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-21T22:09:35.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:34 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/535708841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:35.576 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:09:35.577 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:09:35.581 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 systemd[1]: Starting Ceph mon.b for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:09:35.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 podman[97833]: 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 bash[97833]: 6410d121a71eb832ae0520dc04562f1efa90498f0c992d3125cca7107843a16f 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pidfile_write: ignore empty --pid-file 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: load: jerasure load: lrc 2023-07-21T22:09:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:09:35.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Git sha 0 2023-07-21T22:09:35.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:09:35.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: DB SUMMARY 2023-07-21T22:09:35.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: DB Session ID: 4AQ6N32GYMY4R51XJN6R 2023-07-21T22:09:35.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:09:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.env: 0x555815ca3720 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.info_log: 0x555816dc1260 2023-07-21T22:09:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.statistics: (nil) 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.use_fsync: 0 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:09:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.db_log_dir: 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.wal_dir: 2023-07-21T22:09:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:09:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.write_buffer_manager: 0x555816ce23c0 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:09:35.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.unordered_write: 0 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:09:35.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.row_cache: None 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.wal_filter: None 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.wal_compression: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:09:35.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:09:35.887 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:09:35.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:09:35.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:09:35.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:09:35.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:09:35.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:09:35.889 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_open_files: -1 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:09:35.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Compression algorithms supported: 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kZSTD supported: 0 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:09:35.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:09:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.merge_operator: 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_filter: None 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555816dc1360) 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:09:35.893 INFO:journalctl@ceph.mon.b.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_type: 0 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_index_type: 0 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_shortening: 1 2023-07-21T22:09:35.894 INFO:journalctl@ceph.mon.b.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:09:35.895 INFO:journalctl@ceph.mon.b.smithi134.stdout: checksum: 4 2023-07-21T22:09:35.895 INFO:journalctl@ceph.mon.b.smithi134.stdout: no_block_cache: 0 2023-07-21T22:09:35.895 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache: 0x555816d811f0 2023-07-21T22:09:35.896 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:09:35.896 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_options: 2023-07-21T22:09:35.896 INFO:journalctl@ceph.mon.b.smithi134.stdout: capacity : 536870912 2023-07-21T22:09:35.896 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_shard_bits : 4 2023-07-21T22:09:35.897 INFO:journalctl@ceph.mon.b.smithi134.stdout: strict_capacity_limit : 0 2023-07-21T22:09:35.897 INFO:journalctl@ceph.mon.b.smithi134.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:09:35.897 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_cache_compressed: (nil) 2023-07-21T22:09:35.898 INFO:journalctl@ceph.mon.b.smithi134.stdout: persistent_cache: (nil) 2023-07-21T22:09:35.898 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size: 4096 2023-07-21T22:09:35.898 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_size_deviation: 10 2023-07-21T22:09:35.898 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_restart_interval: 16 2023-07-21T22:09:35.899 INFO:journalctl@ceph.mon.b.smithi134.stdout: index_block_restart_interval: 1 2023-07-21T22:09:35.899 INFO:journalctl@ceph.mon.b.smithi134.stdout: metadata_block_size: 4096 2023-07-21T22:09:35.899 INFO:journalctl@ceph.mon.b.smithi134.stdout: partition_filters: 0 2023-07-21T22:09:35.900 INFO:journalctl@ceph.mon.b.smithi134.stdout: use_delta_encoding: 1 2023-07-21T22:09:35.900 INFO:journalctl@ceph.mon.b.smithi134.stdout: filter_policy: bloomfilter 2023-07-21T22:09:35.900 INFO:journalctl@ceph.mon.b.smithi134.stdout: whole_key_filtering: 1 2023-07-21T22:09:35.900 INFO:journalctl@ceph.mon.b.smithi134.stdout: verify_compression: 0 2023-07-21T22:09:35.901 INFO:journalctl@ceph.mon.b.smithi134.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:09:35.901 INFO:journalctl@ceph.mon.b.smithi134.stdout: format_version: 5 2023-07-21T22:09:35.901 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_index_compression: 1 2023-07-21T22:09:35.901 INFO:journalctl@ceph.mon.b.smithi134.stdout: block_align: 0 2023-07-21T22:09:35.902 INFO:journalctl@ceph.mon.b.smithi134.stdout: max_auto_readahead_size: 262144 2023-07-21T22:09:35.902 INFO:journalctl@ceph.mon.b.smithi134.stdout: prepopulate_block_cache: 0 2023-07-21T22:09:35.902 INFO:journalctl@ceph.mon.b.smithi134.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:09:35.902 INFO:journalctl@ceph.mon.b.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:09:35.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:09:35.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:09:35.903 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression: NoCompression 2023-07-21T22:09:35.904 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:09:35.904 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:09:35.904 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:09:35.904 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.num_levels: 7 2023-07-21T22:09:35.905 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:09:35.905 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:09:35.905 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:09:35.905 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:09:35.906 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:09:35.906 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:09:35.906 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:09:35.908 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:09:35.909 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:09:35.909 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:09:35.909 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:09:35.909 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:09:35.910 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:09:35.910 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:09:35.910 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:09:35.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:09:35.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:09:35.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:09:35.911 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:09:35.912 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:09:35.912 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:09:35.912 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:09:35.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:09:35.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:09:35.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:09:35.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:09:35.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:09:35.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:09:35.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:09:35.915 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:09:35.915 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:09:35.915 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:09:35.915 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:09:35.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:09:35.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:09:35.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:09:35.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:09:35.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:09:35.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:09:35.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:09:35.918 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:09:35.918 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:09:35.918 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:09:35.918 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:09:35.919 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:09:35.919 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:09:35.919 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:09:35.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:09:35.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:09:35.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:09:35.920 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:09:35.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:09:35.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:09:35.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:09:35.921 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:09:35.922 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:09:35.922 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:09:35.922 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:09:35.922 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:09:35.923 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:09:35.923 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:09:35.923 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:09:35.924 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:09:35.924 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:09:35.924 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:09:35.924 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.ttl: 2592000 2023-07-21T22:09:35.925 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:09:35.925 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:09:35.925 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:09:35.926 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:09:35.926 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:09:35.926 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:09:35.926 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:09:35.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:09:35.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:09:35.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:09:35.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:09:35.927 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:09:35.928 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:09:35.928 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:09:35.928 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:09:35.929 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 36c14844-f98d-424f-b6cb-88063012ab75 2023-07-21T22:09:35.929 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977375737066, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:09:35.929 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:09:35.929 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977375737629, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977375, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "36c14844-f98d-424f-b6cb-88063012ab75", "db_session_id": "4AQ6N32GYMY4R51XJN6R", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:09:35.930 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977375737719, "job": 1, "event": "recovery_finished"} 2023-07-21T22:09:35.930 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:09:35.930 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:09:35.930 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555816e1a000 2023-07-21T22:09:35.931 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: DB pointer 0x555816e06000 2023-07-21T22:09:35.931 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:09:35.931 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:09:35.931 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** DB Stats ** 2023-07-21T22:09:35.932 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:09:35.932 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:35.932 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:35.933 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:09:35.933 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:09:35.933 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:09:35.933 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:09:35.934 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.934 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-07-21T22:09:35.934 INFO:journalctl@ceph.mon.b.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:09:35.934 INFO:journalctl@ceph.mon.b.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:09:35.935 INFO:journalctl@ceph.mon.b.smithi134.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:09:35.935 INFO:journalctl@ceph.mon.b.smithi134.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:09:35.935 INFO:journalctl@ceph.mon.b.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:09:35.936 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.936 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** Compaction Stats [default] ** 2023-07-21T22:09:35.936 INFO:journalctl@ceph.mon.b.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:09:35.936 INFO:journalctl@ceph.mon.b.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:09:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:09:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.937 INFO:journalctl@ceph.mon.b.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:09:35.938 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.938 INFO:journalctl@ceph.mon.b.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:09:35.938 INFO:journalctl@ceph.mon.b.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:09:35.938 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:09:35.939 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:09:35.939 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:09:35.939 INFO:journalctl@ceph.mon.b.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:09:35.940 INFO:journalctl@ceph.mon.b.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:09:35.940 INFO:journalctl@ceph.mon.b.smithi134.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:09:35.940 INFO:journalctl@ceph.mon.b.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:09:35.940 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache BinnedLRUCache@0x555816d811f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-21T22:09:35.941 INFO:journalctl@ceph.mon.b.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:09:35.941 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.941 INFO:journalctl@ceph.mon.b.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:09:35.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:09:35.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2023-07-21T22:09:35.942 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:35.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(???) e0 preinit fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:35.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 systemd[1]: Started Ceph mon.b for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:09:35.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).mds e1 new map 2023-07-21T22:09:35.943 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-21T22:09:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout: e1 2023-07-21T22:09:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:09:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:09:35.944 INFO:journalctl@ceph.mon.b.smithi134.stdout: legacy client fscid: -1 2023-07-21T22:09:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout: 2023-07-21T22:09:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout: No filesystems configured 2023-07-21T22:09:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:09:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:09:35.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:09:35.946 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:35.948 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3558849273' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.949 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c;count:3 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.950 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Deploying daemon mon.c on smithi194 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/278764859' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.a calling monitor election 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.951 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.c calling monitor election 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.952 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: fsmap 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mgrmap e13: x(active, since 41s) 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: overall HEALTH_OK 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.953 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: Deploying daemon mon.b on smithi134 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/535708841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:35.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expand map: {default=false} 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expand map: {default=info} 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:09:35.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:09:35.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:35 smithi134 ceph-mon[97873]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:09:36.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:35 smithi194 ceph-mon[98288]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:36.143 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:09:36.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:35 smithi093 ceph-mon[96579]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:41.003 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 -- 172.21.15.194:0/121681185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60800035c0 msgr2=0x7f6080005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:41.004 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 --2- 172.21.15.194:0/121681185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60800035c0 0x7f6080005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6098077990 tx=0x7f608c014ec0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.004 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 -- 172.21.15.194:0/121681185 shutdown_connections 2023-07-21T22:09:41.005 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 --2- 172.21.15.194:0/121681185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60800035c0 0x7f6080005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.005 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 -- 172.21.15.194:0/121681185 >> 172.21.15.194:0/121681185 conn(0x7f6098073d70 msgr2=0x7f6098074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:41.005 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 -- 172.21.15.194:0/121681185 shutdown_connections 2023-07-21T22:09:41.005 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.962+0000 7f609cce1700 1 -- 172.21.15.194:0/121681185 wait complete. 2023-07-21T22:09:41.006 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.963+0000 7f609cce1700 1 Processor -- start 2023-07-21T22:09:41.006 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.963+0000 7f609cce1700 1 -- start start 2023-07-21T22:09:41.006 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:41.006 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 0x7f6098199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:41.008 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:41.008 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981959c0 con 0x7f60981952f0 2023-07-21T22:09:41.008 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981a19c0 con 0x7f60800035c0 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609cce1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981a1b00 con 0x7f6098199c50 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:46988/0 (socket says 172.21.15.194:46988) 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 -- 172.21.15.194:0/995072014 learned_addr learned my addr 172.21.15.194:0/995072014 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 0x7f6098199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 msgr2=0x7f609819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-21T22:09:41.009 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609659c700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 msgr2=0x7f609819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6096d9d700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609659c700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 msgr2=0x7f6098194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609659c700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 msgr2=0x7f6098194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609659c700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:09:41.010 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f609659c700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 msgr2=0x7f609819a0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 msgr2=0x7f6098194db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.964+0000 7f6095d9b700 1 -- 172.21.15.194:0/995072014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c014910 con 0x7f60981952f0 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f6095d9b700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 0x7f6098199710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f608800b580 tx=0x7f608800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088004230 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6088004390 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60981a1db0 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088011a70 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.965+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a2280 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.966+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6088011c10 con 0x7f60981952f0 2023-07-21T22:09:41.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.966+0000 7f60877fe700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 0x7f607803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:41.013 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.966+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f608804bcf0 con 0x7f60981952f0 2023-07-21T22:09:41.013 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.966+0000 7f609659c700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 0x7f607803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:41.013 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.966+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609818f190 con 0x7f60981952f0 2023-07-21T22:09:41.013 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.967+0000 7f609659c700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 0x7f607803b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f608c010010 tx=0x7f608c001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:41.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:40.971+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f608801a400 con 0x7f60981952f0 2023-07-21T22:09:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: mon.c calling monitor election 2023-07-21T22:09:41.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: mon.a calling monitor election 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: fsmap 2023-07-21T22:09:41.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: mgrmap e13: x(active, since 49s) 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.295 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.254+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f609818eb70 con 0x7f60981952f0 2023-07-21T22:09:41.296 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.255+0000 7f60877fe700 1 -- 172.21.15.194:0/995072014 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f6088049020 con 0x7f60981952f0 2023-07-21T22:09:41.296 INFO:teuthology.orchestra.run.smithi194.stdout: 2023-07-21T22:09:41.297 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":3,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:09:35.767446Z","created":"2023-07-21T22:07:59.807368Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T22:09:41.298 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 msgr2=0x7f607803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:41.298 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 0x7f607803b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f608c010010 tx=0x7f608c001e50 comp rx=0 tx=0).stop 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 msgr2=0x7f6098199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 0x7f6098199710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f608800b580 tx=0x7f608800b940 comp rx=0 tx=0).stop 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 shutdown_connections 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098199c50 0x7f609819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6078038c30 0x7f607803b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60800035c0 0x7f6098194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.299 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 --2- 172.21.15.194:0/995072014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60981952f0 0x7f6098199710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:41.300 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 >> 172.21.15.194:0/995072014 conn(0x7f6098073d70 msgr2=0x7f60980fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:41.300 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 shutdown_connections 2023-07-21T22:09:41.300 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:41.257+0000 7f609cce1700 1 -- 172.21.15.194:0/995072014 wait complete. 2023-07-21T22:09:41.300 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 3 2023-07-21T22:09:41.323 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: mon.c calling monitor election 2023-07-21T22:09:41.323 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:41.323 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: mon.a calling monitor election 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.324 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: fsmap 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: mgrmap e13: x(active, since 49s) 2023-07-21T22:09:41.325 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.326 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:41.696 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-21T22:09:41.697 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-21T22:09:42.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:42.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:41 smithi194 ceph-mon[98288]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:42.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:42.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:41 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/995072014' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:42.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:42.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:42.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:41 smithi093 ceph-mon[96579]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:42.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:42.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:41 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/995072014' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:42.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:42.267 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:43.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: mon.c calling monitor election 2023-07-21T22:09:43.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:09:43.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: mon.a calling monitor election 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: fsmap 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: mgrmap e13: x(active, since 49s) 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:09:43.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: mon.b (rank 2) addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] is down (out of quorum) 2023-07-21T22:09:43.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:43.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:43.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/995072014' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:09:43.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:43.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 -- 172.21.15.93:0/3840564097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 msgr2=0x7fbb48105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:43.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 --2- 172.21.15.93:0/3840564097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48105a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009910 tx=0x7fbb44009c20 comp rx=0 tx=0).stop 2023-07-21T22:09:43.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 -- 172.21.15.93:0/3840564097 shutdown_connections 2023-07-21T22:09:43.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 --2- 172.21.15.93:0/3840564097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48105a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 -- 172.21.15.93:0/3840564097 >> 172.21.15.93:0/3840564097 conn(0x7fbb480fb130 msgr2=0x7fbb480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:43.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 -- 172.21.15.93:0/3840564097 shutdown_connections 2023-07-21T22:09:43.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.414+0000 7fbb4fead700 1 -- 172.21.15.93:0/3840564097 wait complete. 2023-07-21T22:09:43.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 Processor -- start 2023-07-21T22:09:43.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 -- start start 2023-07-21T22:09:43.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:43.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb48072d50 0x7fbb480731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb481a0d60 0x7fbb481a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a3870 con 0x7fbb48105630 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a39e0 con 0x7fbb481a0d60 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4fead700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a3ba0 con 0x7fbb48072d50 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.415+0000 7fbb4dc49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41180/0 (socket says 172.21.15.93:41180) 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 -- 172.21.15.93:0/1315770331 learned_addr learned my addr 172.21.15.93:0/1315770331 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:43.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4d448700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb48072d50 0x7fbb480731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 -- 172.21.15.93:0/1315770331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb48072d50 msgr2=0x7fbb480731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb48072d50 0x7fbb480731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 -- 172.21.15.93:0/1315770331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb481a0d60 msgr2=0x7fbb481a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb481a0d60 0x7fbb481a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 -- 172.21.15.93:0/1315770331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440095f0 con 0x7fbb48105630 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4dc49700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb44004c70 tx=0x7fbb44004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:43.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4401d070 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb481a3e20 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4400be20 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44022dd0 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.416+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a4240 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.417+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbb4400f460 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.417+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48061b40 con 0x7fbb48105630 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.417+0000 7fbb3effd700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 0x7fbb3403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:43.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.417+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb44022470 con 0x7fbb48105630 2023-07-21T22:09:43.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.417+0000 7fbb4d448700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 0x7fbb3403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:43.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.418+0000 7fbb4d448700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 0x7fbb3403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009850 tx=0x7fbb38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:43.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.423+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbb44020960 con 0x7fbb48105630 2023-07-21T22:09:43.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.617+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbb48073d40 con 0x7fbb48105630 2023-07-21T22:09:43.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.617+0000 7fbb3effd700 1 -- 172.21.15.93:0/1315770331 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fbb44048130 con 0x7fbb48105630 2023-07-21T22:09:43.659 INFO:teuthology.orchestra.run.smithi093.stdout:# minimal ceph.conf for de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:43.659 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-07-21T22:09:43.660 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:09:43.660 INFO:teuthology.orchestra.run.smithi093.stdout: mon_host = [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2023-07-21T22:09:43.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 msgr2=0x7fbb3403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:43.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 0x7fbb3403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009850 tx=0x7fbb38008040 comp rx=0 tx=0).stop 2023-07-21T22:09:43.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 msgr2=0x7fbb48072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:43.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb44004c70 tx=0x7fbb44004d50 comp rx=0 tx=0).stop 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 shutdown_connections 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbb34038c80 0x7fbb3403b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb481a0d60 0x7fbb481a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb48105630 0x7fbb48072810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 --2- 172.21.15.93:0/1315770331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb48072d50 0x7fbb480731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 >> 172.21.15.93:0/1315770331 conn(0x7fbb480fb130 msgr2=0x7fbb480fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:43.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 shutdown_connections 2023-07-21T22:09:43.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:43.619+0000 7fbb4fead700 1 -- 172.21.15.93:0/1315770331 wait complete. 2023-07-21T22:09:43.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mon.b calling monitor election 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mon.b calling monitor election 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mon.c calling monitor election 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mon.a calling monitor election 2023-07-21T22:09:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: fsmap 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: mgrmap e13: x(active, since 51s) 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:09:43.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Cluster is now healthy 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: overall HEALTH_OK 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:43.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: Reconfiguring daemon mon.a on smithi093 2023-07-21T22:09:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:43 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1315770331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:44.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mon.b calling monitor election 2023-07-21T22:09:44.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mon.b calling monitor election 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mon.c calling monitor election 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mon.a calling monitor election 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:09:44.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: fsmap 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: mgrmap e13: x(active, since 51s) 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Cluster is now healthy 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: overall HEALTH_OK 2023-07-21T22:09:44.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:44.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: Reconfiguring daemon mon.a on smithi093 2023-07-21T22:09:44.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:43 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1315770331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:44.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mon.b calling monitor election 2023-07-21T22:09:44.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mon.b calling monitor election 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mon.c calling monitor election 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mon.a calling monitor election 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:09:44.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: fsmap 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: mgrmap e13: x(active, since 51s) 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Cluster is now healthy 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: overall HEALTH_OK 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:44.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: Reconfiguring daemon mon.a on smithi093 2023-07-21T22:09:44.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:43 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1315770331' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:44.131 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-21T22:09:44.132 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:09:44.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:09:44.160 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:09:44.161 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:44.228 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:09:44.229 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:09:44.259 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:09:44.260 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:44.327 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:09:44.327 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:09:44.360 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:09:44.360 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:09:44.428 INFO:tasks.cephadm:Adding mgr.x on smithi093 2023-07-21T22:09:44.428 INFO:tasks.cephadm:Adding mgr.y on smithi134 2023-07-21T22:09:44.428 INFO:tasks.cephadm:Adding mgr.z on smithi194 2023-07-21T22:09:44.429 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi093=x;smithi134=y;smithi194=z' 2023-07-21T22:09:44.938 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:09:44 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:09:44.767+0000 7fed4a4e6700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-21T22:09:44.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:44.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:44.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:45.016 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:09:45.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:45.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:45.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:45.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:09:45.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:45.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:46.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:09:46.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:46.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:46.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:46.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: Reconfiguring daemon mon.b on smithi134 2023-07-21T22:09:46.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:45 smithi194 ceph-mon[98288]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:46.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:09:46.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:46.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:46.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:46.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: Reconfiguring daemon mon.b on smithi134 2023-07-21T22:09:46.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:45 smithi134 ceph-mon[97873]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:46.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:09:46.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:46.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:46.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:46.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: Reconfiguring daemon mon.b on smithi134 2023-07-21T22:09:46.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:45 smithi093 ceph-mon[96579]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:46.641 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.599+0000 7fd905eae700 1 -- 172.21.15.194:0/463415718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 msgr2=0x7fd9000fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:46.641 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.599+0000 7fd905eae700 1 --2- 172.21.15.194:0/463415718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000fea10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b0a0 tx=0x7fd8f800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.642 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.599+0000 7fd905eae700 1 -- 172.21.15.194:0/463415718 shutdown_connections 2023-07-21T22:09:46.642 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.599+0000 7fd905eae700 1 --2- 172.21.15.194:0/463415718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000fea10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.642 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.600+0000 7fd905eae700 1 -- 172.21.15.194:0/463415718 >> 172.21.15.194:0/463415718 conn(0x7fd9000f9bb0 msgr2=0x7fd9000fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:46.642 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.600+0000 7fd905eae700 1 -- 172.21.15.194:0/463415718 shutdown_connections 2023-07-21T22:09:46.642 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.600+0000 7fd905eae700 1 -- 172.21.15.194:0/463415718 wait complete. 2023-07-21T22:09:46.644 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.600+0000 7fd905eae700 1 Processor -- start 2023-07-21T22:09:46.644 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 -- start start 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd90019a150 0x7fd90019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90019cd70 con 0x7fd9000fe600 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90019cee0 con 0x7fd90019a150 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd905eae700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90019d0a0 con 0x7fd900197790 2023-07-21T22:09:46.645 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:46996/0 (socket says 172.21.15.194:46996) 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 -- 172.21.15.194:0/711156710 learned_addr learned my addr 172.21.15.194:0/711156710 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8ff7fe700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8fffff700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd90019a150 0x7fd90019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 -- 172.21.15.194:0/711156710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd90019a150 msgr2=0x7fd90019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd90019a150 0x7fd90019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.646 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 -- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 msgr2=0x7fd9000690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.601+0000 7fd8feffd700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd8feffd700 1 -- 172.21.15.194:0/711156710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8009d00 con 0x7fd900197790 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd8feffd700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800b780 tx=0x7fd8e800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8004230 con 0x7fd900197790 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90019d380 con 0x7fd900197790 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e8004390 con 0x7fd900197790 2023-07-21T22:09:46.647 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8011ad0 con 0x7fd900197790 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.602+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90019d950 con 0x7fd900197790 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.603+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd8e8011c30 con 0x7fd900197790 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.603+0000 7fd8fcff9700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 0x7fd8ec03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.603+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd8e804aca0 con 0x7fd900197790 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.603+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90018d180 con 0x7fd900197790 2023-07-21T22:09:46.648 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.603+0000 7fd8ff7fe700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 0x7fd8ec03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:46.649 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.604+0000 7fd8ff7fe700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 0x7fd8ec03b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8000f80 tx=0x7fd8f8005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:46.649 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.608+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd8e801a400 con 0x7fd900197790 2023-07-21T22:09:46.856 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.815+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi134=y;smithi194=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9000609e0 con 0x7fd8ec038c80 2023-07-21T22:09:46.862 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.821+0000 7fd8fcff9700 1 -- 172.21.15.194:0/711156710 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd9000609e0 con 0x7fd8ec038c80 2023-07-21T22:09:46.863 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mgr update... 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 msgr2=0x7fd8ec03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 0x7fd8ec03b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8000f80 tx=0x7fd8f8005040 comp rx=0 tx=0).stop 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 msgr2=0x7fd900199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800b780 tx=0x7fd8e800ba90 comp rx=0 tx=0).stop 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 shutdown_connections 2023-07-21T22:09:46.865 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd8ec038c80 0x7fd8ec03b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.866 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd90019a150 0x7fd90019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.866 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9000fe600 0x7fd9000690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.866 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 --2- 172.21.15.194:0/711156710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd900197790 0x7fd900199b70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:46.866 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.823+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 >> 172.21.15.194:0/711156710 conn(0x7fd9000f9bb0 msgr2=0x7fd9000fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:46.867 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.824+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 shutdown_connections 2023-07-21T22:09:46.867 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:09:46.824+0000 7fd905eae700 1 -- 172.21.15.194:0/711156710 wait complete. 2023-07-21T22:09:47.370 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.370 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.370 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:09:47.370 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:47.370 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:47.371 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:47.371 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: Reconfiguring daemon mon.c on smithi194 2023-07-21T22:09:47.371 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:09:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: Reconfiguring daemon mon.c on smithi194 2023-07-21T22:09:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:09:47.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:09:47.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:09:47.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:47.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: Reconfiguring daemon mon.c on smithi194 2023-07-21T22:09:47.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:47.666 DEBUG:teuthology.orchestra.run.smithi134:mgr.y> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-21T22:09:47.668 DEBUG:teuthology.orchestra.run.smithi194:mgr.z> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-21T22:09:47.670 INFO:tasks.cephadm:Deploying OSDs... 2023-07-21T22:09:47.671 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:09:47.671 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:09:47.692 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:09:47.693 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2023-07-21T22:09:47.698 INFO:journalctl@ceph.mgr.z.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:09:47.703 INFO:journalctl@ceph.mgr.y.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:09:47.757 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:09:47.757 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:47.757 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 380426 Links: 1 2023-07-21T22:09:47.757 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:47.757 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:47.758 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-21 22:09:01.363821339 +0000 2023-07-21T22:09:47.758 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-21 21:57:57.656283934 +0000 2023-07-21T22:09:47.758 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-21 21:57:57.656283934 +0000 2023-07-21T22:09:47.758 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-21T22:09:47.758 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:09:47.829 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-21T22:09:47.829 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-21T22:09:47.829 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000304838 s, 1.7 MB/s 2023-07-21T22:09:47.831 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:09:47.892 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2023-07-21T22:09:47.952 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 371638 Links: 1 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-21 22:09:01.363821339 +0000 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-21 21:57:58.181273020 +0000 2023-07-21T22:09:47.953 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-21 21:57:58.181273020 +0000 2023-07-21T22:09:47.954 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-21T22:09:47.954 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:09:48.024 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-21T22:09:48.024 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-21T22:09:48.024 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000249117 s, 2.1 MB/s 2023-07-21T22:09:48.025 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:09:48.086 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2023-07-21T22:09:48.148 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 383143 Links: 1 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-21 22:09:01.363821339 +0000 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-21 21:57:58.723261752 +0000 2023-07-21T22:09:48.149 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-21 21:57:58.723261752 +0000 2023-07-21T22:09:48.150 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-21T22:09:48.150 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:09:48.219 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-21T22:09:48.219 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-21T22:09:48.220 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000288146 s, 1.8 MB/s 2023-07-21T22:09:48.220 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:09:48.282 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2023-07-21T22:09:48.343 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 380603 Links: 1 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-21 22:09:01.363821339 +0000 2023-07-21T22:09:48.344 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-21 21:57:59.240251004 +0000 2023-07-21T22:09:48.345 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-21 21:57:59.240251004 +0000 2023-07-21T22:09:48.345 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-07-21T22:09:48.345 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:09:48.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi134=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:48.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: Saving service mgr spec with placement smithi093=x;smithi134=y;smithi194=z;count:3 2023-07-21T22:09:48.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:48.361 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:48.362 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:48.362 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi134=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:48.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: Saving service mgr spec with placement smithi093=x;smithi134=y;smithi194=z;count:3 2023-07-21T22:09:48.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:48.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:48.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi134=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:48.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: Saving service mgr spec with placement smithi093=x;smithi134=y;smithi194=z;count:3 2023-07-21T22:09:48.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:48.414 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:48.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:48.415 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:48.415 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-07-21T22:09:48.415 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-07-21T22:09:48.416 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000219411 s, 2.3 MB/s 2023-07-21T22:09:48.417 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:09:48.478 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-21T22:09:48.479 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:09:48.499 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:09:48.500 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2023-07-21T22:09:48.559 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:09:48.559 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 378583 Links: 1 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-21 22:09:35.212849902 +0000 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-21 21:58:41.109909198 +0000 2023-07-21T22:09:48.560 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-21 21:58:41.109909198 +0000 2023-07-21T22:09:48.561 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-21T22:09:48.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:09:48.628 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-21T22:09:48.629 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-21T22:09:48.629 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000284556 s, 1.8 MB/s 2023-07-21T22:09:48.632 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:09:48.692 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2023-07-21T22:09:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:09:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:48.751 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 380292 Links: 1 2023-07-21T22:09:48.751 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:48.751 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:48.752 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-21 22:09:35.212849902 +0000 2023-07-21T22:09:48.752 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-21 21:58:41.631897036 +0000 2023-07-21T22:09:48.752 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-21 21:58:41.631897036 +0000 2023-07-21T22:09:48.752 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-21T22:09:48.753 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:09:48.819 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-21T22:09:48.819 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-21T22:09:48.819 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000232919 s, 2.2 MB/s 2023-07-21T22:09:48.821 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:09:48.881 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2023-07-21T22:09:48.940 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:09:48.941 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:48.941 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 374417 Links: 1 2023-07-21T22:09:48.941 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:48.941 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:48.941 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-21 22:09:35.212849902 +0000 2023-07-21T22:09:48.942 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-21 21:58:42.128885457 +0000 2023-07-21T22:09:48.942 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-21 21:58:42.128885457 +0000 2023-07-21T22:09:48.942 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-21T22:09:48.942 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:09:49.008 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-21T22:09:49.008 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-21T22:09:49.008 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000242415 s, 2.1 MB/s 2023-07-21T22:09:49.009 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:09:49.068 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2023-07-21T22:09:49.126 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:09:49.127 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:49.127 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 370484 Links: 1 2023-07-21T22:09:49.127 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:49.128 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:49.128 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-21 22:09:35.212849902 +0000 2023-07-21T22:09:49.128 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-21 21:58:42.662873015 +0000 2023-07-21T22:09:49.129 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-21 21:58:42.662873015 +0000 2023-07-21T22:09:49.129 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-21T22:09:49.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:09:49.192 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-21T22:09:49.193 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-21T22:09:49.193 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000195059 s, 2.6 MB/s 2023-07-21T22:09:49.194 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:09:49.252 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2023-07-21T22:09:49.253 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:09:49.271 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:09:49.271 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2023-07-21T22:09:49.331 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:09:49.331 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:49.331 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 370605 Links: 1 2023-07-21T22:09:49.331 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:49.331 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:49.332 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2023-07-21 22:09:49.205713806 +0000 2023-07-21T22:09:49.332 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2023-07-21 21:58:05.531644535 +0000 2023-07-21T22:09:49.332 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2023-07-21 21:58:05.531644535 +0000 2023-07-21T22:09:49.332 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2023-07-21T22:09:49.332 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:09:49.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:49 smithi194 ceph-mon[98288]: Deploying daemon mgr.z on smithi194 2023-07-21T22:09:49.334 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:49 smithi194 systemd[1]: Starting Ceph mgr.z for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:09:49.364 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2023-07-21T22:09:49.364 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2023-07-21T22:09:49.365 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000233016 s, 2.2 MB/s 2023-07-21T22:09:49.366 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:09:49.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:49 smithi134 ceph-mon[97873]: Deploying daemon mgr.z on smithi194 2023-07-21T22:09:49.425 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2023-07-21T22:09:49.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:49 smithi093 ceph-mon[96579]: Deploying daemon mgr.z on smithi194 2023-07-21T22:09:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:09:49.485 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:49.485 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 378493 Links: 1 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2023-07-21 22:09:49.205713806 +0000 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2023-07-21 21:58:06.057633910 +0000 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2023-07-21 21:58:06.057633910 +0000 2023-07-21T22:09:49.486 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2023-07-21T22:09:49.487 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:09:49.550 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2023-07-21T22:09:49.550 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2023-07-21T22:09:49.550 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000216733 s, 2.4 MB/s 2023-07-21T22:09:49.551 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:09:49.609 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2023-07-21T22:09:49.667 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:09:49.668 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:49.668 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 378638 Links: 1 2023-07-21T22:09:49.668 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:49.668 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:49.668 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2023-07-21 22:09:49.205713806 +0000 2023-07-21T22:09:49.669 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2023-07-21 21:58:06.598622983 +0000 2023-07-21T22:09:49.669 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2023-07-21 21:58:06.598622983 +0000 2023-07-21T22:09:49.669 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2023-07-21T22:09:49.669 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:09:49.735 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2023-07-21T22:09:49.735 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2023-07-21T22:09:49.735 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000220994 s, 2.3 MB/s 2023-07-21T22:09:49.736 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:09:49.793 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2023-07-21T22:09:49.850 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 376435 Links: 1 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2023-07-21 22:09:49.205713806 +0000 2023-07-21T22:09:49.851 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2023-07-21 21:58:07.112612601 +0000 2023-07-21T22:09:49.852 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2023-07-21 21:58:07.112612601 +0000 2023-07-21T22:09:49.852 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2023-07-21T22:09:49.852 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:09:49.916 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:49 smithi194 podman[100210]: 2023-07-21T22:09:49.918 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2023-07-21T22:09:49.919 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2023-07-21T22:09:49.919 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000183093 s, 2.8 MB/s 2023-07-21T22:09:49.920 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:09:49.977 INFO:tasks.cephadm:Deploying osd.0 on smithi093 with /dev/vg_nvme/lv_4... 2023-07-21T22:09:49.978 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:09:50.232 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-mon[98288]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:50.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:49 smithi194 ceph-mgr[100406]: -- 172.21.15.194:0/3611461862 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594a8bc11e0 con 0x5594a8b96800 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:49 smithi194 bash[100210]: 59d66bc02ea6855db34ed9d53497fe63c620320b8302bf00c3bd287c31d6e87e 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:49 smithi194 systemd[1]: Started Ceph mgr.z for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.023+0000 7f3d5935e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.103+0000 7f3d5935e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:09:50.234 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.191+0000 7f3d5935e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:09:50.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:50 smithi134 ceph-mon[97873]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:50.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:50 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:50 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:50 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:50 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:50.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:50 smithi093 ceph-mon[96579]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:50.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:50 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:50 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:50 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:50.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:50 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:50.490 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.282+0000 7f3d5935e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:09:50.543 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:50.794 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.449+0000 7f3d5935e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:09:51.084 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.753+0000 7f3d5935e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:09:51.084 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.842+0000 7f3d5935e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:09:51.084 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:50.969+0000 7f3d5935e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:09:51.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:51.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:51.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:51.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:51 smithi134 ceph-mon[97873]: Deploying daemon mgr.y on smithi134 2023-07-21T22:09:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:51.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:51.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:51 smithi093 ceph-mon[96579]: Deploying daemon mgr.y on smithi134 2023-07-21T22:09:51.521 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:51.063+0000 7f3d5935e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:09:51.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:09:51.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:51.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:51.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-mon[98288]: Deploying daemon mgr.y on smithi134 2023-07-21T22:09:51.834 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:51 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:51.480+0000 7f3d5935e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:09:51.876 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:51 smithi134 systemd[1]: Starting Ceph mgr.y for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:09:52.034 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:09:52.035 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:09:52.035 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-21T22:09:52.035 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-21T22:09:52.035 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109781 s, 955 MB/s 2023-07-21T22:09:52.035 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-21T22:09:52.334 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 podman[99305]: 2023-07-21T22:09:52.393 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_4 2023-07-21T22:09:52.528 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:52.050+0000 7f3d5935e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:09:52.529 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:52.220+0000 7f3d5935e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:09:52.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mon[97873]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:52.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.627 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 bash[99305]: 430624262cb7d405d72060179460ed096c24219ec2ae4a2492d898abad1a5207 2023-07-21T22:09:52.627 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-mgr[99345]: -- 172.21.15.134:0/1045736748 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c409be91e0 con 0x55c409bbe800 2023-07-21T22:09:52.627 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 systemd[1]: Started Ceph mgr.y for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:09:52.627 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:52.421+0000 7fc52b7a1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:09:52.627 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:52.504+0000 7fc52b7a1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:09:52.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:52 smithi093 ceph-mon[96579]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-mon[98288]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:52.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:52.919 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:52.599+0000 7fc52b7a1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:09:52.919 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:52.698+0000 7fc52b7a1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:09:52.958 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:09:53.237 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:52.878+0000 7fc52b7a1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:09:53.334 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.032+0000 7f3d5935e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:09:53.334 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.124+0000 7f3d5935e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:09:53.531 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:53.196+0000 7fc52b7a1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:09:53.531 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:53.301+0000 7fc52b7a1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:09:53.531 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:53.443+0000 7fc52b7a1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:09:53.649 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:53.649 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.308+0000 7f3d5935e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:09:53.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:53.876 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:53.541+0000 7fc52b7a1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:09:53.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:54.040 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.607+0000 7f3d5935e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:09:54.040 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.846+0000 7f3d5935e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:09:54.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 -- 172.21.15.93:0/1728292008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c100610 msgr2=0x7fa61c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:54.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1728292008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c100610 0x7fa61c106ed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa61400b0a0 tx=0x7fa61400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 -- 172.21.15.93:0/1728292008 shutdown_connections 2023-07-21T22:09:54.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1728292008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c100610 0x7fa61c106ed0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1728292008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c0fea80 0x7fa61c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1728292008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 0x7fa61c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 -- 172.21.15.93:0/1728292008 >> 172.21.15.93:0/1728292008 conn(0x7fa61c0fa0b0 msgr2=0x7fa61c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 -- 172.21.15.93:0/1728292008 shutdown_connections 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.174+0000 7fa622ebf700 1 -- 172.21.15.93:0/1728292008 wait complete. 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 Processor -- start 2023-07-21T22:09:54.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 -- start start 2023-07-21T22:09:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 0x7fa61c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c100610 0x7fa61c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19d490 con 0x7fa61c0fea80 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19d600 con 0x7fa61c100610 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa622ebf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c19d7c0 con 0x7fa61c0ffc80 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41194/0 (socket says 172.21.15.93:41194) 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 -- 172.21.15.93:0/1748738907 learned_addr learned my addr 172.21.15.93:0/1748738907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:09:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa61bfff700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 0x7fa61c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa62145c700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c100610 0x7fa61c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 -- 172.21.15.93:0/1748738907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 msgr2=0x7fa61c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 0x7fa61c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 -- 172.21.15.93:0/1748738907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c100610 msgr2=0x7fa61c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c100610 0x7fa61c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.175+0000 7fa620c5b700 1 -- 172.21.15.93:0/1748738907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa614009d00 con 0x7fa61c0fea80 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa620c5b700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d980 tx=0x7fa60400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604004020 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa60400f690 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1944d0 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604003a60 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.176+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1949d0 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa604003bc0 con 0x7fa61c0fea80 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa619ffb700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 0x7fa60803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:09:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa60404ae40 con 0x7fa61c0fea80 2023-07-21T22:09:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c106020 con 0x7fa61c0fea80 2023-07-21T22:09:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa61bfff700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 0x7fa60803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:09:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.177+0000 7fa61bfff700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 0x7fa60803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0690b0 tx=0x7fa60c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:09:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.183+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa60401c410 con 0x7fa61c0fea80 2023-07-21T22:09:54.311 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:53.998+0000 7f3d5935e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:09:54.376 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:53.992+0000 7fc52b7a1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:09:54.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:54.395+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa61c060950 con 0x7fa6080389e0 2023-07-21T22:09:54.584 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:54.269+0000 7f3d5935e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:09:54.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:54.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:09:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:09:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.786 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:54.574+0000 7fc52b7a1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:09:54.786 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:54.786 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.786 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:54.787 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:54.788 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.788 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:09:54.788 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:09:54.788 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.802 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:54.802 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:54.803 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:09:54.804 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:09:54.804 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:54.804 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:09:54.804 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:09:54.804 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:55.126 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:54 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:54.745+0000 7fc52b7a1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:09:55.328 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:54.967+0000 7f3d5935e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:09:55.329 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:55.099+0000 7f3d5935e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:09:55.584 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:55.287+0000 7f3d5935e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:09:55.584 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:55.400+0000 7f3d5935e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:09:55.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-mon[98288]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:09:55.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-mon[98288]: Reconfiguring daemon mgr.x on smithi093 2023-07-21T22:09:55.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-mon[98288]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:55.802 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:55 smithi093 ceph-mon[96579]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:09:55.802 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:55 smithi093 ceph-mon[96579]: Reconfiguring daemon mgr.x on smithi093 2023-07-21T22:09:55.802 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:55 smithi093 ceph-mon[96579]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:55.803 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:55.503+0000 7fc52b7a1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:09:55.803 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:55.593+0000 7fc52b7a1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:09:55.803 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-mon[97873]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:09:55.803 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-mon[97873]: Reconfiguring daemon mgr.x on smithi093 2023-07-21T22:09:55.803 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-mon[97873]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:09:56.080 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:55 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:55.762+0000 7fc52b7a1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:09:56.084 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:55.613+0000 7f3d5935e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:09:56.376 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:56.039+0000 7fc52b7a1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:09:56.376 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:56.266+0000 7fc52b7a1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:09:56.717 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:56.414+0000 7fc52b7a1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:09:56.717 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:56.718 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: Standby manager daemon z started 2023-07-21T22:09:56.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:09:56.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:56.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:09:56.719 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:09:56.834 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:09:56.462+0000 7f3d5935e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:09:56.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:56.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:56.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: Standby manager daemon z started 2023-07-21T22:09:56.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:09:56.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:56.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:09:56.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:56 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:09:56.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:56.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:09:56.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: Standby manager daemon z started 2023-07-21T22:09:56.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:09:56.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:56.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:09:56.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:56 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/2739182833' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:09:57.126 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:56.676+0000 7fc52b7a1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:09:57.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:57.408+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7fa60400fb00 con 0x7fa61c0fea80 2023-07-21T22:09:57.738 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:57.377+0000 7fc52b7a1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:09:57.739 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:57.509+0000 7fc52b7a1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:09:57.739 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]: dispatch 2023-07-21T22:09:57.739 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]': finished 2023-07-21T22:09:57.739 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:09:57.739 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:09:57.740 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T22:09:57.740 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:09:57.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]: dispatch 2023-07-21T22:09:57.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]': finished 2023-07-21T22:09:57.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:09:57.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:09:57.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T22:09:57.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:57 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:09:57.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]: dispatch 2023-07-21T22:09:57.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/59626843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a67ed18-5188-4868-8069-de6b0b8165cd"}]': finished 2023-07-21T22:09:57.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:09:57.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:09:57.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: mgrmap e14: x(active, since 66s), standbys: z 2023-07-21T22:09:57.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:57 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:09:58.066 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:57.698+0000 7fc52b7a1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:09:58.066 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:57.811+0000 7fc52b7a1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:09:58.376 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:58 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:58.025+0000 7fc52b7a1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:09:58.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:58 smithi194 ceph-mon[98288]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:58.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:58 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1807157266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:09:58.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:58 smithi134 ceph-mon[97873]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:58.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:58 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1807157266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:09:58.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:58 smithi093 ceph-mon[96579]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:09:58.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:58 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1807157266' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:09:59.376 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:09:58 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:09:58.881+0000 7fc52b7a1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:09:59.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:09:59.510+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fa60400f800 con 0x7fa61c0fea80 2023-07-21T22:09:59.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:59 smithi194 ceph-mon[98288]: Standby manager daemon y started 2023-07-21T22:09:59.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:59 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:09:59.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:59 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:59.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:59 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:09:59.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:09:59 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:09:59.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:59 smithi134 ceph-mon[97873]: Standby manager daemon y started 2023-07-21T22:09:59.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:59 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:09:59.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:59 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:59.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:59 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:09:59.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:09:59 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:09:59.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:59 smithi093 ceph-mon[96579]: Standby manager daemon y started 2023-07-21T22:09:59.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:59 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:09:59.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:59 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:09:59.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:59 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:09:59.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:09:59 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/1343679793' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:10:00.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:00 smithi194 ceph-mon[98288]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:00.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:00 smithi194 ceph-mon[98288]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-07-21T22:10:00.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:00 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:10:00.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:00 smithi194 ceph-mon[98288]: overall HEALTH_OK 2023-07-21T22:10:00.855 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:00 smithi093 ceph-mon[96579]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:00.855 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:00 smithi093 ceph-mon[96579]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-07-21T22:10:00.855 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:00 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:10:00.855 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:00 smithi093 ceph-mon[96579]: overall HEALTH_OK 2023-07-21T22:10:00.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:00 smithi134 ceph-mon[97873]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:00.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:00 smithi134 ceph-mon[97873]: mgrmap e15: x(active, since 68s), standbys: z, y 2023-07-21T22:10:00.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:00 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:10:00.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:00 smithi134 ceph-mon[97873]: overall HEALTH_OK 2023-07-21T22:10:02.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:02 smithi194 ceph-mon[98288]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:02.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:02 smithi134 ceph-mon[97873]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:02.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:02 smithi093 ceph-mon[96579]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:04.748 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:04 smithi093 ceph-mon[96579]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:04.749 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:10:04.749 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:04.749 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:04 smithi093 ceph-mon[96579]: Deploying daemon osd.0 on smithi093 2023-07-21T22:10:04.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:04 smithi194 ceph-mon[98288]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:04.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:10:04.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:04.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:04 smithi194 ceph-mon[98288]: Deploying daemon osd.0 on smithi093 2023-07-21T22:10:04.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:04 smithi134 ceph-mon[97873]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:04.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:10:04.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:04.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:04 smithi134 ceph-mon[97873]: Deploying daemon osd.0 on smithi093 2023-07-21T22:10:06.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:06 smithi194 ceph-mon[98288]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:06.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:06 smithi134 ceph-mon[97873]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:06.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:06 smithi093 ceph-mon[96579]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:08.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:08 smithi093 ceph-mon[96579]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:08.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:08.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:08 smithi194 ceph-mon[98288]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:08.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:08.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:08 smithi134 ceph-mon[97873]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:08.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:08.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:10.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.695+0000 7fa619ffb700 1 -- 172.21.15.93:0/1748738907 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa61c060950 con 0x7fa6080389e0 2023-07-21T22:10:10.737 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 0 on host 'smithi093' 2023-07-21T22:10:10.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 msgr2=0x7fa60803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:10.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 0x7fa60803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0690b0 tx=0x7fa60c009260 comp rx=0 tx=0).stop 2023-07-21T22:10:10.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 msgr2=0x7fa61c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:10.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d980 tx=0x7fa60400dc90 comp rx=0 tx=0).stop 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 shutdown_connections 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa6080389e0 0x7fa60803ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa61c100610 0x7fa61c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa61c0fea80 0x7fa61c068b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 --2- 172.21.15.93:0/1748738907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa61c0ffc80 0x7fa61c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 >> 172.21.15.93:0/1748738907 conn(0x7fa61c0fa0b0 msgr2=0x7fa61c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:10.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 shutdown_connections 2023-07-21T22:10:10.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:10.698+0000 7fa622ebf700 1 -- 172.21.15.93:0/1748738907 wait complete. 2023-07-21T22:10:10.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:10.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:10.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:10.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.863 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:10.863 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.864 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.864 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:10.864 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:10.864 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:10.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:10.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:10.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:10.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:11.172 DEBUG:teuthology.orchestra.run.smithi093:osd.0> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.0.service 2023-07-21T22:10:11.174 INFO:tasks.cephadm:Deploying osd.1 on smithi134 with /dev/vg_nvme/lv_4... 2023-07-21T22:10:11.174 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:10:11.207 INFO:journalctl@ceph.osd.0.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:10:11.739 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:10:11.937 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:10:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:10:11.835+0000 7f559fb26640 -1 osd.0 0 log_to_monitors true 2023-07-21T22:10:11.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:11 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:11.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:11 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:11.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:11 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:11.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:11 smithi093 ceph-mon[96579]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:11.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:11 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:11.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:11 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:11.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:11 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:11.956 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:11 smithi134 ceph-mon[97873]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:12.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:11 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:12.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:11 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:12.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:11 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:12.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:11 smithi194 ceph-mon[98288]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:12.772 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:12 smithi134 ceph-mon[97873]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:12.773 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:12 smithi134 ceph-mon[97873]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:12.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:12 smithi093 ceph-mon[96579]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:12.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:12 smithi093 ceph-mon[96579]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:13.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:12 smithi194 ceph-mon[98288]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:13.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:12 smithi194 ceph-mon[98288]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109704 s, 956 MB/s 2023-07-21T22:10:13.234 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-21T22:10:13.596 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2023-07-21T22:10:13.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:10:13.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi093 2023-07-21T22:10:13.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:13.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.938 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:10:13.675+0000 7f559499d700 -1 osd.0 0 waiting for initial osdmap 2023-07-21T22:10:13.939 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:10:13.681+0000 7f558d78c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:10:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:10:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:10:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi093 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-21T22:10:13.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:13.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:13.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:13.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:14.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:10:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:10:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:10:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi093 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 17168M 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:14.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:14.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:14.150 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:10:15.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:14 smithi194 ceph-mon[98288]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:10:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:14 smithi194 ceph-mon[98288]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:10:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:14 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:14 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:14 smithi134 ceph-mon[97873]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:10:15.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:14 smithi134 ceph-mon[97873]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:10:15.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:14 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:14 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:14 smithi093 ceph-mon[96579]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:10:15.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:14 smithi093 ceph-mon[96579]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:10:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:14 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:15.457 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 -- 172.21.15.134:0/2944372472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c100e20 msgr2=0x7f0f8c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:15.457 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 --2- 172.21.15.134:0/2944372472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c100e20 0x7f0f8c101270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009990 tx=0x7f0f88009ca0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.457 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 -- 172.21.15.134:0/2944372472 shutdown_connections 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 --2- 172.21.15.134:0/2944372472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 --2- 172.21.15.134:0/2944372472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 --2- 172.21.15.134:0/2944372472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c100e20 0x7f0f8c101270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 -- 172.21.15.134:0/2944372472 >> 172.21.15.134:0/2944372472 conn(0x7f0f8c0fb250 msgr2=0x7f0f8c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 -- 172.21.15.134:0/2944372472 shutdown_connections 2023-07-21T22:10:15.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 -- 172.21.15.134:0/2944372472 wait complete. 2023-07-21T22:10:15.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.415+0000 7f0f9458a700 1 Processor -- start 2023-07-21T22:10:15.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 -- start start 2023-07-21T22:10:15.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:15.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 0x7f0f8c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:15.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0735c0 con 0x7f0f8c100e20 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c073730 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c19f910 con 0x7f0f8c1017b0 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:54228/0 (socket says 172.21.15.134:54228) 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 -- 172.21.15.134:0/597132084 learned_addr learned my addr 172.21.15.134:0/597132084 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-21T22:10:15.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f91b25700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 0x7f0f8c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92b27700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 -- 172.21.15.134:0/597132084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 msgr2=0x7f0f8c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 -- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 msgr2=0x7f0f8c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 0x7f0f8c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 -- 172.21.15.134:0/597132084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880095f0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f91b25700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 0x7f0f8c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92b27700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f92326700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d980 tx=0x7f0f7c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00de60 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f7c0044e0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c19fbf0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.416+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003be0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.417+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1a0140 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.417+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c10b3e0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.418+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f0f7c009bb0 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.419+0000 7f0f837fe700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 0x7f0f780a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.419+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f0f7c0caf00 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.419+0000 7f0f91b25700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 0x7f0f780a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:15.466 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.420+0000 7f0f91b25700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 0x7f0f780a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800b5c0 tx=0x7f0f88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:15.466 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.422+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0f7c096140 con 0x7f0f8c0ffc20 2023-07-21T22:10:15.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:15.618+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f8c060950 con 0x7f0f780a1140 2023-07-21T22:10:16.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:10:16.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:10:16.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] boot 2023-07-21T22:10:16.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:10:16.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:16.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:16.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:16.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:16.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:15 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:10:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] boot 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:16.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:15 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:16.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:10:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:10:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: osd.0 [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] boot 2023-07-21T22:10:16.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:10:16.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:10:16.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:10:16.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:16.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:16.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:16.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:15 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:16.984 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:16 smithi134 ceph-mon[97873]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:10:17.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:16 smithi194 ceph-mon[98288]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:10:17.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:16 smithi093 ceph-mon[96579]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:10:18.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:17 smithi194 ceph-mon[98288]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:18.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:17 smithi134 ceph-mon[97873]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:18.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:17 smithi093 ceph-mon[96579]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:19.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/136733118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]': finished 2023-07-21T22:10:19.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:10:19.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:19.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:18 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/4087886246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:19.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/136733118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]': finished 2023-07-21T22:10:19.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:10:19.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:19.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:18 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/4087886246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:19.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/136733118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]: dispatch 2023-07-21T22:10:19.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6567d033-de01-4296-ab88-b6781b7519ea"}]': finished 2023-07-21T22:10:19.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:10:19.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:19.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:18 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/4087886246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:20.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:19 smithi194 ceph-mon[98288]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:20.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:19 smithi134 ceph-mon[97873]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:20.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:19 smithi093 ceph-mon[96579]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:22.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:22 smithi134 ceph-mon[97873]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:22.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:22 smithi194 ceph-mon[98288]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:22.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:22 smithi093 ceph-mon[96579]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:24.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:24 smithi134 ceph-mon[97873]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:24.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:24 smithi194 ceph-mon[98288]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:24.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:24 smithi093 ceph-mon[96579]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:25.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:10:25.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:25.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:10:25.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:25.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:10:25.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:26.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:26 smithi194 ceph-mon[98288]: Deploying daemon osd.1 on smithi134 2023-07-21T22:10:26.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:26 smithi194 ceph-mon[98288]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:26.857 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:26 smithi134 ceph-mon[97873]: Deploying daemon osd.1 on smithi134 2023-07-21T22:10:26.857 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:26 smithi134 ceph-mon[97873]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:26.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:26 smithi093 ceph-mon[96579]: Deploying daemon osd.1 on smithi134 2023-07-21T22:10:26.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:26 smithi093 ceph-mon[96579]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:28.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:28 smithi194 ceph-mon[98288]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:28.857 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:28 smithi134 ceph-mon[97873]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:28.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:28 smithi093 ceph-mon[96579]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:30.230 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:30.230 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.231 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.231 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:30 smithi134 ceph-mon[97873]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:30.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:30.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:30 smithi194 ceph-mon[98288]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:30.438 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:30.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:30.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:30 smithi093 ceph-mon[96579]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:31.770 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 1 on host 'smithi134' 2023-07-21T22:10:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.728+0000 7f0f837fe700 1 -- 172.21.15.134:0/597132084 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f8c060950 con 0x7f0f780a1140 2023-07-21T22:10:31.772 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 msgr2=0x7f0f780a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:31.773 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 0x7f0f780a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800b5c0 tx=0x7f0f88005fb0 comp rx=0 tx=0).stop 2023-07-21T22:10:31.773 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 msgr2=0x7f0f8c0742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:31.773 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00d980 tx=0x7f0f7c00dd40 comp rx=0 tx=0).stop 2023-07-21T22:10:31.773 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 shutdown_connections 2023-07-21T22:10:31.774 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f8c1017b0 0x7f0f8c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:31.774 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0f780a1140 0x7f0f780a35f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:31.774 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f8c0ffc20 0x7f0f8c0742b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:31.775 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 --2- 172.21.15.134:0/597132084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f8c100e20 0x7f0f8c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:31.775 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 >> 172.21.15.134:0/597132084 conn(0x7f0f8c0fb250 msgr2=0x7f0f8c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:31.775 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 shutdown_connections 2023-07-21T22:10:31.775 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:10:31.730+0000 7f0f9458a700 1 -- 172.21.15.134:0/597132084 wait complete. 2023-07-21T22:10:32.129 DEBUG:teuthology.orchestra.run.smithi134:osd.1> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.1.service 2023-07-21T22:10:32.132 INFO:tasks.cephadm:Deploying osd.2 on smithi194 with /dev/vg_nvme/lv_4... 2023-07-21T22:10:32.132 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:10:32.161 INFO:journalctl@ceph.osd.1.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:10:32.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:32 smithi134 ceph-mon[97873]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:32.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:32 smithi194 ceph-mon[98288]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:32.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:32 smithi093 ceph-mon[96579]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:32.696 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:10:33.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:33 smithi134 ceph-mon[97873]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:33.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:33 smithi134 ceph-mon[97873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:33.377 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:10:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:10:33.054+0000 7f50fe597640 -1 osd.1 0 log_to_monitors true 2023-07-21T22:10:33.555 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:33 smithi194 ceph-mon[98288]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:33.556 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:33 smithi194 ceph-mon[98288]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:33.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:33 smithi093 ceph-mon[96579]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:33.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:33 smithi093 ceph-mon[96579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:10:34.292 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:10:34.292 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:10:34.293 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2023-07-21T22:10:34.293 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2023-07-21T22:10:34.293 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111 s, 945 MB/s 2023-07-21T22:10:34.293 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2023-07-21T22:10:34.377 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:10:34.187+0000 7f50f340e700 -1 osd.1 0 waiting for initial osdmap 2023-07-21T22:10:34.377 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:10:34.191+0000 7f50eda00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:10:34.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:34.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:10:34.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:10:34.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:34.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:34 smithi134 ceph-mon[97873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:34.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:10:34.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:10:34.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:34.483 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.484 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:34 smithi194 ceph-mon[98288]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:10:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:10:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: from='osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:34 smithi093 ceph-mon[96579]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:10:34.741 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2023-07-21T22:10:35.299 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:10:35.545 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:10:35.545 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:10:35.545 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.545 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.545 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi134 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.546 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:35.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:10:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:10:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi134 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:10:35.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:10:35.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi134 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:36.427 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:10:36.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:10:36.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:36.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] boot 2023-07-21T22:10:36.428 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:10:36.429 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:36.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 -- 172.21.15.194:0/3863435981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400ffc80 msgr2=0x7f6d401000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:36.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 --2- 172.21.15.194:0/3863435981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400ffc80 0x7f6d401000d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009990 tx=0x7f6d30009ca0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 -- 172.21.15.194:0/3863435981 shutdown_connections 2023-07-21T22:10:36.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 --2- 172.21.15.194:0/3863435981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d40100610 0x7f6d40106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 --2- 172.21.15.194:0/3863435981 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d400fea80 0x7f6d400fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 --2- 172.21.15.194:0/3863435981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400ffc80 0x7f6d401000d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 -- 172.21.15.194:0/3863435981 >> 172.21.15.194:0/3863435981 conn(0x7f6d400fa0b0 msgr2=0x7f6d400fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 -- 172.21.15.194:0/3863435981 shutdown_connections 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 -- 172.21.15.194:0/3863435981 wait complete. 2023-07-21T22:10:36.618 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.575+0000 7f6d46081700 1 Processor -- start 2023-07-21T22:10:36.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 -- start start 2023-07-21T22:10:36.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:36.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400ffc80 0x7f6d40198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:36.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d40100610 0x7f6d4019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4019f680 con 0x7f6d40100610 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4019f7c0 con 0x7f6d400ffc80 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d46081700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4019f950 con 0x7f6d400fea80 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47026/0 (socket says 172.21.15.194:47026) 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 -- 172.21.15.194:0/4158759515 learned_addr learned my addr 172.21.15.194:0/4158759515 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:10:36.621 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3effd700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400ffc80 0x7f6d40198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3ffff700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d40100610 0x7f6d4019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 -- 172.21.15.194:0/4158759515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400ffc80 msgr2=0x7f6d40198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400ffc80 0x7f6d40198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 -- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d40100610 msgr2=0x7f6d4019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d40100610 0x7f6d4019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 -- 172.21.15.194:0/4158759515 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d300095f0 con 0x7f6d400fea80 2023-07-21T22:10:36.622 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3f7fe700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400d9b0 tx=0x7f6d3400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.576+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34004020 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.577+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d34009730 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.577+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4019fc30 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.577+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003a30 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.577+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401a0130 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.578+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4010a290 con 0x7f6d400fea80 2023-07-21T22:10:36.623 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.578+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f6d340098a0 con 0x7f6d400fea80 2023-07-21T22:10:36.624 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.579+0000 7f6d3cff9700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:36.624 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.579+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f6d340cad60 con 0x7f6d400fea80 2023-07-21T22:10:36.624 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.579+0000 7f6d3effd700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:36.624 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.580+0000 7f6d3effd700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 0x7f6d280a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b5c0 tx=0x7f6d30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:36.625 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.583+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6d34095d00 con 0x7f6d400fea80 2023-07-21T22:10:36.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:10:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:10:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] boot 2023-07-21T22:10:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:10:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:36.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:10:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:10:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:10:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: osd.1 [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] boot 2023-07-21T22:10:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:10:36.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:10:36.821 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:36.780+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d40069070 con 0x7f6d280a10c0 2023-07-21T22:10:38.037 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:10:38.037 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:38.038 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:38.038 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:38.038 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:38.038 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:37 smithi194 ceph-mon[98288]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:38.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:10:38.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:38.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:38.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:38.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:38.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:37 smithi134 ceph-mon[97873]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:38.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:10:38.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:10:38.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:10:38.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:10:38.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:38.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:37 smithi093 ceph-mon[96579]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/2534240131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]': finished 2023-07-21T22:10:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:10:40.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:40.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:40 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/1352666758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:40.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/2534240131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]': finished 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:40.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:40 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/1352666758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:40.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/2534240131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]: dispatch 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4704710d-70ac-4f07-8dbd-509266a7d1c6"}]': finished 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:40.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:40 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/1352666758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:10:42.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:42 smithi194 ceph-mon[98288]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:42.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:42 smithi134 ceph-mon[97873]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:42.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:42 smithi093 ceph-mon[96579]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:44.752 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:44 smithi194 ceph-mon[98288]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:44.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:44 smithi134 ceph-mon[97873]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:44.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:44 smithi093 ceph-mon[96579]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:46.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:46 smithi194 ceph-mon[98288]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:46.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:10:46.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:46.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:46 smithi134 ceph-mon[97873]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:46.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:10:46.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:46.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:46 smithi093 ceph-mon[96579]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:46.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:10:46.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:47.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:47 smithi194 ceph-mon[98288]: Deploying daemon osd.2 on smithi194 2023-07-21T22:10:47.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:47 smithi134 ceph-mon[97873]: Deploying daemon osd.2 on smithi194 2023-07-21T22:10:47.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:47 smithi093 ceph-mon[96579]: Deploying daemon osd.2 on smithi194 2023-07-21T22:10:48.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:48 smithi194 ceph-mon[98288]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:48.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:48 smithi134 ceph-mon[97873]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:48.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:48 smithi093 ceph-mon[96579]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:50.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:50 smithi194 ceph-mon[98288]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:50.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:50 smithi134 ceph-mon[97873]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:50.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:50 smithi093 ceph-mon[96579]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:51.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:51.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:51.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:51.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:51 smithi194 ceph-mon[98288]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:52.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:52.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:52.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:52.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:51 smithi134 ceph-mon[97873]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:52.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:52.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:52.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:52.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:51 smithi093 ceph-mon[96579]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:53.882 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 2 on host 'smithi194' 2023-07-21T22:10:53.883 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.840+0000 7f6d3cff9700 1 -- 172.21.15.194:0/4158759515 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d40069070 con 0x7f6d280a10c0 2023-07-21T22:10:53.885 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 msgr2=0x7f6d280a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:53.885 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 0x7f6d280a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b5c0 tx=0x7f6d30005fb0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.885 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 msgr2=0x7f6d40197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:53.885 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400d9b0 tx=0x7f6d3400dcc0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.885 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 shutdown_connections 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d40100610 0x7f6d4019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d400fea80 0x7f6d40197af0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d280a10c0 0x7f6d280a3570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 --2- 172.21.15.194:0/4158759515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400ffc80 0x7f6d40198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 >> 172.21.15.194:0/4158759515 conn(0x7f6d400fa0b0 msgr2=0x7f6d40068960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 shutdown_connections 2023-07-21T22:10:53.886 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:10:53.843+0000 7f6d46081700 1 -- 172.21.15.194:0/4158759515 wait complete. 2023-07-21T22:10:54.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:54.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:54.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:53 smithi194 ceph-mon[98288]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:54.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:54.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:54.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:53 smithi134 ceph-mon[97873]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:54.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:54.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:54.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:53 smithi093 ceph-mon[96579]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:54.391 DEBUG:teuthology.orchestra.run.smithi194:osd.2> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.2.service 2023-07-21T22:10:54.393 INFO:tasks.cephadm:Deploying osd.3 on smithi093 with /dev/vg_nvme/lv_3... 2023-07-21T22:10:54.394 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:10:54.426 INFO:journalctl@ceph.osd.2.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:10:54.924 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:10:54.599+0000 7f9f7f8db640 -1 osd.2 0 log_to_monitors true 2023-07-21T22:10:54.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:54.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:54.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-mon[98288]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:54.925 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:54 smithi194 ceph-mon[98288]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:10:55.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:55.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:55.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:54 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:55.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:54 smithi134 ceph-mon[97873]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:55.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:54 smithi134 ceph-mon[97873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:55.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:55.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:55.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:54 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:10:55.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:54 smithi093 ceph-mon[96579]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:55.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:54 smithi093 ceph-mon[96579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:10:56.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:10:56.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:10:56.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:56.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-mon[98288]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:56.086 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:10:55.851+0000 7f9f74752700 -1 osd.2 0 waiting for initial osdmap 2023-07-21T22:10:56.086 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:10:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:10:55.856+0000 7f9f6fd46700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:10:56.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:10:56.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:10:56.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:56.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:55 smithi093 ceph-mon[96579]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:56.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:10:56.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:10:56.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:56.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: from='osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:10:56.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:55 smithi134 ceph-mon[97873]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:10:56.462 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:10:56.462 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-21T22:10:56.462 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-21T22:10:56.462 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011082 s, 946 MB/s 2023-07-21T22:10:56.463 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-21T22:10:57.006 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_3 2023-07-21T22:10:57.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:10:57.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:10:57.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:57.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:57.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:57.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:10:57.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:10:57.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:57.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:10:57.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:10:57.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:10:57.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:10:57.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:10:57.554 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:10:58.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:10:58.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi194 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 17168M 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] boot 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:57 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:10:58.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:10:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:10:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi194 2023-07-21T22:10:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 17168M 2023-07-21T22:10:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] boot 2023-07-21T22:10:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:10:58.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:58.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:58.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:57 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:10:58.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi194 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 17168M 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: osd.2 [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] boot 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:10:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:10:58.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:57 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:10:58.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 -- 172.21.15.93:0/1968987690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 msgr2=0x7f2c14106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:58.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/1968987690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c14106ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00b0a0 tx=0x7f2c0c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 -- 172.21.15.93:0/1968987690 shutdown_connections 2023-07-21T22:10:58.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/1968987690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c14106ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/1968987690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c140fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/1968987690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 0x7f2c141000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.710+0000 7f2c1b79d700 1 -- 172.21.15.93:0/1968987690 >> 172.21.15.93:0/1968987690 conn(0x7f2c140fa0b0 msgr2=0x7f2c140fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:10:58.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- 172.21.15.93:0/1968987690 shutdown_connections 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- 172.21.15.93:0/1968987690 wait complete. 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 Processor -- start 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- start start 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c14195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 0x7f2c14195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:58.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1419d620 con 0x7f2c14100610 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1419d760 con 0x7f2c140fea80 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.711+0000 7f2c1b79d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1419d8f0 con 0x7f2c140ffc80 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41276/0 (socket says 172.21.15.93:41276) 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 -- 172.21.15.93:0/2839037152 learned_addr learned my addr 172.21.15.93:0/2839037152 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c18d38700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 0x7f2c14195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19539700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c14195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:58.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 -- 172.21.15.93:0/2839037152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 msgr2=0x7f2c14195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:58.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 0x7f2c14195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 -- 172.21.15.93:0/2839037152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 msgr2=0x7f2c14195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:10:58.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c14195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:10:58.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 -- 172.21.15.93:0/2839037152 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c009d00 con 0x7f2c14100610 2023-07-21T22:10:58.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19539700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c14195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c19d3a700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007ab0 tx=0x7f2c0c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00e040 con 0x7f2c14100610 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c00cec0 con 0x7f2c14100610 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01bc40 con 0x7f2c14100610 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1419db70 con 0x7f2c14100610 2023-07-21T22:10:58.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.712+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1419df90 con 0x7f2c14100610 2023-07-21T22:10:58.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.713+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c14106080 con 0x7f2c14100610 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.714+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f2c0c003a00 con 0x7f2c14100610 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.715+0000 7f2c0a7fc700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 0x7f2c000a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.715+0000 7f2c19539700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 0x7f2c000a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.715+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f2c0c0cd480 con 0x7f2c14100610 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.715+0000 7f2c19539700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 0x7f2c000a3570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c140ffae0 tx=0x7f2c10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:10:58.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.718+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2c0c098120 con 0x7f2c14100610 2023-07-21T22:10:58.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:10:58.913+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c14060950 con 0x7f2c000a10c0 2023-07-21T22:10:59.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:58 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:10:59.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:58 smithi093 ceph-mon[96579]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:10:59.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:58 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:10:59.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:58 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:10:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:58 smithi194 ceph-mon[98288]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:10:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:58 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:10:59.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:58 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:10:59.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:58 smithi134 ceph-mon[97873]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:10:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:58 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:11:00.039 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:11:00.040 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:11:00.040 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:00.040 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:00.041 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:00.041 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:00.041 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:10:59 smithi093 ceph-mon[96579]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:00.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:11:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:11:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:00.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:10:59 smithi194 ceph-mon[98288]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:00.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:11:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:11:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:00.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:10:59 smithi134 ceph-mon[97873]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:00.378 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112600]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:00.378 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112600]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:00.378 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112600]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:00.378 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112600]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:00.630 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112627]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:00.630 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:00.631 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112627]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:00.631 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:11:00 smithi093 sudo[112627]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:00.897 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105243]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:00.897 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:00.897 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105243]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:00.897 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105243]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:01.164 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:00 smithi194 ceph-mon[98288]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:11:01.165 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:00 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:01.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:00 smithi093 ceph-mon[96579]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:11:01.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:00 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:00 smithi134 ceph-mon[97873]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:11:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:00 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:01.377 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105246]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:01.377 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:01.378 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105246]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:01.378 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:11:00 smithi134 sudo[105246]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:01.418 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106240]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:01.419 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:01.419 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106240]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:01.419 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106240]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:01.834 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106243]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:01.835 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:01.835 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106243]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:01.835 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:11:01 smithi194 sudo[106243]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:01.952 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112733]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:01.953 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:01.953 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112733]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:01.953 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112733]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:02.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.220 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]: dispatch 2023-07-21T22:11:02.221 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]': finished 2023-07-21T22:11:02.221 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:11:02.221 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:02.222 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.222 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:01 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:02.222 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106246]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:02.223 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:02.223 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106246]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:02.223 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106246]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:02.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]: dispatch 2023-07-21T22:11:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]': finished 2023-07-21T22:11:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:11:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:01 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:02.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112736]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:02.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112736]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:02.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112736]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 sudo[112736]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]: dispatch 2023-07-21T22:11:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/497680023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5622ee06-de4f-4d56-9829-efced7a3fbcc"}]': finished 2023-07-21T22:11:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:11:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:02.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:01 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:02.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106249]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106249]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 sudo[106249]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:02.954 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105249]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:11:02.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:02.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105249]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:02.955 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105249]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:02.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:02.931+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c0c01bda0 con 0x7f2c14100610 2023-07-21T22:11:03.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:03.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3213180288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:02 smithi194 ceph-mon[98288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105252]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:11:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:11:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105252]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:11:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 sudo[105252]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:11:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3213180288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:02 smithi134 ceph-mon[97873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:03.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3213180288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:03.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:03.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:11:03.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:03.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:11:03.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:11:03.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:02 smithi093 ceph-mon[96579]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:11:04.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:03 smithi194 ceph-mon[98288]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:11:04.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:03 smithi194 ceph-mon[98288]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:04.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:03 smithi134 ceph-mon[97873]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:11:04.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:03 smithi134 ceph-mon[97873]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:04.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:03 smithi093 ceph-mon[96579]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:11:04.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:03 smithi093 ceph-mon[96579]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:06.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:06 smithi194 ceph-mon[98288]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:06.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:06 smithi134 ceph-mon[97873]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:06.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:06 smithi093 ceph-mon[96579]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:08.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:07 smithi194 ceph-mon[98288]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 133 KiB/s wr, 9 op/s 2023-07-21T22:11:08.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:07 smithi134 ceph-mon[97873]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 133 KiB/s wr, 9 op/s 2023-07-21T22:11:08.418 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:07 smithi093 ceph-mon[96579]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 133 KiB/s wr, 9 op/s 2023-07-21T22:11:09.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:11:09.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:09.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:08 smithi194 ceph-mon[98288]: Deploying daemon osd.3 on smithi093 2023-07-21T22:11:09.371 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:11:09.371 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:09.371 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:08 smithi093 ceph-mon[96579]: Deploying daemon osd.3 on smithi093 2023-07-21T22:11:09.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:11:09.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:09.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:08 smithi134 ceph-mon[97873]: Deploying daemon osd.3 on smithi093 2023-07-21T22:11:10.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:09 smithi093 ceph-mon[96579]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 99 KiB/s wr, 7 op/s 2023-07-21T22:11:10.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:09 smithi194 ceph-mon[98288]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 99 KiB/s wr, 7 op/s 2023-07-21T22:11:10.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:09 smithi134 ceph-mon[97873]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 99 KiB/s wr, 7 op/s 2023-07-21T22:11:12.544 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:12 smithi093 ceph-mon[96579]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 79 KiB/s wr, 5 op/s 2023-07-21T22:11:12.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:12 smithi194 ceph-mon[98288]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 79 KiB/s wr, 5 op/s 2023-07-21T22:11:12.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:12 smithi134 ceph-mon[97873]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 79 KiB/s wr, 5 op/s 2023-07-21T22:11:13.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:13.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:13.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:13.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:13 smithi093 ceph-mon[96579]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 66 KiB/s wr, 4 op/s 2023-07-21T22:11:14.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:14.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:14.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:14.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:13 smithi194 ceph-mon[98288]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 66 KiB/s wr, 4 op/s 2023-07-21T22:11:14.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:14.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:14.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:14.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:13 smithi134 ceph-mon[97873]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 66 KiB/s wr, 4 op/s 2023-07-21T22:11:15.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.565+0000 7f2c0a7fc700 1 -- 172.21.15.93:0/2839037152 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c14060950 con 0x7f2c000a10c0 2023-07-21T22:11:15.608 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 3 on host 'smithi093' 2023-07-21T22:11:15.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.567+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 msgr2=0x7f2c000a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.567+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 0x7f2c000a3570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c140ffae0 tx=0x7f2c10008040 comp rx=0 tx=0).stop 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 msgr2=0x7f2c1419d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007ab0 tx=0x7f2c0c004d00 comp rx=0 tx=0).stop 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 shutdown_connections 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c14100610 0x7f2c1419d050 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:15.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2c000a10c0 0x7f2c000a3570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:15.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c140fea80 0x7f2c14195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:15.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 --2- 172.21.15.93:0/2839037152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c140ffc80 0x7f2c14195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:15.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 >> 172.21.15.93:0/2839037152 conn(0x7f2c140fa0b0 msgr2=0x7f2c141031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:15.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 shutdown_connections 2023-07-21T22:11:15.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:11:15.568+0000 7f2c1b79d700 1 -- 172.21.15.93:0/2839037152 wait complete. 2023-07-21T22:11:16.043 DEBUG:teuthology.orchestra.run.smithi093:osd.3> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.3.service 2023-07-21T22:11:16.045 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_3... 2023-07-21T22:11:16.045 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:11:16.075 INFO:journalctl@ceph.osd.3.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:11:16.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:16 smithi134 ceph-mon[97873]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-21T22:11:16.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-mon[96579]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-21T22:11:16.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:16 smithi194 ceph-mon[98288]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-21T22:11:16.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:11:16.870 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:11:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:11:16.787+0000 7f2b8fd44640 -1 osd.3 0 log_to_monitors true 2023-07-21T22:11:17.139 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:17 smithi093 ceph-mon[96579]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:11:17.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:17 smithi134 ceph-mon[97873]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:11:17.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:17 smithi194 ceph-mon[98288]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:11:18.099 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:11:18.100 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:11:18.100 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-21T22:11:18.100 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-21T22:11:18.100 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109137 s, 961 MB/s 2023-07-21T22:11:18.100 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-21T22:11:18.189 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:11:18.082+0000 7f2b84bbb700 -1 osd.3 0 waiting for initial osdmap 2023-07-21T22:11:18.189 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:11:18.091+0000 7f2b7f1ad700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:11:18.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-mon[96579]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:11:18.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-mon[96579]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:11:18.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:18.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-mon[96579]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:11:18.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:18 smithi093 ceph-mon[96579]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:18.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:18 smithi134 ceph-mon[97873]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:11:18.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:18 smithi134 ceph-mon[97873]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:11:18.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:18 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:18.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:18 smithi134 ceph-mon[97873]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:11:18.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:18 smithi134 ceph-mon[97873]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:18.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:18 smithi194 ceph-mon[98288]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:11:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:18 smithi194 ceph-mon[98288]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:11:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:18 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:18 smithi194 ceph-mon[98288]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:11:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:18 smithi194 ceph-mon[98288]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:18.654 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2023-07-21T22:11:19.203 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:11:19.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:11:19.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi093 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:11:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:11:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.445 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi093 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.446 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.447 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:11:19.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:11:19.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi093 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 8584M 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/2751812260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 msgr2=0x7fe2f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/2751812260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4101200 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009990 tx=0x7fe2f0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/2751812260 shutdown_connections 2023-07-21T22:11:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/2751812260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f4101740 0x7fe2f4108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/2751812260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f40ffbb0 0x7fe2f40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/2751812260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4101200 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/2751812260 >> 172.21.15.134:0/2751812260 conn(0x7fe2f40fb1c0 msgr2=0x7fe2f40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/2751812260 shutdown_connections 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/2751812260 wait complete. 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 Processor -- start 2023-07-21T22:11:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.370+0000 7fe2fbfc1700 1 -- start start 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f40ffbb0 0x7fe2f40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f4101740 0x7fe2f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40735c0 con 0x7fe2f40ffbb0 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4073730 con 0x7fe2f4100db0 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419f940 con 0x7fe2f4101740 2023-07-21T22:11:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:54302/0 (socket says 172.21.15.134:54302) 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 -- 172.21.15.134:0/4096899024 learned_addr learned my addr 172.21.15.134:0/4096899024 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fa55e700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f4101740 0x7fe2f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f9d5d700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f40ffbb0 0x7fe2f40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 -- 172.21.15.134:0/4096899024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f4101740 msgr2=0x7fe2f4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f4101740 0x7fe2f4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 -- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f40ffbb0 msgr2=0x7fe2f40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f40ffbb0 0x7fe2f40742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 -- 172.21.15.134:0/4096899024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00095f0 con 0x7fe2f4100db0 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2f955c700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0000c00 tx=0x7fe2f000f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001d070 con 0x7fe2f4100db0 2023-07-21T22:11:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f419fbc0 con 0x7fe2f4100db0 2023-07-21T22:11:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2f000fb50 con 0x7fe2f4100db0 2023-07-21T22:11:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017650 con 0x7fe2f4100db0 2023-07-21T22:11:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.371+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f41a00b0 con 0x7fe2f4100db0 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.372+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f4107200 con 0x7fe2f4100db0 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.373+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe2f000fcc0 con 0x7fe2f4100db0 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.374+0000 7fe2e6ffd700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 0x7fe2e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.374+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fe2f00ccdd0 con 0x7fe2f4100db0 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.374+0000 7fe2f9d5d700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 0x7fe2e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.375+0000 7fe2f9d5d700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 0x7fe2e00a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8005950 tx=0x7fe2e80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.377+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe2f00977b0 con 0x7fe2f4100db0 2023-07-21T22:11:20.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:11:20.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:11:20.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] boot 2023-07-21T22:11:20.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:11:20.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:20.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:20 smithi093 ceph-mon[96579]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:20.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:11:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:11:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] boot 2023-07-21T22:11:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:11:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:20 smithi194 ceph-mon[98288]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:20.609 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:20.568+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2f4060950 con 0x7fe2e00a1150 2023-07-21T22:11:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:11:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:11:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: osd.3 [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] boot 2023-07-21T22:11:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:11:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:11:20.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:20 smithi134 ceph-mon[97873]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:11:21.402 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:21 smithi134 ceph-mon[97873]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:11:21.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:21 smithi134 ceph-mon[97873]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:21.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:21.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:21.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:21.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:21 smithi093 ceph-mon[96579]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:11:21.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:21 smithi093 ceph-mon[96579]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:21.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:21.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:21.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:21.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:21 smithi194 ceph-mon[98288]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:11:21.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:21 smithi194 ceph-mon[98288]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:21.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:21.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:21.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:22.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:22 smithi093 ceph-mon[96579]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:11:22.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:22 smithi093 ceph-mon[96579]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:22.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:22 smithi093 ceph-mon[96579]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:11:22.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:22 smithi194 ceph-mon[98288]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:11:22.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:22 smithi194 ceph-mon[98288]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:22.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:22 smithi194 ceph-mon[98288]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:11:22.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:22 smithi134 ceph-mon[97873]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:11:22.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:22 smithi134 ceph-mon[97873]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:22.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:22 smithi134 ceph-mon[97873]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:11:23.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:23 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/4161043921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:23 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:23 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]': finished 2023-07-21T22:11:23.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:23 smithi093 ceph-mon[96579]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T22:11:23.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:23.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:23 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/4161043921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:23 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:23 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]': finished 2023-07-21T22:11:23.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:23 smithi194 ceph-mon[98288]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T22:11:23.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:23.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:23 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/4161043921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:23 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]: dispatch 2023-07-21T22:11:23.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:23 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bfd6964-22c3-4847-b0a2-095f09b257e7"}]': finished 2023-07-21T22:11:23.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:23 smithi134 ceph-mon[97873]: osdmap e29: 5 total, 4 up, 5 in 2023-07-21T22:11:23.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:24.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:24 smithi093 ceph-mon[96579]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:11:24.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:24 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/3175051196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:24.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:24 smithi194 ceph-mon[98288]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:11:24.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:24 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/3175051196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:24.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:24 smithi134 ceph-mon[97873]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:11:24.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:24 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/3175051196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:26.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:26 smithi194 ceph-mon[98288]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T22:11:26.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:26 smithi134 ceph-mon[97873]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T22:11:26.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:26 smithi093 ceph-mon[96579]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T22:11:28.304 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:27 smithi134 ceph-mon[97873]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-07-21T22:11:28.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:27 smithi194 ceph-mon[98288]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-07-21T22:11:28.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:27 smithi093 ceph-mon[96579]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-07-21T22:11:30.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:30 smithi134 ceph-mon[97873]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:11:30.546 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:11:30.547 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:30.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:30 smithi194 ceph-mon[98288]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:11:30.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:11:30.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:30.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:30 smithi093 ceph-mon[96579]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:11:30.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:11:30.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:31.475 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:31 smithi134 ceph-mon[97873]: Deploying daemon osd.4 on smithi134 2023-07-21T22:11:31.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:31 smithi194 ceph-mon[98288]: Deploying daemon osd.4 on smithi134 2023-07-21T22:11:31.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:31 smithi093 ceph-mon[96579]: Deploying daemon osd.4 on smithi134 2023-07-21T22:11:32.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:32 smithi194 ceph-mon[98288]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:32.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:32 smithi134 ceph-mon[97873]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:32 smithi093 ceph-mon[96579]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:34 smithi194 ceph-mon[98288]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:34.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:34 smithi134 ceph-mon[97873]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:34.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:34 smithi093 ceph-mon[96579]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:34.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:36.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:36 smithi194 ceph-mon[98288]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:36.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:36 smithi134 ceph-mon[97873]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:36 smithi093 ceph-mon[96579]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:36.767 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.725+0000 7fe2e6ffd700 1 -- 172.21.15.134:0/4096899024 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe2f4060950 con 0x7fe2e00a1150 2023-07-21T22:11:36.768 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2023-07-21T22:11:36.769 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 msgr2=0x7fe2e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 0x7fe2e00a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8005950 tx=0x7fe2e80058e0 comp rx=0 tx=0).stop 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 msgr2=0x7fe2f4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0000c00 tx=0x7fe2f000f710 comp rx=0 tx=0).stop 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 shutdown_connections 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2f4101740 0x7fe2f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:36.770 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2f40ffbb0 0x7fe2f40742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:36.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2e00a1150 0x7fe2e00a3600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:36.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 --2- 172.21.15.134:0/4096899024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2f4100db0 0x7fe2f4072900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:36.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 >> 172.21.15.134:0/4096899024 conn(0x7fe2f40fb1c0 msgr2=0x7fe2f4104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:36.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 shutdown_connections 2023-07-21T22:11:36.771 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:11:36.727+0000 7fe2fbfc1700 1 -- 172.21.15.134:0/4096899024 wait complete. 2023-07-21T22:11:37.190 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.4.service 2023-07-21T22:11:37.193 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_3... 2023-07-21T22:11:37.193 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:11:37.223 INFO:journalctl@ceph.osd.4.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:11:37.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:37.757 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:11:38.377 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:11:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:11:37.985+0000 7fb3d145f640 -1 osd.4 0 log_to_monitors true 2023-07-21T22:11:38.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:38 smithi134 ceph-mon[97873]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:38.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:38 smithi134 ceph-mon[97873]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:38.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:38 smithi134 ceph-mon[97873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:38.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:38 smithi194 ceph-mon[98288]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:38.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:38 smithi194 ceph-mon[98288]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:38.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:38 smithi194 ceph-mon[98288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:38.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:38 smithi093 ceph-mon[96579]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:38.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:38 smithi093 ceph-mon[96579]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:38.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:38 smithi093 ceph-mon[96579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:11:39.280 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:11:39.281 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:11:39.281 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2023-07-21T22:11:39.281 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2023-07-21T22:11:39.281 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011147 s, 941 MB/s 2023-07-21T22:11:39.281 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 2023-07-21T22:11:39.282 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2023-07-21T22:11:39.710 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2023-07-21T22:11:40.276 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:11:40.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:11:40.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:11:40.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:40.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi134 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.377 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:11:39.941+0000 7fb3c62d6700 -1 osd.4 0 waiting for initial osdmap 2023-07-21T22:11:40.377 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:11:39.948+0000 7fb3c28cc700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi134 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:11:40.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi134 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:40.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:40.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:40.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:41.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:40 smithi194 ceph-mon[98288]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:11:41.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:40 smithi194 ceph-mon[98288]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:11:41.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:40 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:40 smithi134 ceph-mon[97873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:11:41.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:40 smithi134 ceph-mon[97873]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:11:41.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:40 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:40 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:40 smithi093 ceph-mon[96579]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:11:41.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:40 smithi093 ceph-mon[96579]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:11:41.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:40 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:41.631 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3879675476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b00ffbb0 msgr2=0x7fc7b00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:41.631 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3879675476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00fffc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009990 tx=0x7fc7a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3879675476 shutdown_connections 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3879675476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0101740 0x7fc7b0108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3879675476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00fffc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3879675476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b0100db0 0x7fc7b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3879675476 >> 172.21.15.194:0/3879675476 conn(0x7fc7b00fb1c0 msgr2=0x7fc7b00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:41.632 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3879675476 shutdown_connections 2023-07-21T22:11:41.633 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.589+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3879675476 wait complete. 2023-07-21T22:11:41.634 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 Processor -- start 2023-07-21T22:11:41.634 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 -- start start 2023-07-21T22:11:41.634 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 0x7fc7b0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b00735c0 con 0x7fc7b0100db0 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0073730 con 0x7fc7b00ffbb0 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7b4d6e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019f940 con 0x7fc7b0101740 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:41.635 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47118/0 (socket says 172.21.15.194:47118) 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 -- 172.21.15.194:0/3068360824 learned_addr learned my addr 172.21.15.194:0/3068360824 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7ae59c700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 -- 172.21.15.194:0/3068360824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b00ffbb0 msgr2=0x7fc7b00742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7add9b700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 0x7fc7b0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 -- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 msgr2=0x7fc7b0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 0x7fc7b0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:41.636 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7aed9d700 1 -- 172.21.15.194:0/3068360824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a40095f0 con 0x7fc7b0101740 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.590+0000 7fc7add9b700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 0x7fc7b0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc7aed9d700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800ada0 tx=0x7fc7a800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800f070 con 0x7fc7b0101740 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b019fc20 con 0x7fc7b0101740 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a8007b30 con 0x7fc7b0101740 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80053e0 con 0x7fc7b0101740 2023-07-21T22:11:41.637 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.591+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01a0170 con 0x7fc7b0101740 2023-07-21T22:11:41.638 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.592+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b01071a0 con 0x7fc7b0101740 2023-07-21T22:11:41.638 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.593+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc7a800c980 con 0x7fc7b0101740 2023-07-21T22:11:41.638 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.593+0000 7fc79f7fe700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 0x7fc7980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:41.638 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.594+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fc7a80cae20 con 0x7fc7b0101740 2023-07-21T22:11:41.638 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.594+0000 7fc7ae59c700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 0x7fc7980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:41.639 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.594+0000 7fc7ae59c700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 0x7fc7980a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b5c0 tx=0x7fc7a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:41.639 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.598+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc7a8095670 con 0x7fc7b0101740 2023-07-21T22:11:41.832 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:41.790+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7b0060950 con 0x7fc7980a1150 2023-07-21T22:11:42.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:11:42.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:11:42.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] boot 2023-07-21T22:11:42.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T22:11:42.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:42.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:42.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:42.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:42.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:42.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:11:42.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:11:42.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] boot 2023-07-21T22:11:42.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T22:11:42.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:42.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:42.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:42.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:42.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:42.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:11:42.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:11:42.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: osd.4 [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] boot 2023-07-21T22:11:42.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: osdmap e32: 5 total, 5 up, 5 in 2023-07-21T22:11:42.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:11:42.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:11:42.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:11:42.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:11:42.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:43.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:42 smithi194 ceph-mon[98288]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:43.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:42 smithi194 ceph-mon[98288]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:11:43.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:42 smithi134 ceph-mon[97873]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:43.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:42 smithi134 ceph-mon[97873]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:11:43.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:42 smithi093 ceph-mon[96579]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:11:43.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:42 smithi093 ceph-mon[96579]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:11:44.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:43 smithi194 ceph-mon[98288]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:44.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:43 smithi134 ceph-mon[97873]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:44.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:43 smithi093 ceph-mon[96579]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:45.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/946815195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]': finished 2023-07-21T22:11:45.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T22:11:45.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:11:45.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:44 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/663923362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:45.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/946815195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]': finished 2023-07-21T22:11:45.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T22:11:45.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:11:45.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:44 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/663923362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:45.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/946815195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]: dispatch 2023-07-21T22:11:45.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ece1a413-eba9-4d35-bc27-352be61010b1"}]': finished 2023-07-21T22:11:45.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: osdmap e34: 6 total, 5 up, 6 in 2023-07-21T22:11:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:11:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/663923362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:11:46.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:45 smithi194 ceph-mon[98288]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:46.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:45 smithi134 ceph-mon[97873]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:46.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:45 smithi093 ceph-mon[96579]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:48.266 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:47 smithi194 ceph-mon[98288]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:47 smithi134 ceph-mon[97873]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:48.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:47 smithi093 ceph-mon[96579]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:50.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:50 smithi194 ceph-mon[98288]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:50.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:50 smithi134 ceph-mon[97873]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:50.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:50 smithi093 ceph-mon[96579]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:51.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:11:51.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:51 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:51.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:11:51.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:51 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:51.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:11:51.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:51 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:52.439 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:52 smithi194 ceph-mon[98288]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:52.439 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:52 smithi194 ceph-mon[98288]: Deploying daemon osd.5 on smithi194 2023-07-21T22:11:52.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:52 smithi134 ceph-mon[97873]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:52.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:52 smithi134 ceph-mon[97873]: Deploying daemon osd.5 on smithi194 2023-07-21T22:11:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:52 smithi093 ceph-mon[96579]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:52 smithi093 ceph-mon[96579]: Deploying daemon osd.5 on smithi194 2023-07-21T22:11:54.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:54 smithi194 ceph-mon[98288]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:54.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:54 smithi134 ceph-mon[97873]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:54.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:54 smithi093 ceph-mon[96579]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:56.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:56 smithi194 ceph-mon[98288]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:56.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:56.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:56 smithi134 ceph-mon[97873]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:56.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:56.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:56 smithi093 ceph-mon[96579]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:56.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:56.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:57 smithi194 ceph-mon[98288]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:57 smithi134 ceph-mon[97873]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:58.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:57 smithi093 ceph-mon[96579]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:11:58.892 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.850+0000 7fc79f7fe700 1 -- 172.21.15.194:0/3068360824 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc7b0060950 con 0x7fc7980a1150 2023-07-21T22:11:58.893 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 msgr2=0x7fc7980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 0x7fc7980a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b5c0 tx=0x7fc7a4005fd0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 msgr2=0x7fc7b0072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800ada0 tx=0x7fc7a800c330 comp rx=0 tx=0).stop 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 shutdown_connections 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7b0101740 0x7fc7b0072e40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.895 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7980a1150 0x7fc7980a3600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.896 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b00ffbb0 0x7fc7b00742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.896 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 --2- 172.21.15.194:0/3068360824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7b0100db0 0x7fc7b0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.896 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 >> 172.21.15.194:0/3068360824 conn(0x7fc7b00fb1c0 msgr2=0x7fc7b01042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:58.896 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 shutdown_connections 2023-07-21T22:11:58.896 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:11:58.852+0000 7fc7b4d6e700 1 -- 172.21.15.194:0/3068360824 wait complete. 2023-07-21T22:11:59.269 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.5.service 2023-07-21T22:11:59.272 INFO:tasks.cephadm:Deploying osd.6 on smithi093 with /dev/vg_nvme/lv_2... 2023-07-21T22:11:59.272 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:11:59.301 INFO:journalctl@ceph.osd.5.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.320 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:59.321 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.321 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:11:59 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:59.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:11:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:11:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:11:59 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:11:59.854 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:11:59.974 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:11:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:11:59.795+0000 7f7abd31b640 -1 osd.5 0 log_to_monitors true 2023-07-21T22:12:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:00 smithi194 ceph-mon[98288]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:00.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:00 smithi194 ceph-mon[98288]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:12:00.372 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:00 smithi093 ceph-mon[96579]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:00.373 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:00 smithi093 ceph-mon[96579]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:12:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:00 smithi134 ceph-mon[97873]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:00.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:00 smithi134 ceph-mon[97873]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:12:01.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:01 smithi093 ceph-mon[96579]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:12:01.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:01 smithi093 ceph-mon[96579]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:12:01.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:01 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:01.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:01 smithi093 ceph-mon[96579]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:12:01.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-mon[98288]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:12:01.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-mon[98288]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:12:01.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:01.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-mon[98288]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:12:01.337 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:12:01.043+0000 7f7ab2192700 -1 osd.5 0 waiting for initial osdmap 2023-07-21T22:12:01.337 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:12:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:12:01.054+0000 7f7aaa780700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:12:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:12:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:12:01.360 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-21T22:12:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-21T22:12:01.361 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128192 s, 818 MB/s 2023-07-21T22:12:01.361 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-21T22:12:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:01 smithi134 ceph-mon[97873]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:12:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:01 smithi134 ceph-mon[97873]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:12:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:01 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:01.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:01 smithi134 ceph-mon[97873]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:12:01.806 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_2 2023-07-21T22:12:02.358 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:12:02.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:12:02.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:12:02.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi194 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 8584M 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] boot 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T22:12:02.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi194 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 8584M 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:02.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:02.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] boot 2023-07-21T22:12:02.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T22:12:02.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:12:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:12:02.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi194 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 8584M 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: osd.5 [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] boot 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: osdmap e37: 6 total, 6 up, 6 in 2023-07-21T22:12:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:12:03.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:03 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:12:03.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:03 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:12:03.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:03 smithi093 ceph-mon[96579]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:12:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:03 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:12:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:03 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:12:03.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:03 smithi134 ceph-mon[97873]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:12:03.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 -- 172.21.15.93:0/3582318701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 msgr2=0x7ff1900fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 --2- 172.21.15.93:0/3582318701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff1900fee90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b0a0 tx=0x7ff18800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 -- 172.21.15.93:0/3582318701 shutdown_connections 2023-07-21T22:12:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 --2- 172.21.15.93:0/3582318701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff190100610 0x7ff190106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 --2- 172.21.15.93:0/3582318701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff1900fee90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 --2- 172.21.15.93:0/3582318701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1900ffc80 0x7ff1901000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 -- 172.21.15.93:0/3582318701 >> 172.21.15.93:0/3582318701 conn(0x7ff1900fa0b0 msgr2=0x7ff1900fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 -- 172.21.15.93:0/3582318701 shutdown_connections 2023-07-21T22:12:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.526+0000 7ff1984df700 1 -- 172.21.15.93:0/3582318701 wait complete. 2023-07-21T22:12:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 Processor -- start 2023-07-21T22:12:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 -- start start 2023-07-21T22:12:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 0x7ff19019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190100610 0x7ff19019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19019f800 con 0x7ff1900fea80 2023-07-21T22:12:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19019f970 con 0x7ff190100610 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff1984df700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19019fb30 con 0x7ff1900ffc80 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41400/0 (socket says 172.21.15.93:41400) 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 -- 172.21.15.93:0/2115616379 learned_addr learned my addr 172.21.15.93:0/2115616379 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff195a7a700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 0x7ff19019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 -- 172.21.15.93:0/2115616379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 msgr2=0x7ff19019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 0x7ff19019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 -- 172.21.15.93:0/2115616379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190100610 msgr2=0x7ff19019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190100610 0x7ff19019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 -- 172.21.15.93:0/2115616379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff188009d00 con 0x7ff1900fea80 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff195a7a700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 0x7ff19019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.527+0000 7ff19627b700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff188004d00 tx=0x7ff188004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800e040 con 0x7ff1900fea80 2023-07-21T22:12:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19019fdb0 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18801bd20 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188005960 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1901a0290 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.528+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1901060d0 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.529+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff188005040 con 0x7ff1900fea80 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.530+0000 7ff18f7fe700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 0x7ff1780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.530+0000 7ff195a7a700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 0x7ff1780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.530+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7ff1880cd810 con 0x7ff1900fea80 2023-07-21T22:12:03.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.530+0000 7ff195a7a700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 0x7ff1780a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1900690b0 tx=0x7ff184008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:03.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.534+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff188097e90 con 0x7ff1900fea80 2023-07-21T22:12:03.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:03 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:12:03.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:03 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:12:03.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:03 smithi194 ceph-mon[98288]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:12:03.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:03.730+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff190060950 con 0x7ff1780a1150 2023-07-21T22:12:04.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:04 smithi134 ceph-mon[97873]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:04.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:04.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:04.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:04.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:04 smithi134 ceph-mon[97873]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:12:04.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:04 smithi093 ceph-mon[96579]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:04.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:04.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:04.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:04.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:04 smithi093 ceph-mon[96579]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:12:04.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:04 smithi194 ceph-mon[98288]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:04.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:04.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:04.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:04.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:04 smithi194 ceph-mon[98288]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:12:05.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:05 smithi093 ceph-mon[96579]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:05.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:05 smithi134 ceph-mon[97873]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:05.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:05 smithi194 ceph-mon[98288]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:06.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:06 smithi134 ceph-mon[97873]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:12:06.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:06 smithi134 ceph-mon[97873]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:06.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:06 smithi093 ceph-mon[96579]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:12:06.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:06 smithi093 ceph-mon[96579]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:06.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:06 smithi194 ceph-mon[98288]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:12:06.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:06 smithi194 ceph-mon[98288]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:07.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:12:07.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]: dispatch 2023-07-21T22:12:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]': finished 2023-07-21T22:12:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:12:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:07 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/668183126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:07.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:12:07.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]: dispatch 2023-07-21T22:12:07.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]': finished 2023-07-21T22:12:07.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:12:07.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:07.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:07 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/668183126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:07.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:12:07.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]: dispatch 2023-07-21T22:12:07.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1525188483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05d761d-7844-4e6d-99e7-94acab20daad"}]': finished 2023-07-21T22:12:07.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:12:07.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:07.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:07 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/668183126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:08.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:08 smithi093 ceph-mon[96579]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:12:08.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:08 smithi194 ceph-mon[98288]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:12:08.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:08 smithi134 ceph-mon[97873]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:12:10.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:10 smithi093 ceph-mon[96579]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-07-21T22:12:10.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:10 smithi194 ceph-mon[98288]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-07-21T22:12:10.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:10 smithi134 ceph-mon[97873]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2023-07-21T22:12:12.571 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:12 smithi093 ceph-mon[96579]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-07-21T22:12:12.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:12 smithi194 ceph-mon[98288]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-07-21T22:12:12.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:12 smithi134 ceph-mon[97873]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-07-21T22:12:13.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:12:13.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:13 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:13.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:12:13.608 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:13 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:13.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:12:13.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:13 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:14.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:14 smithi194 ceph-mon[98288]: Deploying daemon osd.6 on smithi093 2023-07-21T22:12:14.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:14 smithi194 ceph-mon[98288]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:12:14.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:14 smithi134 ceph-mon[97873]: Deploying daemon osd.6 on smithi093 2023-07-21T22:12:14.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:14 smithi134 ceph-mon[97873]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:12:14.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:14 smithi093 ceph-mon[96579]: Deploying daemon osd.6 on smithi093 2023-07-21T22:12:14.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:14 smithi093 ceph-mon[96579]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:12:16.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:16 smithi194 ceph-mon[98288]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-21T22:12:16.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:16 smithi134 ceph-mon[97873]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-21T22:12:16.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:16 smithi093 ceph-mon[96579]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-21T22:12:18.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:18 smithi093 ceph-mon[96579]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T22:12:18.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:18 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:18 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:18 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:18 smithi194 ceph-mon[98288]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T22:12:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:18 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:18.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:18 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:18 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:18.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:18 smithi134 ceph-mon[97873]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-07-21T22:12:18.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:18 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:20.121 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 6 on host 'smithi093' 2023-07-21T22:12:20.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.079+0000 7ff18f7fe700 1 -- 172.21.15.93:0/2115616379 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff190060950 con 0x7ff1780a1150 2023-07-21T22:12:20.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 msgr2=0x7ff1780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 0x7ff1780a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff1900690b0 tx=0x7ff184008040 comp rx=0 tx=0).stop 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 msgr2=0x7ff190068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff188004d00 tx=0x7ff188004de0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 shutdown_connections 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1780a1150 0x7ff1780a3600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190100610 0x7ff19019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1900fea80 0x7ff190068b30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 --2- 172.21.15.93:0/2115616379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1900ffc80 0x7ff19019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 >> 172.21.15.93:0/2115616379 conn(0x7ff1900fa0b0 msgr2=0x7ff190103210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:20.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 shutdown_connections 2023-07-21T22:12:20.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:12:20.081+0000 7ff1984df700 1 -- 172.21.15.93:0/2115616379 wait complete. 2023-07-21T22:12:20.455 DEBUG:teuthology.orchestra.run.smithi093:osd.6> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.6.service 2023-07-21T22:12:20.457 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_2... 2023-07-21T22:12:20.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:12:20.494 INFO:journalctl@ceph.osd.6.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:12:20.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:20.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:20 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:20.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:20.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:20 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:20.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:20 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:21.040 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:12:21.403 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:12:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:12:21.146+0000 7f7deea52640 -1 osd.6 0 log_to_monitors true 2023-07-21T22:12:21.672 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:21 smithi093 ceph-mon[96579]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:12:21.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:21 smithi134 ceph-mon[97873]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:12:21.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:21 smithi194 ceph-mon[98288]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:12:22.497 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:12:22.497 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:12:22.497 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-21T22:12:22.498 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-21T22:12:22.498 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124767 s, 840 MB/s 2023-07-21T22:12:22.498 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-21T22:12:22.689 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:12:22.515+0000 7f7de38c9700 -1 osd.6 0 waiting for initial osdmap 2023-07-21T22:12:22.689 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:12:22.525+0000 7f7dde6bc700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:12:22.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-mon[96579]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:22.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-mon[96579]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:12:22.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-mon[96579]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:12:22.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:22.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:22 smithi093 ceph-mon[96579]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:12:22.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:22 smithi194 ceph-mon[98288]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:22.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:22 smithi194 ceph-mon[98288]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:12:22.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:22 smithi194 ceph-mon[98288]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:12:22.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:22.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:22 smithi194 ceph-mon[98288]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:12:22.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:22 smithi134 ceph-mon[97873]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:22.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:22 smithi134 ceph-mon[97873]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:12:22.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:22 smithi134 ceph-mon[97873]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:12:22.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:22.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:22 smithi134 ceph-mon[97873]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:12:22.884 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2023-07-21T22:12:23.433 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:12:23.834 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.835 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:23.836 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:23.837 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:23 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:12:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:12:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:23 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:12:23.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:12:23.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:23.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:23.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:23.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:23 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:24.681 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 -- 172.21.15.134:0/1853895630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 msgr2=0x7fb938106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 --2- 172.21.15.134:0/1853895630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb938106ed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b0a0 tx=0x7fb93000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 -- 172.21.15.134:0/1853895630 shutdown_connections 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 --2- 172.21.15.134:0/1853895630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb938106ed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 --2- 172.21.15.134:0/1853895630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb9380fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 --2- 172.21.15.134:0/1853895630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 0x7fb9381000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 -- 172.21.15.134:0/1853895630 >> 172.21.15.134:0/1853895630 conn(0x7fb9380fa0b0 msgr2=0x7fb9380fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:24.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 -- 172.21.15.134:0/1853895630 shutdown_connections 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 -- 172.21.15.134:0/1853895630 wait complete. 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.638+0000 7fb93df86700 1 Processor -- start 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 -- start start 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb938195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 0x7fb938195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.683 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93819d4d0 con 0x7fb9380fea80 2023-07-21T22:12:24.684 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93819d610 con 0x7fb938100610 2023-07-21T22:12:24.684 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb93df86700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93819d7a0 con 0x7fb9380ffc80 2023-07-21T22:12:24.684 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.684 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:54416/0 (socket says 172.21.15.134:54416) 2023-07-21T22:12:24.684 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 -- 172.21.15.134:0/2718845058 learned_addr learned my addr 172.21.15.134:0/2718845058 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-21T22:12:24.685 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb936ffd700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 0x7fb938195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb9377fe700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb938195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 -- 172.21.15.134:0/2718845058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 msgr2=0x7fb938195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 0x7fb938195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 -- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 msgr2=0x7fb938195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb938195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 -- 172.21.15.134:0/2718845058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009d00 con 0x7fb938100610 2023-07-21T22:12:24.686 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb9377fe700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb938195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.639+0000 7fb937fff700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b780 tx=0x7fb93000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.640+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000e040 con 0x7fb938100610 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.640+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93819da20 con 0x7fb938100610 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.640+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb930004e60 con 0x7fb938100610 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.640+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930005350 con 0x7fb938100610 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.640+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93819dee0 con 0x7fb938100610 2023-07-21T22:12:24.687 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.641+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938106ad0 con 0x7fb938100610 2023-07-21T22:12:24.688 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.642+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9300054b0 con 0x7fb938100610 2023-07-21T22:12:24.688 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.642+0000 7fb934ff9700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 0x7fb9200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.688 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.642+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fb9300cd320 con 0x7fb938100610 2023-07-21T22:12:24.688 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.643+0000 7fb9377fe700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 0x7fb9200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.688 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.643+0000 7fb9377fe700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 0x7fb9200a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005950 tx=0x7fb92c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:24.689 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.647+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb930097810 con 0x7fb938100610 2023-07-21T22:12:24.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:12:24.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:12:24.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi093 2023-07-21T22:12:24.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-21T22:12:24.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:24.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] boot 2023-07-21T22:12:24.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:12:24.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:24.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:12:24.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:12:24.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi093 2023-07-21T22:12:24.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-21T22:12:24.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:24.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] boot 2023-07-21T22:12:24.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:12:24.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:24.883 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:24.840+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb938060950 con 0x7fb9200a1150 2023-07-21T22:12:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:12:24.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:12:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi093 2023-07-21T22:12:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 5722M 2023-07-21T22:12:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:12:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: osd.6 [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] boot 2023-07-21T22:12:24.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:12:24.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:12:25.971 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:12:25.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:25.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:25.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:25.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:25.972 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:25 smithi134 ceph-mon[97873]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:26.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:12:26.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:26.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:26.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:26.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:26.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:25 smithi194 ceph-mon[98288]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:26.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:12:26.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:26.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:26.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:26.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:26.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:25 smithi093 ceph-mon[96579]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:28.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:28.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/478751205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]': finished 2023-07-21T22:12:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:12:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:27 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/329950091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:28.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/478751205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]': finished 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:27 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/329950091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:28.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/478751205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]: dispatch 2023-07-21T22:12:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1147542a-1a5b-4935-99ae-2c196c6bb096"}]': finished 2023-07-21T22:12:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:12:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:27 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/329950091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:30.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:30 smithi194 ceph-mon[98288]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:30.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:30 smithi134 ceph-mon[97873]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:30.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:30 smithi093 ceph-mon[96579]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:32.524 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:32 smithi134 ceph-mon[97873]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:32.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:32 smithi194 ceph-mon[98288]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:32.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:32 smithi093 ceph-mon[96579]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:34.544 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:34 smithi134 ceph-mon[97873]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:34.544 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:12:34.544 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:34 smithi194 ceph-mon[98288]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:12:34.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:34.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:34 smithi093 ceph-mon[96579]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:34.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:12:34.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:35.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:35 smithi194 ceph-mon[98288]: Deploying daemon osd.7 on smithi134 2023-07-21T22:12:35.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:35 smithi134 ceph-mon[97873]: Deploying daemon osd.7 on smithi134 2023-07-21T22:12:35.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:35 smithi093 ceph-mon[96579]: Deploying daemon osd.7 on smithi134 2023-07-21T22:12:36.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:36 smithi194 ceph-mon[98288]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:36.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:36 smithi134 ceph-mon[97873]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:36 smithi093 ceph-mon[96579]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:38.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:37 smithi194 ceph-mon[98288]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:38.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:37 smithi134 ceph-mon[97873]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:38.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:37 smithi093 ceph-mon[96579]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:39.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.286 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:39.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:39.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:39.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:40.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:40 smithi134 ceph-mon[97873]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:40.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:40 smithi093 ceph-mon[96579]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:40 smithi194 ceph-mon[98288]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:40.889 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2023-07-21T22:12:40.889 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.846+0000 7fb934ff9700 1 -- 172.21.15.134:0/2718845058 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb938060950 con 0x7fb9200a1150 2023-07-21T22:12:40.891 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 msgr2=0x7fb9200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:40.891 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 0x7fb9200a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005950 tx=0x7fb92c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.891 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 msgr2=0x7fb93819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb93000b780 tx=0x7fb93000be10 comp rx=0 tx=0).stop 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 shutdown_connections 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9200a1150 0x7fb9200a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb938100610 0x7fb93819cf00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9380fea80 0x7fb938195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 --2- 172.21.15.134:0/2718845058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9380ffc80 0x7fb938195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.892 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.848+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 >> 172.21.15.134:0/2718845058 conn(0x7fb9380fa0b0 msgr2=0x7fb9380fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:40.893 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.849+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 shutdown_connections 2023-07-21T22:12:40.893 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:12:40.849+0000 7fb93df86700 1 -- 172.21.15.134:0/2718845058 wait complete. 2023-07-21T22:12:41.314 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.7.service 2023-07-21T22:12:41.316 INFO:tasks.cephadm:Deploying osd.8 on smithi194 with /dev/vg_nvme/lv_2... 2023-07-21T22:12:41.317 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:12:41.345 INFO:journalctl@ceph.osd.7.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:12:41.881 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:12:41.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.893 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:41.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:42.082 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:42.082 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:42.082 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:42.082 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:42.627 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:12:42 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:12:42.154+0000 7fddc3979640 -1 osd.7 0 log_to_monitors true 2023-07-21T22:12:42.659 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:42 smithi194 ceph-mon[98288]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:42.660 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:42 smithi194 ceph-mon[98288]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:42.660 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:42 smithi194 ceph-mon[98288]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:42.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:42 smithi093 ceph-mon[96579]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:42.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:42 smithi093 ceph-mon[96579]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:42.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:42 smithi093 ceph-mon[96579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:42 smithi134 ceph-mon[97873]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:42 smithi134 ceph-mon[97873]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:43.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:42 smithi134 ceph-mon[97873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:12:43.314 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:12:43.314 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:12:43.314 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2023-07-21T22:12:43.314 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2023-07-21T22:12:43.314 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110349 s, 950 MB/s 2023-07-21T22:12:43.315 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2023-07-21T22:12:43.736 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2023-07-21T22:12:44.295 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:12:44.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:12:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:12:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi134 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-21T22:12:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.377 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:12:43.979+0000 7fddb87f0700 -1 osd.7 0 waiting for initial osdmap 2023-07-21T22:12:44.378 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:12:43.989+0000 7fddb2de2700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:12:44.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:12:44.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:12:44.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:44.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:44.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi134 2023-07-21T22:12:44.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-21T22:12:44.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:12:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi134 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-21T22:12:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:44.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:12:45.114 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:44 smithi194 ceph-mon[98288]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:12:45.114 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:44 smithi194 ceph-mon[98288]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:12:45.115 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.115 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:44 smithi134 ceph-mon[97873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:12:45.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:44 smithi134 ceph-mon[97873]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:12:45.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:44 smithi093 ceph-mon[96579]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:12:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:44 smithi093 ceph-mon[96579]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:12:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:45.607 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- 172.21.15.194:0/418849973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080ffc80 msgr2=0x7f27081000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:45.608 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- 172.21.15.194:0/418849973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080ffc80 0x7f27081000d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009990 tx=0x7f26f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.608 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- 172.21.15.194:0/418849973 shutdown_connections 2023-07-21T22:12:45.608 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- 172.21.15.194:0/418849973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2708100610 0x7f2708106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.609 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- 172.21.15.194:0/418849973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27080fea80 0x7f27080fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.609 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- 172.21.15.194:0/418849973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080ffc80 0x7f27081000d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.609 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- 172.21.15.194:0/418849973 >> 172.21.15.194:0/418849973 conn(0x7f27080fa0b0 msgr2=0x7f27080fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:45.609 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- 172.21.15.194:0/418849973 shutdown_connections 2023-07-21T22:12:45.610 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- 172.21.15.194:0/418849973 wait complete. 2023-07-21T22:12:45.610 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 Processor -- start 2023-07-21T22:12:45.610 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 -- start start 2023-07-21T22:12:45.610 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:45.611 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 0x7f2708197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:45.611 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.562+0000 7f270ed50700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 0x7f270819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:45.611 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270ed50700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270819f580 con 0x7f27080ffc80 2023-07-21T22:12:45.611 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270ed50700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270819f6c0 con 0x7f2708100610 2023-07-21T22:12:45.612 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270ed50700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270819f850 con 0x7f27080fea80 2023-07-21T22:12:45.612 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:45.613 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47230/0 (socket says 172.21.15.194:47230) 2023-07-21T22:12:45.613 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 -- 172.21.15.194:0/4165768664 learned_addr learned my addr 172.21.15.194:0/4165768664 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:12:45.613 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270d2ed700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 0x7f270819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:45.613 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f2707fff700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 0x7f2708197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:45.614 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 -- 172.21.15.194:0/4165768664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 msgr2=0x7f270819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:45.614 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 0x7f270819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.614 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 -- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 msgr2=0x7f2708197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:45.614 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 0x7f2708197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:45.615 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 -- 172.21.15.194:0/4165768664 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40095f0 con 0x7f27080fea80 2023-07-21T22:12:45.615 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270d2ed700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 0x7f270819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:12:45.615 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f2707fff700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 0x7f2708197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:12:45.616 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270caec700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00d9b0 tx=0x7f26fc00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:45.616 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00be80 con 0x7f27080fea80 2023-07-21T22:12:45.616 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26fc0044e0 con 0x7f27080fea80 2023-07-21T22:12:45.616 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270819fb30 con 0x7f27080fea80 2023-07-21T22:12:45.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.563+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc003b40 con 0x7f27080fea80 2023-07-21T22:12:45.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.564+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27081a0080 con 0x7f27080fea80 2023-07-21T22:12:45.617 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.564+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270810a240 con 0x7f27080fea80 2023-07-21T22:12:45.619 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.565+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f26fc009940 con 0x7f27080fea80 2023-07-21T22:12:45.619 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.566+0000 7f2705ffb700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 0x7f26f80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:45.619 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.566+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f26fc0cbb10 con 0x7f27080fea80 2023-07-21T22:12:45.619 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.566+0000 7f2707fff700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 0x7f26f80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:45.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.566+0000 7f2707fff700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 0x7f26f80a3680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f26f400b5c0 tx=0x7f26f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:45.620 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.570+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f26fc095da0 con 0x7f27080fea80 2023-07-21T22:12:45.804 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:12:45.762+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f27080686e0 con 0x7f26f80a11d0 2023-07-21T22:12:46.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:12:46.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] boot 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:45 smithi194 ceph-mon[98288]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:12:46.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] boot 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:45 smithi134 ceph-mon[97873]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:12:46.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:12:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:12:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: osd.7 [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] boot 2023-07-21T22:12:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:12:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:12:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:12:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:12:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:45 smithi093 ceph-mon[96579]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:12:47.180 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:46 smithi194 ceph-mon[98288]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:47.180 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:46 smithi194 ceph-mon[98288]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:12:47.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:46 smithi134 ceph-mon[97873]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:47.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:46 smithi134 ceph-mon[97873]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:12:47.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:46 smithi093 ceph-mon[96579]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:47.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:46 smithi093 ceph-mon[96579]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:12:48.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:48 smithi194 ceph-mon[98288]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:48.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:48 smithi134 ceph-mon[97873]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:48.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:48 smithi093 ceph-mon[96579]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:49.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/1298596341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]': finished 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:12:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:49 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/3118854978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:49.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:12:49.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/1298596341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]': finished 2023-07-21T22:12:49.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:12:49.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:12:49.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:49 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/3118854978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:49.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:12:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/1298596341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]: dispatch 2023-07-21T22:12:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34cab3d-d40c-47b9-a691-b95f36965de5"}]': finished 2023-07-21T22:12:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:12:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:12:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:49 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/3118854978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:12:50.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:50 smithi194 ceph-mon[98288]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:50.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:50 smithi134 ceph-mon[97873]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:50.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:50 smithi093 ceph-mon[96579]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:51.689 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:12:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:12:51.159+0000 7fed20f3b700 -1 calc_pg_upmaps abort due to max <= 0 2023-07-21T22:12:52.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:52 smithi194 ceph-mon[98288]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:52.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:52 smithi134 ceph-mon[97873]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:52.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:52 smithi093 ceph-mon[96579]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:54 smithi194 ceph-mon[98288]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:54.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:54 smithi134 ceph-mon[97873]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:54.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:54 smithi093 ceph-mon[96579]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:55.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:12:55.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:55.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:12:55.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:55.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:12:55.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:56.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:56 smithi194 ceph-mon[98288]: Deploying daemon osd.8 on smithi194 2023-07-21T22:12:56.547 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:56 smithi194 ceph-mon[98288]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:56.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:56 smithi134 ceph-mon[97873]: Deploying daemon osd.8 on smithi194 2023-07-21T22:12:56.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:56 smithi134 ceph-mon[97873]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:56.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:56 smithi093 ceph-mon[96579]: Deploying daemon osd.8 on smithi194 2023-07-21T22:12:56.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:56 smithi093 ceph-mon[96579]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:58.335 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:12:57 smithi194 ceph-mon[98288]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:58.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:12:57 smithi134 ceph-mon[97873]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:12:58.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:12:57 smithi093 ceph-mon[96579]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:00.571 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:00 smithi194 ceph-mon[98288]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:00.571 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:00 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.571 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:00 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.571 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:00 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:00.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:00 smithi093 ceph-mon[96579]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:00.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:00 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:00 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:00 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:00.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:00 smithi134 ceph-mon[97873]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:00.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:00 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:00 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:00.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:00 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:02.264 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.221+0000 7f2705ffb700 1 -- 172.21.15.194:0/4165768664 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27080686e0 con 0x7f26f80a11d0 2023-07-21T22:13:02.265 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 8 on host 'smithi194' 2023-07-21T22:13:02.267 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 msgr2=0x7f26f80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:02.267 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 0x7f26f80a3680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f26f400b5c0 tx=0x7f26f4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:13:02.267 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 msgr2=0x7f27081979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:02.267 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00d9b0 tx=0x7f26fc00dd70 comp rx=0 tx=0).stop 2023-07-21T22:13:02.268 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 shutdown_connections 2023-07-21T22:13:02.268 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f26f80a11d0 0x7f26f80a3680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:02.268 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2708100610 0x7f270819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:02.269 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27080fea80 0x7f27081979f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:02.269 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 --2- 172.21.15.194:0/4165768664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27080ffc80 0x7f2708197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:02.269 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 >> 172.21.15.194:0/4165768664 conn(0x7f27080fa0b0 msgr2=0x7f2708067fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:02.269 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 shutdown_connections 2023-07-21T22:13:02.270 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:02.224+0000 7f270ed50700 1 -- 172.21.15.194:0/4165768664 wait complete. 2023-07-21T22:13:02.679 DEBUG:teuthology.orchestra.run.smithi194:osd.8> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.8.service 2023-07-21T22:13:02.681 INFO:tasks.cephadm:Deploying osd.9 on smithi093 with /dev/vg_nvme/lv_1... 2023-07-21T22:13:02.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:13:02.710 INFO:journalctl@ceph.osd.8.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:13:02.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:02.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:02.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:02.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:02 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:03.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:03.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:03.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:03.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:02 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:03.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:03.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:03.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:03.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:03.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:02 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:03.260 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:03.447 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:13:03 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:13:03.298+0000 7f4ba47f5640 -1 osd.8 0 log_to_monitors true 2023-07-21T22:13:03.764 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:03 smithi093 ceph-mon[96579]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:03.764 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:03 smithi093 ceph-mon[96579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:03.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:03 smithi194 ceph-mon[98288]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:03.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:03 smithi194 ceph-mon[98288]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:04.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:03 smithi134 ceph-mon[97873]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:04.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:03 smithi134 ceph-mon[97873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:13:04.724 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:13:04.724 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:13:04.724 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-07-21T22:13:04.725 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-07-21T22:13:04.725 INFO:teuthology.orchestra.run.smithi093.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110636 s, 948 MB/s 2023-07-21T22:13:04.725 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 2023-07-21T22:13:04.725 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-07-21T22:13:04.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:04.987 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:13:04.987 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:13:04.987 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:04.987 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:04.987 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-mon[98288]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:04.988 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:13:04.651+0000 7f4b9966c700 -1 osd.8 0 waiting for initial osdmap 2023-07-21T22:13:04.988 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:13:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:13:04.663+0000 7f4b95461700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:13:05.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:05.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:13:05.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:13:05.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:05.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:05.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:04 smithi093 ceph-mon[96579]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:05.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi093:vg_nvme/lv_1 2023-07-21T22:13:05.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:05.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:13:05.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:13:05.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:05.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: from='osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:05.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:04 smithi134 ceph-mon[97873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:13:05.667 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:06.012 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:13:06.012 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.013 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:06.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:06.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:05 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:13:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:13:06.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:06.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:06.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:05 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:13:06.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:13:06.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:06.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:06.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:05 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:06.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 -- 172.21.15.93:0/3714930840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c0100160 msgr2=0x7f99c0100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:06.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 --2- 172.21.15.93:0/3714930840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c0100160 0x7f99c0100570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009990 tx=0x7f99b0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 -- 172.21.15.93:0/3714930840 shutdown_connections 2023-07-21T22:13:06.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 --2- 172.21.15.93:0/3714930840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 0x7f99c0106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 --2- 172.21.15.93:0/3714930840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c0100160 0x7f99c0100570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 --2- 172.21.15.93:0/3714930840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00fed30 0x7f99c00ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 -- 172.21.15.93:0/3714930840 >> 172.21.15.93:0/3714930840 conn(0x7f99c00f9890 msgr2=0x7f99c00fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:06.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.751+0000 7f99c5500700 1 -- 172.21.15.93:0/3714930840 shutdown_connections 2023-07-21T22:13:06.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 -- 172.21.15.93:0/3714930840 wait complete. 2023-07-21T22:13:06.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 Processor -- start 2023-07-21T22:13:06.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 -- start start 2023-07-21T22:13:06.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:06.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 0x7f99c0195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:06.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c0100160 0x7f99c019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:06.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d480 con 0x7f99c00fed30 2023-07-21T22:13:06.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d5c0 con 0x7f99c0100160 2023-07-21T22:13:06.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99c5500700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d750 con 0x7f99c00ff6e0 2023-07-21T22:13:06.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99beffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:06.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99beffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41534/0 (socket says 172.21.15.93:41534) 2023-07-21T22:13:06.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.752+0000 7f99beffd700 1 -- 172.21.15.93:0/50786842 learned_addr learned my addr 172.21.15.93:0/50786842 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:13:06.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99bf7fe700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c0100160 0x7f99c019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:06.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99be7fc700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 0x7f99c0195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:06.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 -- 172.21.15.93:0/50786842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 msgr2=0x7f99c0195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:06.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 0x7f99c0195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 -- 172.21.15.93:0/50786842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c0100160 msgr2=0x7f99c019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:06.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c0100160 0x7f99c019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:06.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 -- 172.21.15.93:0/50786842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00095f0 con 0x7f99c00fed30 2023-07-21T22:13:06.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99beffd700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005910 tx=0x7f99b000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:06.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b001d070 con 0x7f99c00fed30 2023-07-21T22:13:06.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b000fd10 con 0x7f99c00fed30 2023-07-21T22:13:06.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b00176e0 con 0x7f99c00fed30 2023-07-21T22:13:06.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c019d9d0 con 0x7f99c00fed30 2023-07-21T22:13:06.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.753+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c019dea0 con 0x7f99c00fed30 2023-07-21T22:13:06.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.754+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0106be0 con 0x7f99c00fed30 2023-07-21T22:13:06.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.755+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f99b0022470 con 0x7f99c00fed30 2023-07-21T22:13:06.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.756+0000 7f99a7fff700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 0x7f99a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:06.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.756+0000 7f99be7fc700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 0x7f99a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:06.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.756+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f99b00cd8e0 con 0x7f99c00fed30 2023-07-21T22:13:06.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.756+0000 7f99be7fc700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 0x7f99a80a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006fd0 tx=0x7f99b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:06.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.759+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f99b0097a50 con 0x7f99c00fed30 2023-07-21T22:13:06.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:13:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:13:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi194 2023-07-21T22:13:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 5722M 2023-07-21T22:13:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] boot 2023-07-21T22:13:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:13:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:06.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:06.953+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c0060950 con 0x7f99a80a1150 2023-07-21T22:13:07.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:13:07.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:13:07.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi194 2023-07-21T22:13:07.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 5722M 2023-07-21T22:13:07.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:07.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:07.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] boot 2023-07-21T22:13:07.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:13:07.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:07.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:13:07.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:13:07.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi194 2023-07-21T22:13:07.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 5722M 2023-07-21T22:13:07.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: pgmap v173: 1 pgs: 1 peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:13:07.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:07.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: osd.8 [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] boot 2023-07-21T22:13:07.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:13:07.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:13:07.965 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:13:07.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:07.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:07.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:07.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:07.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:07 smithi093 ceph-mon[96579]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:08.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:13:08.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:08.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:08.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:08.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:08.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:07 smithi194 ceph-mon[98288]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:08.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:13:08.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:08.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:08.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:08.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:08.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:07 smithi134 ceph-mon[97873]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:09.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:08 smithi194 ceph-mon[98288]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:13:09.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:08 smithi134 ceph-mon[97873]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:13:09.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:08 smithi093 ceph-mon[96579]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:13:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:13:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]: dispatch 2023-07-21T22:13:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]': finished 2023-07-21T22:13:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T22:13:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:10.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:13:10.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:10.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]: dispatch 2023-07-21T22:13:10.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]': finished 2023-07-21T22:13:10.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T22:13:10.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:10.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:13:10.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:10.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]: dispatch 2023-07-21T22:13:10.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1577423338' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7aca17f8-875e-421f-a4b8-9b85656aa761"}]': finished 2023-07-21T22:13:10.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: osdmap e61: 10 total, 9 up, 10 in 2023-07-21T22:13:10.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:10 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2094158635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:10 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2094158635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:11.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:10 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2094158635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:12.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:11 smithi194 ceph-mon[98288]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-21T22:13:12.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:11 smithi134 ceph-mon[97873]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-21T22:13:12.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:11 smithi093 ceph-mon[96579]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-07-21T22:13:14.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:14 smithi194 ceph-mon[98288]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:14.607 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:14 smithi093 ceph-mon[96579]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:14.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:14 smithi134 ceph-mon[97873]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:16.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:16 smithi093 ceph-mon[96579]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:13:16.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:13:16.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:16 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:16.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:16 smithi194 ceph-mon[98288]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:13:16.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:13:16.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:16 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:16.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:16 smithi134 ceph-mon[97873]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:13:16.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:13:16.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:16 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:17.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:17 smithi194 ceph-mon[98288]: Deploying daemon osd.9 on smithi093 2023-07-21T22:13:17.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:17 smithi134 ceph-mon[97873]: Deploying daemon osd.9 on smithi093 2023-07-21T22:13:17.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:17 smithi093 ceph-mon[96579]: Deploying daemon osd.9 on smithi093 2023-07-21T22:13:18.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:18 smithi093 ceph-mon[96579]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:13:18.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:18 smithi194 ceph-mon[98288]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:13:18.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:18 smithi134 ceph-mon[97873]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:13:20.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:20 smithi093 ceph-mon[96579]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:13:20.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:20 smithi194 ceph-mon[98288]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:13:20.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:20 smithi134 ceph-mon[97873]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:13:21.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:21 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:21.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:21 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:21.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:21.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:21 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:22.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:22 smithi093 ceph-mon[96579]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:13:22.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:22 smithi134 ceph-mon[97873]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:13:22.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:22 smithi194 ceph-mon[98288]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:13:23.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.051+0000 7f99a7fff700 1 -- 172.21.15.93:0/50786842 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f99c0060950 con 0x7f99a80a1150 2023-07-21T22:13:23.095 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 9 on host 'smithi093' 2023-07-21T22:13:23.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 msgr2=0x7f99a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 0x7f99a80a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006fd0 tx=0x7f99b4008040 comp rx=0 tx=0).stop 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 msgr2=0x7f99c01958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005910 tx=0x7f99b000f830 comp rx=0 tx=0).stop 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 shutdown_connections 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f99a80a1150 0x7f99a80a3600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c0100160 0x7f99c019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99c00fed30 0x7f99c01958f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 --2- 172.21.15.93:0/50786842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99c00ff6e0 0x7f99c0195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 >> 172.21.15.93:0/50786842 conn(0x7f99c00f9890 msgr2=0x7f99c00fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:23.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 shutdown_connections 2023-07-21T22:13:23.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:13:23.054+0000 7f99c5500700 1 -- 172.21.15.93:0/50786842 wait complete. 2023-07-21T22:13:23.455 DEBUG:teuthology.orchestra.run.smithi093:osd.9> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.9.service 2023-07-21T22:13:23.458 INFO:tasks.cephadm:Deploying osd.10 on smithi134 with /dev/vg_nvme/lv_1... 2023-07-21T22:13:23.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:13:23.487 INFO:journalctl@ceph.osd.9.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:13:24.027 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:13:24.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:24 smithi194 ceph-mon[98288]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:24.364 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.365 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.365 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.365 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.365 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-mon[96579]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:24.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:24.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:24 smithi134 ceph-mon[97873]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:24.649 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:13:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:13:24.320+0000 7f131a19b640 -1 osd.9 0 log_to_monitors true 2023-07-21T22:13:25.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:25 smithi093 ceph-mon[96579]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:13:25.377 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:25 smithi134 ceph-mon[97873]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:13:25.513 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:13:25.513 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:13:25.514 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-21T22:13:25.514 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-21T22:13:25.514 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110304 s, 951 MB/s 2023-07-21T22:13:25.514 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-21T22:13:25.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:25 smithi194 ceph-mon[98288]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:13:25.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2023-07-21T22:13:26.433 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:13:26.440 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:13:26.093+0000 7f130f012700 -1 osd.9 0 waiting for initial osdmap 2023-07-21T22:13:26.440 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:13:26.108+0000 7f1309604700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:13:26.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-mon[96579]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:13:26.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-mon[96579]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:13:26.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:26.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-mon[96579]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:13:26.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:26 smithi093 ceph-mon[96579]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:26 smithi194 ceph-mon[98288]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:13:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:26 smithi194 ceph-mon[98288]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:13:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:26 smithi194 ceph-mon[98288]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:13:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:26 smithi194 ceph-mon[98288]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:26.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:26 smithi134 ceph-mon[97873]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:13:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:26 smithi134 ceph-mon[97873]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:13:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:26 smithi134 ceph-mon[97873]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-07-21T22:13:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:26 smithi134 ceph-mon[97873]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:27.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:13:27.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:13:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi093 2023-07-21T22:13:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-21T22:13:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:27 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:13:27.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:13:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi093 2023-07-21T22:13:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-21T22:13:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:27 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi093 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 4292M 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:27 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:27.629 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 -- 172.21.15.134:0/3017047813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 msgr2=0x7f7d90106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:27.629 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3017047813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d90106e90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b0a0 tx=0x7f7d8800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.629 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 -- 172.21.15.134:0/3017047813 shutdown_connections 2023-07-21T22:13:27.629 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3017047813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d90106e90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3017047813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 0x7f7d900fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3017047813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 0x7f7d90100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 -- 172.21.15.134:0/3017047813 >> 172.21.15.134:0/3017047813 conn(0x7f7d900fa090 msgr2=0x7f7d900fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 -- 172.21.15.134:0/3017047813 shutdown_connections 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.585+0000 7f7d94f78700 1 -- 172.21.15.134:0/3017047813 wait complete. 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 Processor -- start 2023-07-21T22:13:27.630 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 -- start start 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 0x7f7d90195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 0x7f7d90195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019d610 con 0x7f7d900ffc40 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019d750 con 0x7f7d901005d0 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.586+0000 7f7d94f78700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9019d8e0 con 0x7f7d900fea40 2023-07-21T22:13:27.631 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:54562/0 (socket says 172.21.15.134:54562) 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8e59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 0x7f7d90195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 -- 172.21.15.134:0/3238462935 learned_addr learned my addr 172.21.15.134:0/3238462935 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8dd9b700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 0x7f7d90195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 -- 172.21.15.134:0/3238462935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 msgr2=0x7f7d90195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 0x7f7d90195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 -- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 msgr2=0x7f7d90195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:27.632 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 0x7f7d90195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:27.633 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 -- 172.21.15.134:0/3238462935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88009d00 con 0x7f7d901005d0 2023-07-21T22:13:27.633 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d8ed9d700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b070 tx=0x7f7d88004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:27.633 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800e040 con 0x7f7d901005d0 2023-07-21T22:13:27.634 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9019db60 con 0x7f7d901005d0 2023-07-21T22:13:27.634 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d8801bb20 con 0x7f7d901005d0 2023-07-21T22:13:27.634 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88005740 con 0x7f7d901005d0 2023-07-21T22:13:27.634 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.587+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9019e020 con 0x7f7d901005d0 2023-07-21T22:13:27.634 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.588+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9010b430 con 0x7f7d901005d0 2023-07-21T22:13:27.635 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.589+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d8801bc90 con 0x7f7d901005d0 2023-07-21T22:13:27.635 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.590+0000 7f7d877fe700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 0x7f7d7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:27.635 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.590+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f7d880ce4c0 con 0x7f7d901005d0 2023-07-21T22:13:27.635 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.590+0000 7f7d8e59c700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 0x7f7d7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:27.635 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.590+0000 7f7d8e59c700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 0x7f7d7c0a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ffaa0 tx=0x7f7d78009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:27.636 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.593+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7d880984a0 con 0x7f7d901005d0 2023-07-21T22:13:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:27.784+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d90060950 con 0x7f7d7c0a1150 2023-07-21T22:13:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:13:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:13:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] boot 2023-07-21T22:13:28.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: osdmap e64: 10 total, 10 up, 10 in 2023-07-21T22:13:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:28.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:28.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:13:28.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:13:28.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] boot 2023-07-21T22:13:28.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: osdmap e64: 10 total, 10 up, 10 in 2023-07-21T22:13:28.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:28.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:28.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:28.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:13:28.625 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: osd.9 [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] boot 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: osdmap e64: 10 total, 10 up, 10 in 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:28.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:28.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:29.205 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:29 smithi134 ceph-mon[97873]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:29.206 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:29 smithi134 ceph-mon[97873]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:13:29.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:29 smithi093 ceph-mon[96579]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:29.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:29 smithi093 ceph-mon[96579]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:13:29.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:29 smithi194 ceph-mon[98288]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:29.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:29 smithi194 ceph-mon[98288]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:13:30.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:30 smithi093 ceph-mon[96579]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:30.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:30 smithi194 ceph-mon[98288]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:30.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:30 smithi134 ceph-mon[97873]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:31.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/1479596872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]': finished 2023-07-21T22:13:31.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T22:13:31.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:31.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:31 smithi093 ceph-mon[96579]: from='client.? 172.21.15.134:0/4142325198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:31.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/1479596872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]': finished 2023-07-21T22:13:31.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T22:13:31.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:31.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:31 smithi194 ceph-mon[98288]: from='client.? 172.21.15.134:0/4142325198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:31.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/1479596872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]: dispatch 2023-07-21T22:13:31.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef648196-6eb5-4d15-bb0f-b6dedf1c03b7"}]': finished 2023-07-21T22:13:31.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: osdmap e66: 11 total, 10 up, 11 in 2023-07-21T22:13:31.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:31.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:31 smithi134 ceph-mon[97873]: from='client.? 172.21.15.134:0/4142325198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:32.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:32 smithi093 ceph-mon[96579]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:32.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:32 smithi194 ceph-mon[98288]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:32.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:32 smithi134 ceph-mon[97873]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:34.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:34 smithi194 ceph-mon[98288]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:34.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:34 smithi134 ceph-mon[97873]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:34.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:34 smithi093 ceph-mon[96579]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:36.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:36 smithi194 ceph-mon[98288]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:36 smithi134 ceph-mon[97873]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:36.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:36 smithi093 ceph-mon[96579]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:38.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:13:38.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:37 smithi194 ceph-mon[98288]: Deploying daemon osd.10 on smithi134 2023-07-21T22:13:38.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:37 smithi194 ceph-mon[98288]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:38.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:13:38.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:37 smithi134 ceph-mon[97873]: Deploying daemon osd.10 on smithi134 2023-07-21T22:13:38.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:37 smithi134 ceph-mon[97873]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:38.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:13:38.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:37 smithi093 ceph-mon[96579]: Deploying daemon osd.10 on smithi134 2023-07-21T22:13:38.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:37 smithi093 ceph-mon[96579]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:40.555 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:40 smithi134 ceph-mon[97873]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:40.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:40 smithi194 ceph-mon[98288]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:40.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:40 smithi093 ceph-mon[96579]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:42.627 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:42 smithi134 ceph-mon[97873]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:42.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:42.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:42 smithi194 ceph-mon[98288]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:42.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:42.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:42 smithi093 ceph-mon[96579]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:42.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:42.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:44.286 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 10 on host 'smithi134' 2023-07-21T22:13:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.243+0000 7f7d877fe700 1 -- 172.21.15.134:0/3238462935 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7d90060950 con 0x7f7d7c0a1150 2023-07-21T22:13:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 msgr2=0x7f7d7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 0x7f7d7c0a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ffaa0 tx=0x7f7d78009190 comp rx=0 tx=0).stop 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 msgr2=0x7f7d9019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b070 tx=0x7f7d88004d00 comp rx=0 tx=0).stop 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 shutdown_connections 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f7d7c0a1150 0x7f7d7c0a3600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d901005d0 0x7f7d9019d040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d900fea40 0x7f7d90195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 --2- 172.21.15.134:0/3238462935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d900ffc40 0x7f7d90195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 >> 172.21.15.134:0/3238462935 conn(0x7f7d900fa090 msgr2=0x7f7d900fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.245+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 shutdown_connections 2023-07-21T22:13:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-21T22:13:44.246+0000 7f7d94f78700 1 -- 172.21.15.134:0/3238462935 wait complete. 2023-07-21T22:13:44.730 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:44 smithi134 ceph-mon[97873]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:44.730 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.730 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.730 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.731 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:44 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.732 DEBUG:teuthology.orchestra.run.smithi134:osd.10> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.10.service 2023-07-21T22:13:44.734 INFO:tasks.cephadm:Deploying osd.11 on smithi194 with /dev/vg_nvme/lv_1... 2023-07-21T22:13:44.734 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:13:44.769 INFO:journalctl@ceph.osd.10.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:13:44.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:44 smithi194 ceph-mon[98288]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:44.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:44 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:44 smithi093 ceph-mon[96579]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:44.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:44.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:44 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:45.299 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:13:45.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:45 smithi134 ceph-mon[97873]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:45.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:45 smithi134 ceph-mon[97873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:45.629 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:13:45 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:13:45.402+0000 7f2ef90f9640 -1 osd.10 0 log_to_monitors true 2023-07-21T22:13:45.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:45 smithi194 ceph-mon[98288]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:45.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:45 smithi194 ceph-mon[98288]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:45.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:45 smithi093 ceph-mon[96579]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:45.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:45 smithi093 ceph-mon[96579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:13:46.792 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:13:46.793 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:13:46.793 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2023-07-21T22:13:46.793 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2023-07-21T22:13:46.793 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111281 s, 942 MB/s 2023-07-21T22:13:46.793 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2023-07-21T22:13:46.825 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:46.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:13:46.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:13:46.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:46.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:46.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:46 smithi194 ceph-mon[98288]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:46.827 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:13:46.524+0000 7f2eedf70700 -1 osd.10 0 waiting for initial osdmap 2023-07-21T22:13:46.827 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:13:46.538+0000 7f2ee9564700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:13:46.827 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:46.827 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:13:46.828 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:13:46.828 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:46.828 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:46.828 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:46 smithi134 ceph-mon[97873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:46.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:46.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:13:46.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:13:46.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:46.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: from='osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:46.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:46 smithi093 ceph-mon[96579]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-21T22:13:47.152 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2023-07-21T22:13:47.710 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:13:47.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:13:47.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:13:47.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] boot 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:47.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:13:47.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] boot 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: osd.10 [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] boot 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: osdmap e69: 11 total, 11 up, 11 in 2023-07-21T22:13:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:47.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi134 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:48.830 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:48 smithi194 ceph-mon[98288]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:13:48.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:13:48.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:13:48.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi134 2023-07-21T22:13:48.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-21T22:13:48.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:48.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:48 smithi134 ceph-mon[97873]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:13:48.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:13:48.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:13:48.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi134 2023-07-21T22:13:48.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-21T22:13:48.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:13:48.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:48 smithi093 ceph-mon[96579]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:13:49.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 -- 172.21.15.194:0/3906678517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac100610 msgr2=0x7fb9ac106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:49.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 --2- 172.21.15.194:0/3906678517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac100610 0x7fb9ac106ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b0a0 tx=0x7fb9a400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 -- 172.21.15.194:0/3906678517 shutdown_connections 2023-07-21T22:13:49.011 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 --2- 172.21.15.194:0/3906678517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac100610 0x7fb9ac106ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 --2- 172.21.15.194:0/3906678517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 --2- 172.21.15.194:0/3906678517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 -- 172.21.15.194:0/3906678517 >> 172.21.15.194:0/3906678517 conn(0x7fb9ac0fa0b0 msgr2=0x7fb9ac0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 -- 172.21.15.194:0/3906678517 shutdown_connections 2023-07-21T22:13:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.968+0000 7fb9b1002700 1 -- 172.21.15.194:0/3906678517 wait complete. 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 Processor -- start 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 -- start start 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 0x7fb9ac19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:49.014 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac19d500 con 0x7fb9ac0ffc80 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac19d670 con 0x7fb9ac100610 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9b1002700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac19d830 con 0x7fb9ac0fea80 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47382/0 (socket says 172.21.15.194:47382) 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 -- 172.21.15.194:0/2923280276 learned_addr learned my addr 172.21.15.194:0/2923280276 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aa59c700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9ab59e700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 0x7fb9ac19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 -- 172.21.15.194:0/2923280276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 msgr2=0x7fb9ac19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 0x7fb9ac19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 -- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 msgr2=0x7fb9ac19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aad9d700 1 -- 172.21.15.194:0/2923280276 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009d00 con 0x7fb9ac0fea80 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9ab59e700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 0x7fb9ac19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.969+0000 7fb9aa59c700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb9aad9d700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c970 tx=0x7fb99c00cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004220 con 0x7fb9ac0fea80 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac19db10 con 0x7fb9ac0fea80 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99c007d60 con 0x7fb9ac0fea80 2023-07-21T22:13:49.017 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c003d00 con 0x7fb9ac0fea80 2023-07-21T22:13:49.018 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.970+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac19e060 con 0x7fb9ac0fea80 2023-07-21T22:13:49.018 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.971+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac106030 con 0x7fb9ac0fea80 2023-07-21T22:13:49.018 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.972+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb99c0078a0 con 0x7fb9ac0fea80 2023-07-21T22:13:49.019 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.972+0000 7fb993fff700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 0x7fb9940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:49.019 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.972+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fb99c0cc260 con 0x7fb9ac0fea80 2023-07-21T22:13:49.019 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.972+0000 7fb9aa59c700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 0x7fb9940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:49.019 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.973+0000 7fb9aa59c700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 0x7fb9940a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac069940 tx=0x7fb9a000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:49.019 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:48.976+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb99c095fe0 con 0x7fb9ac0fea80 2023-07-21T22:13:49.217 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:13:49.174+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9ac060950 con 0x7fb9940a1150 2023-07-21T22:13:50.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:13:50.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:50.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:50.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:50.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:50.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:49 smithi194 ceph-mon[98288]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:13:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:50.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:49 smithi134 ceph-mon[97873]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:50.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:13:50.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:50.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:50.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:50.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:50.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:49 smithi093 ceph-mon[96579]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:51.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:50 smithi194 ceph-mon[98288]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:13:51.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:50 smithi134 ceph-mon[97873]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:13:51.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:50 smithi093 ceph-mon[96579]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:13:51.440 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:13:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:13:51.161+0000 7fed20f3b700 -1 calc_pg_upmaps abort due to max <= 0 2023-07-21T22:13:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:13:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:13:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/4046668999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]': finished 2023-07-21T22:13:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:13:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:13:52.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:13:52.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:13:52.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/4046668999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]': finished 2023-07-21T22:13:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:13:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:13:52.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:13:52.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:13:52.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/4046668999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]: dispatch 2023-07-21T22:13:52.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68ec4cd3-029b-45b3-af2a-16ef9ea3be47"}]': finished 2023-07-21T22:13:52.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:13:52.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:13:53.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:53 smithi194 ceph-mon[98288]: from='client.? 172.21.15.194:0/1614687412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:53.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:53 smithi134 ceph-mon[97873]: from='client.? 172.21.15.194:0/1614687412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:53.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:53 smithi093 ceph-mon[96579]: from='client.? 172.21.15.194:0/1614687412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:54.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:54 smithi194 ceph-mon[98288]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-21T22:13:54.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:54 smithi134 ceph-mon[97873]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-21T22:13:54.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:54 smithi093 ceph-mon[96579]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-21T22:13:56.585 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:56 smithi194 ceph-mon[98288]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:56.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:56 smithi134 ceph-mon[97873]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:56.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:56 smithi093 ceph-mon[96579]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:13:58.306 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:57 smithi194 ceph-mon[98288]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:58.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:57 smithi134 ceph-mon[97873]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:58.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:57 smithi093 ceph-mon[96579]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:13:59.238 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:58 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:13:59.238 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:58 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:59.239 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:58 smithi194 ceph-mon[98288]: Deploying daemon osd.11 on smithi194 2023-07-21T22:13:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:58 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:13:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:58 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:58 smithi134 ceph-mon[97873]: Deploying daemon osd.11 on smithi194 2023-07-21T22:13:59.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:58 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:13:59.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:58 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:59.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:58 smithi093 ceph-mon[96579]: Deploying daemon osd.11 on smithi194 2023-07-21T22:14:00.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:13:59 smithi194 ceph-mon[98288]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:00.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:13:59 smithi134 ceph-mon[97873]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:00.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:13:59 smithi093 ceph-mon[96579]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:02.579 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:02 smithi194 ceph-mon[98288]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:02.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:02 smithi134 ceph-mon[97873]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:02.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:02 smithi093 ceph-mon[96579]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:04.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:03 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:03 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:03 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:04.081 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:03 smithi194 ceph-mon[98288]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:04.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:03 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:03 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:03 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:04.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:03 smithi134 ceph-mon[97873]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:04.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:03 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:03 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:04.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:03 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:04.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:03 smithi093 ceph-mon[96579]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:05.676 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 11 on host 'smithi194' 2023-07-21T22:14:05.676 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.632+0000 7fb993fff700 1 -- 172.21.15.194:0/2923280276 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb9ac060950 con 0x7fb9940a1150 2023-07-21T22:14:05.678 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.634+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 msgr2=0x7fb9940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:05.678 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.634+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 0x7fb9940a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac069940 tx=0x7fb9a000b2c0 comp rx=0 tx=0).stop 2023-07-21T22:14:05.678 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.634+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 msgr2=0x7fb9ac0693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:05.679 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.634+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c970 tx=0x7fb99c00cd30 comp rx=0 tx=0).stop 2023-07-21T22:14:05.679 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 shutdown_connections 2023-07-21T22:14:05.679 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb9940a1150 0x7fb9940a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:05.680 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac100610 0x7fb9ac19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:05.680 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ac0fea80 0x7fb9ac0693c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:05.680 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 --2- 172.21.15.194:0/2923280276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9ac0ffc80 0x7fb9ac19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:05.680 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 >> 172.21.15.194:0/2923280276 conn(0x7fb9ac0fa0b0 msgr2=0x7fb9ac103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:05.681 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 shutdown_connections 2023-07-21T22:14:05.681 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:05.635+0000 7fb9b1002700 1 -- 172.21.15.194:0/2923280276 wait complete. 2023-07-21T22:14:06.100 DEBUG:teuthology.orchestra.run.smithi194:osd.11> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.11.service 2023-07-21T22:14:06.102 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-21T22:14:06.102 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:14:06.131 INFO:journalctl@ceph.osd.11.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:14:06.670 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:06.685 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.686 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.687 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:06.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:06.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:06 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:06.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:06.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:06 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:07.054 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:14:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:14:06.688+0000 7f5cf41bb640 -1 osd.11 0 log_to_monitors true 2023-07-21T22:14:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:07 smithi093 ceph-mon[96579]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:07.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:07 smithi093 ceph-mon[96579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:07.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.772+0000 7feb382b1700 1 -- 172.21.15.93:0/21838927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb301017c0 msgr2=0x7feb30108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:07.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.772+0000 7feb382b1700 1 --2- 172.21.15.93:0/21838927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb301017c0 0x7feb30108080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb2800b0a0 tx=0x7feb2800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 -- 172.21.15.93:0/21838927 shutdown_connections 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 --2- 172.21.15.93:0/21838927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb301017c0 0x7feb30108080 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 --2- 172.21.15.93:0/21838927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300fde50 0x7feb300fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 --2- 172.21.15.93:0/21838927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb30100e30 0x7feb30101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 -- 172.21.15.93:0/21838927 >> 172.21.15.93:0/21838927 conn(0x7feb300f9860 msgr2=0x7feb300fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 -- 172.21.15.93:0/21838927 shutdown_connections 2023-07-21T22:14:07.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 -- 172.21.15.93:0/21838927 wait complete. 2023-07-21T22:14:07.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 Processor -- start 2023-07-21T22:14:07.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.773+0000 7feb382b1700 1 -- start start 2023-07-21T22:14:07.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 0x7feb30195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:07.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:07.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 0x7feb3019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:07.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3019d590 con 0x7feb30100e30 2023-07-21T22:14:07.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3019d6d0 con 0x7feb300fde50 2023-07-21T22:14:07.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3019d860 con 0x7feb301017c0 2023-07-21T22:14:07.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3584c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:07.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3584c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41694/0 (socket says 172.21.15.93:41694) 2023-07-21T22:14:07.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 0x7feb3019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:07.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3584c700 1 -- 172.21.15.93:0/2452426008 learned_addr learned my addr 172.21.15.93:0/2452426008 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:07.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3604d700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 0x7feb30195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:07.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 -- 172.21.15.93:0/2452426008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 msgr2=0x7feb30195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:07.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 0x7feb30195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 -- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 msgr2=0x7feb30195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:07.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:07.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 -- 172.21.15.93:0/2452426008 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28009d00 con 0x7feb301017c0 2023-07-21T22:14:07.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3604d700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 0x7feb30195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:07.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3584c700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:14:07.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb3684e700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 0x7feb3019cfc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb2800b0a0 tx=0x7feb2800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:07.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800e040 con 0x7feb301017c0 2023-07-21T22:14:07.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3019dae0 con 0x7feb301017c0 2023-07-21T22:14:07.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.774+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb28004c50 con 0x7feb301017c0 2023-07-21T22:14:07.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.775+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3019dfd0 con 0x7feb301017c0 2023-07-21T22:14:07.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.775+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280052a0 con 0x7feb301017c0 2023-07-21T22:14:07.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.775+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb30107190 con 0x7feb301017c0 2023-07-21T22:14:07.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.777+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb2801d470 con 0x7feb301017c0 2023-07-21T22:14:07.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.777+0000 7feb277fe700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 0x7feb1c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:07.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.777+0000 7feb3604d700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 0x7feb1c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:07.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.777+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7feb280cd930 con 0x7feb301017c0 2023-07-21T22:14:07.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.778+0000 7feb3604d700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 0x7feb1c0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb30067dd0 tx=0x7feb2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:07.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.781+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feb280976e0 con 0x7feb301017c0 2023-07-21T22:14:07.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:07 smithi194 ceph-mon[98288]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:07.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:07 smithi194 ceph-mon[98288]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:07.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:07 smithi134 ceph-mon[97873]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:07.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:07 smithi134 ceph-mon[97873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:14:08.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.970+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7feb3002caa0 con 0x7feb301017c0 2023-07-21T22:14:08.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.970+0000 7feb277fe700 1 -- 172.21.15.93:0/2452426008 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7feb2809ad00 con 0x7feb301017c0 2023-07-21T22:14:08.015 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:08.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 msgr2=0x7feb1c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:08.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 0x7feb1c0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb30067dd0 tx=0x7feb2c009260 comp rx=0 tx=0).stop 2023-07-21T22:14:08.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 msgr2=0x7feb3019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:08.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 0x7feb3019cfc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb2800b0a0 tx=0x7feb2800bab0 comp rx=0 tx=0).stop 2023-07-21T22:14:08.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 shutdown_connections 2023-07-21T22:14:08.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb301017c0 0x7feb3019cfc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:08.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb1c0a1150 0x7feb1c0a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:08.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb300fde50 0x7feb30195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:08.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 --2- 172.21.15.93:0/2452426008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb30100e30 0x7feb30195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:08.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.972+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 >> 172.21.15.93:0/2452426008 conn(0x7feb300f9860 msgr2=0x7feb301042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:08.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.973+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 shutdown_connections 2023-07-21T22:14:08.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:07.973+0000 7feb382b1700 1 -- 172.21.15.93:0/2452426008 wait complete. 2023-07-21T22:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1689977626,"num_in_osds":12,"osd_in_since":1689977631,"num_remapped_pgs":0} 2023-07-21T22:14:08.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:08.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:14:08.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:14:08.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:08.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:08 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2452426008' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:08.836 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:14:08.364+0000 7f5ce9032700 -1 osd.11 0 waiting for initial osdmap 2023-07-21T22:14:08.836 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:14:08.379+0000 7f5ce2e23700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:14:08.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:08 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2452426008' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:08.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:08.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:14:08.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:14:08.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:08.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: from='osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2023-07-21T22:14:08.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:08 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2452426008' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:09.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:14:09.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:14:09.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:14:09.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: Detected new or changed devices on smithi194 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 4292M 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:09 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:14:09.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:14:09.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: Detected new or changed devices on smithi194 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 4292M 2023-07-21T22:14:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:09 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2023-07-21T22:14:09.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:14:09.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: Detected new or changed devices on smithi194 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 4292M 2023-07-21T22:14:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:09 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:09.985 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:10.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: purged_snaps scrub starts 2023-07-21T22:14:10.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: purged_snaps scrub ok 2023-07-21T22:14:10.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:10.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:10.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] boot 2023-07-21T22:14:10.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:14:10.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:10 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: purged_snaps scrub starts 2023-07-21T22:14:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: purged_snaps scrub ok 2023-07-21T22:14:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] boot 2023-07-21T22:14:10.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:14:10.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:10 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: purged_snaps scrub starts 2023-07-21T22:14:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: purged_snaps scrub ok 2023-07-21T22:14:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:14:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: osd.11 [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] boot 2023-07-21T22:14:10.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:14:10.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:10 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:14:11.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 -- 172.21.15.93:0/3375809174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c0ffc80 msgr2=0x7f1b4c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 --2- 172.21.15.93:0/3375809174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c1000d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009990 tx=0x7f1b34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 -- 172.21.15.93:0/3375809174 shutdown_connections 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 --2- 172.21.15.93:0/3375809174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c100610 0x7f1b4c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 --2- 172.21.15.93:0/3375809174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 --2- 172.21.15.93:0/3375809174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c1000d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 -- 172.21.15.93:0/3375809174 >> 172.21.15.93:0/3375809174 conn(0x7f1b4c0fa0b0 msgr2=0x7f1b4c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 -- 172.21.15.93:0/3375809174 shutdown_connections 2023-07-21T22:14:11.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 -- 172.21.15.93:0/3375809174 wait complete. 2023-07-21T22:14:11.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.155+0000 7f1b51b38700 1 Processor -- start 2023-07-21T22:14:11.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 -- start start 2023-07-21T22:14:11.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19c3d0 con 0x7f1b4c100610 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19c510 con 0x7f1b4c0fea80 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b51b38700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c19c6a0 con 0x7f1b4c0ffc80 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41708/0 (socket says 172.21.15.93:41708) 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 -- 172.21.15.93:0/596056523 learned_addr learned my addr 172.21.15.93:0/596056523 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4affd700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4b7fe700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 -- 172.21.15.93:0/596056523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c0ffc80 msgr2=0x7f1b4c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 -- 172.21.15.93:0/596056523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 msgr2=0x7f1b4c194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 -- 172.21.15.93:0/596056523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340095f0 con 0x7f1b4c100610 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4b7fe700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.156+0000 7f1b4bfff700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007e60 tx=0x7f1b4400e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.157+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44011070 con 0x7f1b4c100610 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.157+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b44007b30 con 0x7f1b4c100610 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.157+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c068030 con 0x7f1b4c100610 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.157+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440153f0 con 0x7f1b4c100610 2023-07-21T22:14:11.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.157+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c0684e0 con 0x7f1b4c100610 2023-07-21T22:14:11.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.158+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c10a2e0 con 0x7f1b4c100610 2023-07-21T22:14:11.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.159+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1b4400ea00 con 0x7f1b4c100610 2023-07-21T22:14:11.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.159+0000 7f1b48ff9700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 0x7f1b380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.159+0000 7f1b4b7fe700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 0x7f1b380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.159+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1b440ce5d0 con 0x7f1b4c100610 2023-07-21T22:14:11.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.160+0000 7f1b4b7fe700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 0x7f1b380a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0ffae0 tx=0x7f1b3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:11.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.163+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1b440981c0 con 0x7f1b4c100610 2023-07-21T22:14:11.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.350+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1b4c10a4a0 con 0x7f1b4c100610 2023-07-21T22:14:11.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.350+0000 7f1b48ff9700 1 -- 172.21.15.93:0/596056523 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1b4400ecb0 con 0x7f1b4c100610 2023-07-21T22:14:11.394 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:11.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 msgr2=0x7f1b380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 0x7f1b380a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0ffae0 tx=0x7f1b3c008040 comp rx=0 tx=0).stop 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 msgr2=0x7f1b4c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007e60 tx=0x7f1b4400e330 comp rx=0 tx=0).stop 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 shutdown_connections 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b4c100610 0x7f1b4c19be00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b380a1150 0x7f1b380a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b4c0fea80 0x7f1b4c194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 --2- 172.21.15.93:0/596056523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b4c0ffc80 0x7f1b4c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.352+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 >> 172.21.15.93:0/596056523 conn(0x7f1b4c0fa0b0 msgr2=0x7f1b4c0fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:11.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.353+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 shutdown_connections 2023-07-21T22:14:11.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:11.353+0000 7f1b51b38700 1 -- 172.21.15.93:0/596056523 wait complete. 2023-07-21T22:14:11.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:11 smithi093 ceph-mon[96579]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:14:11.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:11 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/596056523' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:11.728 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1689977649,"num_in_osds":12,"osd_in_since":1689977631,"num_remapped_pgs":0} 2023-07-21T22:14:11.728 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:14:11.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:11 smithi134 ceph-mon[97873]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:14:11.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:11 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/596056523' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:12.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:11 smithi194 ceph-mon[98288]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:14:12.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:11 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/596056523' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:14:12.278 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:12.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:12 smithi093 ceph-mon[96579]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:12.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:12 smithi134 ceph-mon[97873]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:13.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:12 smithi194 ceph-mon[98288]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:13.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 -- 172.21.15.93:0/3172509814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 msgr2=0x7fc38810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:13.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 --2- 172.21.15.93:0/3172509814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38810b8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc38000b0a0 tx=0x7fc38000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 -- 172.21.15.93:0/3172509814 shutdown_connections 2023-07-21T22:14:13.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 --2- 172.21.15.93:0/3172509814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38810b8f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 --2- 172.21.15.93:0/3172509814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc388104ab0 0x7fc388104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 --2- 172.21.15.93:0/3172509814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3880feab0 0x7fc3880fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 -- 172.21.15.93:0/3172509814 >> 172.21.15.93:0/3172509814 conn(0x7fc388075b70 msgr2=0x7fc388075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:13.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 -- 172.21.15.93:0/3172509814 shutdown_connections 2023-07-21T22:14:13.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.417+0000 7fc38f378700 1 -- 172.21.15.93:0/3172509814 wait complete. 2023-07-21T22:14:13.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 Processor -- start 2023-07-21T22:14:13.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 -- start start 2023-07-21T22:14:13.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3880feab0 0x7fc38819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:13.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:13.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc388104ab0 0x7fc388194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:13.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc388195080 con 0x7fc3880ff460 2023-07-21T22:14:13.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3881951f0 con 0x7fc3880feab0 2023-07-21T22:14:13.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38f378700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3881953b0 con 0x7fc388104ab0 2023-07-21T22:14:13.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:13.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41732/0 (socket says 172.21.15.93:41732) 2023-07-21T22:14:13.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 -- 172.21.15.93:0/230019275 learned_addr learned my addr 172.21.15.93:0/230019275 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:13.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38d114700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3880feab0 0x7fc38819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:13.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38d915700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc388104ab0 0x7fc388194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:13.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 -- 172.21.15.93:0/230019275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc388104ab0 msgr2=0x7fc388194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:13.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc388104ab0 0x7fc388194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 -- 172.21.15.93:0/230019275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3880feab0 msgr2=0x7fc38819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:13.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3880feab0 0x7fc38819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 -- 172.21.15.93:0/230019275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380009d00 con 0x7fc3880ff460 2023-07-21T22:14:13.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.418+0000 7fc38c913700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c920 tx=0x7fc38400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:13.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384004a30 con 0x7fc3880ff460 2023-07-21T22:14:13.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3840054e0 con 0x7fc3880ff460 2023-07-21T22:14:13.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc388195690 con 0x7fc3880ff460 2023-07-21T22:14:13.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384007bd0 con 0x7fc3880ff460 2023-07-21T22:14:13.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880681a0 con 0x7fc3880ff460 2023-07-21T22:14:13.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.419+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38804e210 con 0x7fc3880ff460 2023-07-21T22:14:13.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.421+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc384007d50 con 0x7fc3880ff460 2023-07-21T22:14:13.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.421+0000 7fc37e7fc700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 0x7fc3740a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:13.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.421+0000 7fc38d114700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 0x7fc3740a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:13.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.421+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3840cc360 con 0x7fc3880ff460 2023-07-21T22:14:13.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.422+0000 7fc38d114700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 0x7fc3740a5890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ffbf0 tx=0x7fc37800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:13.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.425+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc384095fd0 con 0x7fc3880ff460 2023-07-21T22:14:13.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.610+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc388068520 con 0x7fc3880ff460 2023-07-21T22:14:13.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.611+0000 7fc37e7fc700 1 -- 172.21.15.93:0/230019275 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18387 (secure 0 0 0) 0x7fc38401d090 con 0x7fc3880ff460 2023-07-21T22:14:13.655 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:13.655 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":78,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:10.572001+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:13.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.613+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 msgr2=0x7fc3740a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.613+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 0x7fc3740a5890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ffbf0 tx=0x7fc37800b410 comp rx=0 tx=0).stop 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.613+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 msgr2=0x7fc38819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.613+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc38400c920 tx=0x7fc38400cc30 comp rx=0 tx=0).stop 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 shutdown_connections 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc388104ab0 0x7fc388194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc3740a33e0 0x7fc3740a5890 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3880feab0 0x7fc38819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 --2- 172.21.15.93:0/230019275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3880ff460 0x7fc38819aed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 >> 172.21.15.93:0/230019275 conn(0x7fc388075b70 msgr2=0x7fc388105930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 shutdown_connections 2023-07-21T22:14:13.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:13.614+0000 7fc38f378700 1 -- 172.21.15.93:0/230019275 wait complete. 2023-07-21T22:14:14.135 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:14:14.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-21T22:14:14.688 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:14.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:14 smithi134 ceph-mon[97873]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:14.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:14 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/230019275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:14.890 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:14 smithi093 ceph-mon[96579]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:14.891 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:14 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/230019275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:15.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:14 smithi194 ceph-mon[98288]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:15.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:14 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/230019275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:15.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 -- 172.21.15.93:0/1091543496 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 msgr2=0x7ff13010b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:15.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 --2- 172.21.15.93:0/1091543496 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff13010b8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff12800b0d0 tx=0x7ff12800b3e0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 -- 172.21.15.93:0/1091543496 shutdown_connections 2023-07-21T22:14:15.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 --2- 172.21.15.93:0/1091543496 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff13010b8b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 --2- 172.21.15.93:0/1091543496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff130103a30 0x7ff130103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 --2- 172.21.15.93:0/1091543496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1300ff270 0x7ff1300ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 -- 172.21.15.93:0/1091543496 >> 172.21.15.93:0/1091543496 conn(0x7ff130075150 msgr2=0x7ff130075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:15.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 -- 172.21.15.93:0/1091543496 shutdown_connections 2023-07-21T22:14:15.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.823+0000 7ff137f83700 1 -- 172.21.15.93:0/1091543496 wait complete. 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 Processor -- start 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 -- start start 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1300ff270 0x7ff130195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff130103a30 0x7ff13019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:15.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019acf0 con 0x7ff1300ffc20 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019ae60 con 0x7ff130103a30 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff137f83700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019b020 con 0x7ff1300ff270 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41746/0 (socket says 172.21.15.93:41746) 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 -- 172.21.15.93:0/3545266014 learned_addr learned my addr 172.21.15.93:0/3545266014 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff135d1f700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1300ff270 0x7ff130195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:15.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff136520700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff130103a30 0x7ff13019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:15.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 -- 172.21.15.93:0/3545266014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1300ff270 msgr2=0x7ff130195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:15.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1300ff270 0x7ff130195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 -- 172.21.15.93:0/3545266014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff130103a30 msgr2=0x7ff13019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:15.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff130103a30 0x7ff13019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:15.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 -- 172.21.15.93:0/3545266014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009d00 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.824+0000 7ff13551e700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c920 tx=0x7ff12000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.825+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120007b00 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.825+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13019b300 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.825+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12000ce70 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.825+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120005460 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.825+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1301a2b60 con 0x7ff1300ffc20 2023-07-21T22:14:15.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.826+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff114005320 con 0x7ff1300ffc20 2023-07-21T22:14:15.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.827+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff120004750 con 0x7ff1300ffc20 2023-07-21T22:14:15.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.827+0000 7ff126ffd700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:15.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.827+0000 7ff135d1f700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:15.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.827+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff1200cc370 con 0x7ff1300ffc20 2023-07-21T22:14:15.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.828+0000 7ff135d1f700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 0x7ff11c0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff12c005950 tx=0x7ff12c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:15.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:15.831+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff120095f60 con 0x7ff1300ffc20 2023-07-21T22:14:16.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.018+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff114005f70 con 0x7ff1300ffc20 2023-07-21T22:14:16.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.018+0000 7ff126ffd700 1 -- 172.21.15.93:0/3545266014 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff12001d0a0 con 0x7ff1300ffc20 2023-07-21T22:14:16.062 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:14:16.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 msgr2=0x7ff11c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:16.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 0x7ff11c0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff12c005950 tx=0x7ff12c00b410 comp rx=0 tx=0).stop 2023-07-21T22:14:16.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 msgr2=0x7ff130196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:16.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c920 tx=0x7ff12000cc30 comp rx=0 tx=0).stop 2023-07-21T22:14:16.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 shutdown_connections 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff130103a30 0x7ff13019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1300ff270 0x7ff130195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 --2- 172.21.15.93:0/3545266014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1300ffc20 0x7ff130196270 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 >> 172.21.15.93:0/3545266014 conn(0x7ff130075150 msgr2=0x7ff130102890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 shutdown_connections 2023-07-21T22:14:16.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:16.021+0000 7ff137f83700 1 -- 172.21.15.93:0/3545266014 wait complete. 2023-07-21T22:14:16.430 INFO:tasks.cephadm:Adding mds.a on smithi093 2023-07-21T22:14:16.431 INFO:tasks.cephadm:Adding mds.d on smithi093 2023-07-21T22:14:16.432 INFO:tasks.cephadm:Adding mds.g on smithi093 2023-07-21T22:14:16.432 INFO:tasks.cephadm:Adding mds.j on smithi093 2023-07-21T22:14:16.432 INFO:tasks.cephadm:Adding mds.b on smithi134 2023-07-21T22:14:16.433 INFO:tasks.cephadm:Adding mds.e on smithi134 2023-07-21T22:14:16.433 INFO:tasks.cephadm:Adding mds.h on smithi134 2023-07-21T22:14:16.433 INFO:tasks.cephadm:Adding mds.k on smithi134 2023-07-21T22:14:16.433 INFO:tasks.cephadm:Adding mds.c on smithi194 2023-07-21T22:14:16.434 INFO:tasks.cephadm:Adding mds.f on smithi194 2023-07-21T22:14:16.434 INFO:tasks.cephadm:Adding mds.i on smithi194 2023-07-21T22:14:16.434 INFO:tasks.cephadm:Adding mds.l on smithi194 2023-07-21T22:14:16.434 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l' 2023-07-21T22:14:16.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:16 smithi093 ceph-mon[96579]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:16.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:16 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3545266014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:16.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:16 smithi194 ceph-mon[98288]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:16.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:16 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3545266014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:16.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:16 smithi134 ceph-mon[97873]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:16.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:16 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3545266014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:17.004 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:14:18.235 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:17 smithi194 ceph-mon[98288]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:18.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:17 smithi134 ceph-mon[97873]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:18.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:17 smithi093 ceph-mon[96579]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:18.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3983600189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480ffc80 msgr2=0x7ff1481000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:18.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3983600189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480ffc80 0x7ff1481000d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff138009990 tx=0x7ff138009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.451 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3983600189 shutdown_connections 2023-07-21T22:14:18.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3983600189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff148106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3983600189 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480fea80 0x7ff1480fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.452 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3983600189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480ffc80 0x7ff1481000d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.453 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3983600189 >> 172.21.15.194:0/3983600189 conn(0x7ff1480fa0b0 msgr2=0x7ff1480fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:18.453 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3983600189 shutdown_connections 2023-07-21T22:14:18.453 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.407+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3983600189 wait complete. 2023-07-21T22:14:18.455 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 Processor -- start 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 -- start start 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480ffc80 0x7ff148194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff14819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14819c2f0 con 0x7ff1480ffc80 2023-07-21T22:14:18.456 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14819c430 con 0x7ff148100610 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14c8d6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14819c5c0 con 0x7ff1480fea80 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47508/0 (socket says 172.21.15.194:47508) 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.408+0000 7ff14659c700 1 -- 172.21.15.194:0/3557557097 learned_addr learned my addr 172.21.15.194:0/3557557097 (peer_addr_for_me v2:172.21.15.194:0/0) 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff145d9b700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480ffc80 0x7ff148194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff146d9d700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff14819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:18.457 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 -- 172.21.15.194:0/3557557097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 msgr2=0x7ff14819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff14819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 -- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480ffc80 msgr2=0x7ff148194ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480ffc80 0x7ff148194ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 -- 172.21.15.194:0/3557557097 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1380095f0 con 0x7ff1480fea80 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff146d9d700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff14819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14659c700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c9a0 tx=0x7ff13c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:18.458 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c004220 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff13c007d20 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14819c8a0 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c003c00 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.409+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14819cda0 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.410+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff148106080 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.411+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff13c007860 con 0x7ff1480fea80 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.412+0000 7ff1377fe700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 0x7ff1300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:18.459 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.412+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff13c0cc0c0 con 0x7ff1480fea80 2023-07-21T22:14:18.460 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.412+0000 7ff145d9b700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 0x7ff1300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:18.460 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.412+0000 7ff145d9b700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 0x7ff1300a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff13800b5c0 tx=0x7ff138005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:18.460 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.416+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff13c095c10 con 0x7ff1480fea80 2023-07-21T22:14:18.654 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.610+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}) v1 -- 0x7ff148060b50 con 0x7ff1300a1150 2023-07-21T22:14:18.662 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.618+0000 7ff1377fe700 1 -- 172.21.15.194:0/3557557097 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff148060b50 con 0x7ff1300a1150 2023-07-21T22:14:18.662 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mds.all update... 2023-07-21T22:14:18.663 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 msgr2=0x7ff1300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 0x7ff1300a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff13800b5c0 tx=0x7ff138005960 comp rx=0 tx=0).stop 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 msgr2=0x7ff148194760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c9a0 tx=0x7ff13c00ccb0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 shutdown_connections 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1300a1150 0x7ff1300a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.664 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff148100610 0x7ff14819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.665 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1480fea80 0x7ff148194760 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.665 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 --2- 172.21.15.194:0/3557557097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1480ffc80 0x7ff148194ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:18.665 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 >> 172.21.15.194:0/3557557097 conn(0x7ff1480fa0b0 msgr2=0x7ff1481031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:18.665 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 shutdown_connections 2023-07-21T22:14:18.665 INFO:teuthology.orchestra.run.smithi194.stderr:2023-07-21T22:14:18.620+0000 7ff14c8d6700 1 -- 172.21.15.194:0/3557557097 wait complete. 2023-07-21T22:14:19.149 DEBUG:teuthology.orchestra.run.smithi093:mds.a> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.a.service 2023-07-21T22:14:19.152 DEBUG:teuthology.orchestra.run.smithi093:mds.d> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.d.service 2023-07-21T22:14:19.189 INFO:journalctl@ceph.mds.a.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:14:19.190 DEBUG:teuthology.orchestra.run.smithi093:mds.g> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.g.service 2023-07-21T22:14:19.220 INFO:journalctl@ceph.mds.d.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:14:19.221 DEBUG:teuthology.orchestra.run.smithi093:mds.j> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.j.service 2023-07-21T22:14:19.249 INFO:journalctl@ceph.mds.g.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:14:19.250 DEBUG:teuthology.orchestra.run.smithi134:mds.b> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.b.service 2023-07-21T22:14:19.253 DEBUG:teuthology.orchestra.run.smithi134:mds.e> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.e.service 2023-07-21T22:14:19.276 INFO:journalctl@ceph.mds.j.smithi093.stdout:-- Logs begin at Fri 2023-07-21 21:46:54 UTC. -- 2023-07-21T22:14:19.291 INFO:journalctl@ceph.mds.b.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:14:19.292 DEBUG:teuthology.orchestra.run.smithi134:mds.h> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.h.service 2023-07-21T22:14:19.321 INFO:journalctl@ceph.mds.e.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:14:19.322 DEBUG:teuthology.orchestra.run.smithi134:mds.k> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.k.service 2023-07-21T22:14:19.349 INFO:journalctl@ceph.mds.h.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:14:19.350 DEBUG:teuthology.orchestra.run.smithi194:mds.c> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.c.service 2023-07-21T22:14:19.352 DEBUG:teuthology.orchestra.run.smithi194:mds.f> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.f.service 2023-07-21T22:14:19.376 INFO:journalctl@ceph.mds.k.smithi134.stdout:-- Logs begin at Fri 2023-07-21 21:46:59 UTC. -- 2023-07-21T22:14:19.380 INFO:journalctl@ceph.mds.c.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:14:19.381 DEBUG:teuthology.orchestra.run.smithi194:mds.i> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.i.service 2023-07-21T22:14:19.407 INFO:journalctl@ceph.mds.f.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:14:19.408 DEBUG:teuthology.orchestra.run.smithi194:mds.l> sudo journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.l.service 2023-07-21T22:14:19.433 INFO:journalctl@ceph.mds.i.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:14:19.435 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-21T22:14:19.436 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:14:19.459 INFO:journalctl@ceph.mds.l.smithi194.stdout:-- Logs begin at Fri 2023-07-21 21:46:48 UTC. -- 2023-07-21T22:14:19.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 -- 172.21.15.93:0/3135372015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 msgr2=0x7f4efc107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/3135372015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc107fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009990 tx=0x7f4eec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 -- 172.21.15.93:0/3135372015 shutdown_connections 2023-07-21T22:14:19.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/3135372015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc107fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/3135372015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc0fdd90 0x7f4efc0fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/3135372015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc100d60 0x7f4efc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 -- 172.21.15.93:0/3135372015 >> 172.21.15.93:0/3135372015 conn(0x7f4efc0f9770 msgr2=0x7f4efc0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:19.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 -- 172.21.15.93:0/3135372015 shutdown_connections 2023-07-21T22:14:19.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.528+0000 7f4f02dbb700 1 -- 172.21.15.93:0/3135372015 wait complete. 2023-07-21T22:14:19.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 Processor -- start 2023-07-21T22:14:19.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 -- start start 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc0fdd90 0x7f4efc1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc100d60 0x7f4efc0fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0ff9f0 con 0x7f4efc1016f0 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0ffb60 con 0x7f4efc0fdd90 2023-07-21T22:14:19.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f02dbb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc19f790 con 0x7f4efc100d60 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41762/0 (socket says 172.21.15.93:41762) 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f00b57700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc0fdd90 0x7f4efc1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 -- 172.21.15.93:0/180721291 learned_addr learned my addr 172.21.15.93:0/180721291 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4efbfff700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc100d60 0x7f4efc0fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 -- 172.21.15.93:0/180721291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc100d60 msgr2=0x7f4efc0fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc100d60 0x7f4efc0fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 -- 172.21.15.93:0/180721291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc0fdd90 msgr2=0x7f4efc1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc0fdd90 0x7f4efc1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.529+0000 7f4f01358700 1 -- 172.21.15.93:0/180721291 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0095f0 con 0x7f4efc1016f0 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4f01358700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005910 tx=0x7f4eec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec01d070 con 0x7f4efc1016f0 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eec0047e0 con 0x7f4efc1016f0 2023-07-21T22:14:19.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec005340 con 0x7f4efc1016f0 2023-07-21T22:14:19.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc19fa10 con 0x7f4efc1016f0 2023-07-21T22:14:19.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.530+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc19fde0 con 0x7f4efc1016f0 2023-07-21T22:14:19.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.531+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc10b320 con 0x7f4efc1016f0 2023-07-21T22:14:19.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.532+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4eec004950 con 0x7f4efc1016f0 2023-07-21T22:14:19.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.532+0000 7f4ef9ffb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.532+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4eec0cd810 con 0x7f4efc1016f0 2023-07-21T22:14:19.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.532+0000 7f4f00b57700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.533+0000 7f4f00b57700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 0x7f4ee80a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009550 tx=0x7f4ef40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:19.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.536+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4eec097490 con 0x7f4efc1016f0 2023-07-21T22:14:19.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.729+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4efc106970 con 0x7f4efc1016f0 2023-07-21T22:14:19.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.730+0000 7f4ef9ffb700 1 -- 172.21.15.93:0/180721291 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f4eec09aab0 con 0x7f4efc1016f0 2023-07-21T22:14:19.773 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:19.774 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":78,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:10.572001+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:19.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 msgr2=0x7f4ee80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 0x7f4ee80a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009550 tx=0x7f4ef40091b0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 msgr2=0x7f4efc0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4eec005910 tx=0x7f4eec00f690 comp rx=0 tx=0).stop 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 shutdown_connections 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4efc1016f0 0x7f4efc0ff250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4efc0fdd90 0x7f4efc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 --2- 172.21.15.93:0/180721291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4efc100d60 0x7f4efc0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 >> 172.21.15.93:0/180721291 conn(0x7f4efc0f9770 msgr2=0x7f4efc0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:19.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 shutdown_connections 2023-07-21T22:14:19.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.732+0000 7f4f02dbb700 1 -- 172.21.15.93:0/180721291 wait complete. 2023-07-21T22:14:19.787 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:19.787 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2023-07-21T22:14:19.787 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:19.788 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:19.789 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:19 smithi093 ceph-mon[96579]: Deploying daemon mds.l on smithi194 2023-07-21T22:14:19.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:14:19.791 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:14:19.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 -- 172.21.15.93:0/1076001400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 msgr2=0x7f9adc106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 --2- 172.21.15.93:0/1076001400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc106e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400b0a0 tx=0x7f9ad400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 -- 172.21.15.93:0/1076001400 shutdown_connections 2023-07-21T22:14:19.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 --2- 172.21.15.93:0/1076001400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc106e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 --2- 172.21.15.93:0/1076001400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 0x7f9adc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 --2- 172.21.15.93:0/1076001400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 -- 172.21.15.93:0/1076001400 >> 172.21.15.93:0/1076001400 conn(0x7f9adc0fa000 msgr2=0x7f9adc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:19.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 -- 172.21.15.93:0/1076001400 shutdown_connections 2023-07-21T22:14:19.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.872+0000 7f9ae282d700 1 -- 172.21.15.93:0/1076001400 wait complete. 2023-07-21T22:14:19.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 Processor -- start 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 -- start start 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 0x7f9adc068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d4e0 con 0x7f9adc100560 2023-07-21T22:14:19.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d650 con 0x7f9adc0ffbd0 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae282d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d810 con 0x7f9adc0fe9e0 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae0dca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae0dca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41776/0 (socket says 172.21.15.93:41776) 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.873+0000 7f9ae0dca700 1 -- 172.21.15.93:0/467396138 learned_addr learned my addr 172.21.15.93:0/467396138 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9adbfff700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 0x7f9adc068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9adb7fe700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 -- 172.21.15.93:0/467396138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 msgr2=0x7f9adc068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 0x7f9adc068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 -- 172.21.15.93:0/467396138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 msgr2=0x7f9adc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 -- 172.21.15.93:0/467396138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009d00 con 0x7f9adc100560 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9adb7fe700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae0dca700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400d010 tx=0x7f9ad400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:19.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400e040 con 0x7f9adc100560 2023-07-21T22:14:19.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc19da90 con 0x7f9adc100560 2023-07-21T22:14:19.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad4003e10 con 0x7f9adc100560 2023-07-21T22:14:19.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40053b0 con 0x7f9adc100560 2023-07-21T22:14:19.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.874+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc19df10 con 0x7f9adc100560 2023-07-21T22:14:19.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.875+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc10a210 con 0x7f9adc100560 2023-07-21T22:14:19.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.876+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9ad401d9d0 con 0x7f9adc100560 2023-07-21T22:14:19.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.877+0000 7f9ad97fa700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 0x7f9ac40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:19.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.877+0000 7f9adbfff700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 0x7f9ac40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:19.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.877+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ad40cd8a0 con 0x7f9adc100560 2023-07-21T22:14:19.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.877+0000 7f9adbfff700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 0x7f9ac40a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0ffa30 tx=0x7f9acc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:19.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:19.880+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ad4097520 con 0x7f9adc100560 2023-07-21T22:14:20.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:20.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2023-07-21T22:14:20.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:20.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:20.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:19 smithi194 ceph-mon[98288]: Deploying daemon mds.l on smithi194 2023-07-21T22:14:20.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.065+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9adc105870 con 0x7f9adc100560 2023-07-21T22:14:20.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.066+0000 7f9ad97fa700 1 -- 172.21.15.93:0/467396138 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ad409ab40 con 0x7f9adc100560 2023-07-21T22:14:20.109 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:14:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 msgr2=0x7f9ac40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 0x7f9ac40a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0ffa30 tx=0x7f9acc009260 comp rx=0 tx=0).stop 2023-07-21T22:14:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 msgr2=0x7f9adc19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400d010 tx=0x7f9ad400bd30 comp rx=0 tx=0).stop 2023-07-21T22:14:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 shutdown_connections 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc100560 0x7f9adc19cd10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9adc0fe9e0 0x7f9adc068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ac40a1150 0x7f9ac40a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 --2- 172.21.15.93:0/467396138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9adc0ffbd0 0x7f9adc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 >> 172.21.15.93:0/467396138 conn(0x7f9adc0fa000 msgr2=0x7f9adc0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 shutdown_connections 2023-07-21T22:14:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.068+0000 7f9ae282d700 1 -- 172.21.15.93:0/467396138 wait complete. 2023-07-21T22:14:20.123 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:20.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:20.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2023-07-21T22:14:20.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:20.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:20.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:20.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:20.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:20.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:20.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:20.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:20.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:19 smithi134 ceph-mon[97873]: Deploying daemon mds.l on smithi194 2023-07-21T22:14:20.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 -- 172.21.15.93:0/2326365061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd0100550 msgr2=0x7fbdd0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 --2- 172.21.15.93:0/2326365061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd0100550 0x7fbdd0106e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b0a0 tx=0x7fbdc800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 -- 172.21.15.93:0/2326365061 shutdown_connections 2023-07-21T22:14:20.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 --2- 172.21.15.93:0/2326365061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd0100550 0x7fbdd0106e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 --2- 172.21.15.93:0/2326365061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd00fe9d0 0x7fbdd00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 --2- 172.21.15.93:0/2326365061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.201+0000 7fbdd7521700 1 -- 172.21.15.93:0/2326365061 >> 172.21.15.93:0/2326365061 conn(0x7fbdd00f9fd0 msgr2=0x7fbdd00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:20.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- 172.21.15.93:0/2326365061 shutdown_connections 2023-07-21T22:14:20.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- 172.21.15.93:0/2326365061 wait complete. 2023-07-21T22:14:20.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 Processor -- start 2023-07-21T22:14:20.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- start start 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00fe9d0 0x7fbdd0195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd0100550 0x7fbdd019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd019d2f0 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd019d430 con 0x7fbdd00fe9d0 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd7521700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd019d5c0 con 0x7fbdd0100550 2023-07-21T22:14:20.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd4abc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd4abc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41788/0 (socket says 172.21.15.93:41788) 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd4abc700 1 -- 172.21.15.93:0/3641333525 learned_addr learned my addr 172.21.15.93:0/3641333525 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd5abe700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd0100550 0x7fbdd019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.202+0000 7fbdd52bd700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00fe9d0 0x7fbdd0195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 -- 172.21.15.93:0/3641333525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd0100550 msgr2=0x7fbdd019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd0100550 0x7fbdd019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 -- 172.21.15.93:0/3641333525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00fe9d0 msgr2=0x7fbdd0195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00fe9d0 0x7fbdd0195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 -- 172.21.15.93:0/3641333525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8009d00 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd4abc700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c9b0 tx=0x7fbdc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0007b00 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdc0007c60 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd019d8a0 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0018660 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.203+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd019ddb0 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.204+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd01067a0 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.205+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbdc000f450 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.205+0000 7fbdc67fc700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 0x7fbdbc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.205+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbdc00cc460 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.205+0000 7fbdd52bd700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 0x7fbdbc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.206+0000 7fbdd52bd700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 0x7fbdbc0a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:20.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.209+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbdc0096050 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.439 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 systemd[1]: Starting Ceph mds.l for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:20.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.441+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdd002ca10 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.441+0000 7fbdc67fc700 1 -- 172.21.15.93:0/3641333525 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fbdc0099670 con 0x7fbdd00ffbc0 2023-07-21T22:14:20.485 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:20.485 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-21T22:14:20.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 msgr2=0x7fbdbc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 0x7fbdbc0a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005fd0 tx=0x7fbdcc005e50 comp rx=0 tx=0).stop 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 msgr2=0x7fbdd0195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c9b0 tx=0x7fbdc000ccc0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 shutdown_connections 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdd0100550 0x7fbdd019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbdbc0a11a0 0x7fbdbc0a3650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd00fe9d0 0x7fbdd0195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 --2- 172.21.15.93:0/3641333525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdd00ffbc0 0x7fbdd0195cb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:20.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 >> 172.21.15.93:0/3641333525 conn(0x7fbdd00f9fd0 msgr2=0x7fbdd0068000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:20.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 shutdown_connections 2023-07-21T22:14:20.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:20.443+0000 7fbdd7521700 1 -- 172.21.15.93:0/3641333525 wait complete. 2023-07-21T22:14:20.488 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 1 2023-07-21T22:14:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:20 smithi093 ceph-mon[96579]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/180721291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/467396138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:20.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3641333525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:21.024 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-mon[98288]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:21.025 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/180721291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:21.025 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/467396138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:21.025 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3641333525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:21.026 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 podman[123527]: 2023-07-21T22:14:21.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:20 smithi134 ceph-mon[97873]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:21.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/180721291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:21.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/467396138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:21.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3641333525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:21.336 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-mds[123568]: -- 172.21.15.194:0/2331424687 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559df45d11e0 con 0x559df45a6c00 2023-07-21T22:14:21.336 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 bash[123527]: 04effaa8794f5c9b2c96f2190c48926732effe0153b79fc931c80ffe313aad66 2023-07-21T22:14:21.336 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: starting mds.l at 2023-07-21T22:14:21.337 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:14:20 smithi194 systemd[1]: Started Ceph mds.l for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:22.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6832/3279794938,v1:172.21.15.194:6833/3279794938] up:boot 2023-07-21T22:14:22.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: fsmap 1 up:standby 2023-07-21T22:14:22.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: Deploying daemon mds.k on smithi134 2023-07-21T22:14:22.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:22 smithi194 ceph-mon[98288]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:22.351 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6832/3279794938,v1:172.21.15.194:6833/3279794938] up:boot 2023-07-21T22:14:22.352 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: fsmap 1 up:standby 2023-07-21T22:14:22.352 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:14:22.352 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.352 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.352 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.353 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:22.353 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:22.353 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:22.353 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: Deploying daemon mds.k on smithi134 2023-07-21T22:14:22.353 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:22 smithi134 ceph-mon[97873]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:22.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6832/3279794938,v1:172.21.15.194:6833/3279794938] up:boot 2023-07-21T22:14:22.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: fsmap 1 up:standby 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:22.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: Deploying daemon mds.k on smithi134 2023-07-21T22:14:22.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:22 smithi093 ceph-mon[96579]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:22.499 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:22.628 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:22 smithi134 systemd[1]: Starting Ceph mds.k for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:22.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 -- 172.21.15.93:0/1610627634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 msgr2=0x7fc56c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 --2- 172.21.15.93:0/1610627634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c107f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b0a0 tx=0x7fc56400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 -- 172.21.15.93:0/1610627634 shutdown_connections 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 --2- 172.21.15.93:0/1610627634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c107f50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 --2- 172.21.15.93:0/1610627634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c0ffb10 0x7fc56c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 --2- 172.21.15.93:0/1610627634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c100d00 0x7fc56c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 -- 172.21.15.93:0/1610627634 >> 172.21.15.93:0/1610627634 conn(0x7fc56c0fb130 msgr2=0x7fc56c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:22.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 -- 172.21.15.93:0/1610627634 shutdown_connections 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.594+0000 7fc57331b700 1 -- 172.21.15.93:0/1610627634 wait complete. 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 Processor -- start 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 -- start start 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 0x7fc56c1958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c100d00 0x7fc56c195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c19d430 con 0x7fc56c101690 2023-07-21T22:14:22.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c19d570 con 0x7fc56c100d00 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc57331b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c19d700 con 0x7fc56c0ffb10 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41804/0 (socket says 172.21.15.93:41804) 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 -- 172.21.15.93:0/3750192664 learned_addr learned my addr 172.21.15.93:0/3750192664 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5708b6700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c100d00 0x7fc56c195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:22.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5710b7700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 0x7fc56c1958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 -- 172.21.15.93:0/3750192664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 msgr2=0x7fc56c1958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 0x7fc56c1958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 -- 172.21.15.93:0/3750192664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c100d00 msgr2=0x7fc56c195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c100d00 0x7fc56c195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 -- 172.21.15.93:0/3750192664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564009d00 con 0x7fc56c101690 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5710b7700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 0x7fc56c1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.595+0000 7fc5718b8700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b830 tx=0x7fc564004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:22.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.596+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400e040 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.596+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5640094d0 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.596+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c19d980 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.596+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56401bd20 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.596+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c19de50 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.597+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c106f10 con 0x7fc56c101690 2023-07-21T22:14:22.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.598+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc56401b450 con 0x7fc56c101690 2023-07-21T22:14:22.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.599+0000 7fc5627fc700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 0x7fc5580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:22.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.599+0000 7fc5710b7700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 0x7fc5580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:22.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.599+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5640ce790 con 0x7fc56c101690 2023-07-21T22:14:22.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.599+0000 7fc5710b7700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 0x7fc5580a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc56c100b60 tx=0x7fc55c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:22.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.602+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc564098410 con 0x7fc56c101690 2023-07-21T22:14:22.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.835+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc56c02ca60 con 0x7fc56c101690 2023-07-21T22:14:22.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.835+0000 7fc5627fc700 1 -- 172.21.15.93:0/3750192664 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fc56409ba30 con 0x7fc56c101690 2023-07-21T22:14:22.879 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:22.879 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:14:22.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 msgr2=0x7fc5580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:22.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 0x7fc5580a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc56c100b60 tx=0x7fc55c008040 comp rx=0 tx=0).stop 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 msgr2=0x7fc56c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b830 tx=0x7fc564004d00 comp rx=0 tx=0).stop 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 shutdown_connections 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc56c101690 0x7fc56c19ce60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc56c0ffb10 0x7fc56c1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc5580a1220 0x7fc5580a36d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 --2- 172.21.15.93:0/3750192664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc56c100d00 0x7fc56c195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:22.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 >> 172.21.15.93:0/3750192664 conn(0x7fc56c0fb130 msgr2=0x7fc56c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:22.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 shutdown_connections 2023-07-21T22:14:22.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:22.837+0000 7fc57331b700 1 -- 172.21.15.93:0/3750192664 wait complete. 2023-07-21T22:14:22.882 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 2 2023-07-21T22:14:23.131 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:22 smithi134 podman[122154]: 2023-07-21T22:14:23.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:23 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3750192664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:23.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:23 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3750192664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:23.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:23 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3750192664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:23.628 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:23 smithi134 ceph-mds[122194]: -- 172.21.15.134:0/1556321139 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581fb7bb1e0 con 0x5581fb791000 2023-07-21T22:14:23.629 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: starting mds.k at 2023-07-21T22:14:23.629 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:23 smithi134 bash[122154]: fbb34323aecf953711c010697feae9a75f59b8b9c4630577ba00d894a6b8fa54 2023-07-21T22:14:23.629 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:14:23 smithi134 systemd[1]: Started Ceph mds.k for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:24.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: Deploying daemon mds.j on smithi093 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6832/2616544286,v1:172.21.15.134:6833/2616544286] up:boot 2023-07-21T22:14:24.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: fsmap 2 up:standby 2023-07-21T22:14:24.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:24 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:14:24.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:24.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:24.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:24.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:24.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: Deploying daemon mds.j on smithi093 2023-07-21T22:14:24.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6832/2616544286,v1:172.21.15.134:6833/2616544286] up:boot 2023-07-21T22:14:24.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: fsmap 2 up:standby 2023-07-21T22:14:24.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:24 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:14:24.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: Deploying daemon mds.j on smithi093 2023-07-21T22:14:24.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6832/2616544286,v1:172.21.15.134:6833/2616544286] up:boot 2023-07-21T22:14:24.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: fsmap 2 up:standby 2023-07-21T22:14:24.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:24 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:14:24.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:24.920 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:24 smithi093 systemd[1]: Starting Ceph mds.j for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:25.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 -- 172.21.15.93:0/2939547507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068101680 msgr2=0x7f6068107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:25.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 --2- 172.21.15.93:0/2939547507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068101680 0x7f6068107f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f606000b0a0 tx=0x7f606000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 -- 172.21.15.93:0/2939547507 shutdown_connections 2023-07-21T22:14:25.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 --2- 172.21.15.93:0/2939547507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068101680 0x7f6068107f40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 --2- 172.21.15.93:0/2939547507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f60680fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 --2- 172.21.15.93:0/2939547507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6068100cf0 0x7f6068101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 -- 172.21.15.93:0/2939547507 >> 172.21.15.93:0/2939547507 conn(0x7f60680fb0e0 msgr2=0x7f60680fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:25.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 -- 172.21.15.93:0/2939547507 shutdown_connections 2023-07-21T22:14:25.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.980+0000 7f6070374700 1 -- 172.21.15.93:0/2939547507 wait complete. 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 Processor -- start 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 -- start start 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f6068195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6068101680 0x7f606819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606819d410 con 0x7f6068100cf0 2023-07-21T22:14:25.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606819d550 con 0x7f60680ffb00 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f6070374700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606819d6e0 con 0x7f6068101680 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f606d90f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f606d90f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41814/0 (socket says 172.21.15.93:41814) 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f606d90f700 1 -- 172.21.15.93:0/1191231949 learned_addr learned my addr 172.21.15.93:0/1191231949 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f606e110700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f6068195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.981+0000 7f606d90f700 1 -- 172.21.15.93:0/1191231949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6068101680 msgr2=0x7f606819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:25.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606d90f700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6068101680 0x7f606819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606d90f700 1 -- 172.21.15.93:0/1191231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 msgr2=0x7f6068195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606d90f700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f6068195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606d90f700 1 -- 172.21.15.93:0/1191231949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060009d00 con 0x7f6068100cf0 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606e110700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f6068195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f606d90f700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f605800d950 tx=0x7f605800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058009b40 con 0x7f6068100cf0 2023-07-21T22:14:25.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6058009ca0 con 0x7f6068100cf0 2023-07-21T22:14:25.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605800f620 con 0x7f6068100cf0 2023-07-21T22:14:25.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606819d9c0 con 0x7f6068100cf0 2023-07-21T22:14:25.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.982+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606819ded0 con 0x7f6068100cf0 2023-07-21T22:14:25.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.983+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6068106f50 con 0x7f6068100cf0 2023-07-21T22:14:25.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.984+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6058010460 con 0x7f6068100cf0 2023-07-21T22:14:25.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.985+0000 7f605f7fe700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 0x7f60540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:25.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.985+0000 7f606e110700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 0x7f60540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:25.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.985+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60580cc7f0 con 0x7f6068100cf0 2023-07-21T22:14:25.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.985+0000 7f606e110700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 0x7f60540a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6064005fd0 tx=0x7f6064005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:25.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:24.988+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6058096340 con 0x7f6068100cf0 2023-07-21T22:14:25.190 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:24 smithi093 podman[129945]: 2023-07-21T22:14:25.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.231+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f606802cce0 con 0x7f6068100cf0 2023-07-21T22:14:25.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.232+0000 7f605f7fe700 1 -- 172.21.15.93:0/1191231949 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f6058020800 con 0x7f6068100cf0 2023-07-21T22:14:25.275 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:25.275 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:14:25.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 msgr2=0x7f60540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:25.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 0x7f60540a36d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6064005fd0 tx=0x7f6064005f00 comp rx=0 tx=0).stop 2023-07-21T22:14:25.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 msgr2=0x7f6068195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f605800d950 tx=0x7f605800dd10 comp rx=0 tx=0).stop 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 shutdown_connections 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6068101680 0x7f606819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f60540a1220 0x7f60540a36d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60680ffb00 0x7f6068195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 --2- 172.21.15.93:0/1191231949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6068100cf0 0x7f6068195dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:25.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 >> 172.21.15.93:0/1191231949 conn(0x7f60680fb0e0 msgr2=0x7f6068104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:25.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 shutdown_connections 2023-07-21T22:14:25.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:25.234+0000 7f6070374700 1 -- 172.21.15.93:0/1191231949 wait complete. 2023-07-21T22:14:25.279 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 3 2023-07-21T22:14:25.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:25 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1191231949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:25.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:25 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6834/3390846004,v1:172.21.15.93:6835/3390846004] up:boot 2023-07-21T22:14:25.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:25 smithi194 ceph-mon[98288]: fsmap 3 up:standby 2023-07-21T22:14:25.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:25 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:14:25.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:25 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1191231949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:25.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:25 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6834/3390846004,v1:172.21.15.93:6835/3390846004] up:boot 2023-07-21T22:14:25.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:25 smithi134 ceph-mon[97873]: fsmap 3 up:standby 2023-07-21T22:14:25.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:25 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:14:25.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1191231949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6834/3390846004,v1:172.21.15.93:6835/3390846004] up:boot 2023-07-21T22:14:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-mon[96579]: fsmap 3 up:standby 2023-07-21T22:14:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:14:25.692 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-mds[130028]: -- 172.21.15.93:0/4254081810 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e350e51e0 con 0x560e350bac00 2023-07-21T22:14:25.692 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:25 smithi093 bash[129945]: 0dcf366153446a69c687c74a9242849a2fae89e162170dfbe9bb02dd6d3bc6c1 2023-07-21T22:14:25.693 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-j[130004]: starting mds.j at 2023-07-21T22:14:25.693 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:14:25 smithi093 systemd[1]: Started Ceph mds.j for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:26.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:26.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:26.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:26 smithi194 ceph-mon[98288]: Deploying daemon mds.d on smithi093 2023-07-21T22:14:26.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:26.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:26.613 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:26 smithi093 ceph-mon[96579]: Deploying daemon mds.d on smithi093 2023-07-21T22:14:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:26.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:26.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:26.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:26.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:26 smithi134 ceph-mon[97873]: Deploying daemon mds.d on smithi093 2023-07-21T22:14:26.940 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:26 smithi093 systemd[1]: Starting Ceph mds.d for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:27.291 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:27.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.378+0000 7f529427b700 1 -- 172.21.15.93:0/4180195489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 msgr2=0x7f528c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.378+0000 7f529427b700 1 --2- 172.21.15.93:0/4180195489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c100030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f527c009990 tx=0x7f527c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- 172.21.15.93:0/4180195489 shutdown_connections 2023-07-21T22:14:27.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- 172.21.15.93:0/4180195489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 0x7f528c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- 172.21.15.93:0/4180195489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 0x7f528c0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- 172.21.15.93:0/4180195489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c100030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- 172.21.15.93:0/4180195489 >> 172.21.15.93:0/4180195489 conn(0x7f528c0f9fd0 msgr2=0x7f528c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:27.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- 172.21.15.93:0/4180195489 shutdown_connections 2023-07-21T22:14:27.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- 172.21.15.93:0/4180195489 wait complete. 2023-07-21T22:14:27.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 Processor -- start 2023-07-21T22:14:27.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- start start 2023-07-21T22:14:27.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 0x7f528c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:27.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:27.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 0x7f528c19cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c19d310 con 0x7f528c0ffbe0 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c19d480 con 0x7f528c0fe9f0 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f529427b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c19d640 con 0x7f528c100570 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f5291816700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f5291816700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41854/0 (socket says 172.21.15.93:41854) 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f5291816700 1 -- 172.21.15.93:0/2414061452 learned_addr learned my addr 172.21.15.93:0/2414061452 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:27.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.379+0000 7f5292017700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 0x7f528c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5292818700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 0x7f528c19cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 -- 172.21.15.93:0/2414061452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 msgr2=0x7f528c19cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 0x7f528c19cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 -- 172.21.15.93:0/2414061452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 msgr2=0x7f528c069230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 0x7f528c069230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 -- 172.21.15.93:0/2414061452 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0095f0 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f5291816700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f527c000c00 tx=0x7f527c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c01d070 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f527c022470 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c19d8c0 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c017650 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.380+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c19dce0 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.381+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c105f40 con 0x7f528c0ffbe0 2023-07-21T22:14:27.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.382+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f527c0225e0 con 0x7f528c0ffbe0 2023-07-21T22:14:27.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.382+0000 7f52837fe700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 0x7f52780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:27.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.383+0000 7f5292017700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 0x7f52780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:27.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.383+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f527c0cde90 con 0x7f528c0ffbe0 2023-07-21T22:14:27.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.383+0000 7f5292017700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 0x7f52780a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f528c0ffa40 tx=0x7f5288009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:27.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.386+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f527c097b10 con 0x7f528c0ffbe0 2023-07-21T22:14:27.433 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:27 smithi093 podman[130329]: 2023-07-21T22:14:27.433 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:27 smithi093 ceph-mds[130392]: -- 172.21.15.93:0/60483840 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e6df06f1e0 con 0x55e6df044c00 2023-07-21T22:14:27.434 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-d[130345]: starting mds.d at 2023-07-21T22:14:27.434 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:27 smithi093 bash[130329]: 3c500534fd8bd7cb53d2919aa4eb4b889a0366b4305d70f28c70c25064b8fd36 2023-07-21T22:14:27.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.627+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f528c02c850 con 0x7f528c0ffbe0 2023-07-21T22:14:27.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.628+0000 7f52837fe700 1 -- 172.21.15.93:0/2414061452 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f527c09b130 con 0x7f528c0ffbe0 2023-07-21T22:14:27.672 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:27.672 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:14:27.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 msgr2=0x7f52780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 0x7f52780a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f528c0ffa40 tx=0x7f5288009260 comp rx=0 tx=0).stop 2023-07-21T22:14:27.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 msgr2=0x7f528c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f527c000c00 tx=0x7f527c0047a0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 shutdown_connections 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c100570 0x7f528c19cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f52780a11a0 0x7f52780a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f528c0fe9f0 0x7f528c069230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 --2- 172.21.15.93:0/2414061452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f528c0ffbe0 0x7f528c19a4f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 >> 172.21.15.93:0/2414061452 conn(0x7f528c0f9fd0 msgr2=0x7f528c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:27.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 shutdown_connections 2023-07-21T22:14:27.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:27.630+0000 7f529427b700 1 -- 172.21.15.93:0/2414061452 wait complete. 2023-07-21T22:14:27.676 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 4 2023-07-21T22:14:27.685 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:14:27 smithi093 systemd[1]: Started Ceph mds.d for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:28.751 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:28.751 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:28.752 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: Deploying daemon mds.e on smithi134 2023-07-21T22:14:28.753 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2414061452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:28.753 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6836/3934873649,v1:172.21.15.93:6837/3934873649] up:boot 2023-07-21T22:14:28.753 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: fsmap 4 up:standby 2023-07-21T22:14:28.753 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:28 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:14:28.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:28.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: Deploying daemon mds.e on smithi134 2023-07-21T22:14:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2414061452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:28.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6836/3934873649,v1:172.21.15.93:6837/3934873649] up:boot 2023-07-21T22:14:28.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: fsmap 4 up:standby 2023-07-21T22:14:28.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:28 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:14:28.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:28.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: Deploying daemon mds.e on smithi134 2023-07-21T22:14:28.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2414061452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:28.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6836/3934873649,v1:172.21.15.93:6837/3934873649] up:boot 2023-07-21T22:14:28.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: fsmap 4 up:standby 2023-07-21T22:14:28.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:28 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:14:29.128 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:28 smithi134 systemd[1]: Starting Ceph mds.e for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:29.542 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:29 smithi134 podman[122495]: 2023-07-21T22:14:29.686 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:29.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/2770127628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 msgr2=0x7f0b34106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:29.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/2770127628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b34106e90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009990 tx=0x7f0b2c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/2770127628 shutdown_connections 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/2770127628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b34106e90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/2770127628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 0x7f0b341003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/2770127628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 0x7f0b340ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/2770127628 >> 172.21.15.93:0/2770127628 conn(0x7f0b340f9770 msgr2=0x7f0b340fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/2770127628 shutdown_connections 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.784+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/2770127628 wait complete. 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 Processor -- start 2023-07-21T22:14:29.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 -- start start 2023-07-21T22:14:29.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 0x7f0b34109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:29.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:29.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 0x7f0b34107e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b341085a0 con 0x7f0b340ff5e0 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34108710 con 0x7f0b340fffe0 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.785+0000 7f0b3a0b3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419f860 con 0x7f0b340fec30 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41870/0 (socket says 172.21.15.93:41870) 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 -- 172.21.15.93:0/3399790256 learned_addr learned my addr 172.21.15.93:0/3399790256 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:29.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b337fe700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 0x7f0b34109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b33fff700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 0x7f0b34107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 -- 172.21.15.93:0/3399790256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 msgr2=0x7f0b34109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 0x7f0b34109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 -- 172.21.15.93:0/3399790256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 msgr2=0x7f0b34107e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 0x7f0b34107e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 -- 172.21.15.93:0/3399790256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0095f0 con 0x7f0b340ff5e0 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b2affd700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00d980 tx=0x7f0b1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:29.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c004220 con 0x7f0b340ff5e0 2023-07-21T22:14:29.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b1c007640 con 0x7f0b340ff5e0 2023-07-21T22:14:29.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3419fb40 con 0x7f0b340ff5e0 2023-07-21T22:14:29.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c005690 con 0x7f0b340ff5e0 2023-07-21T22:14:29.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.786+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b341a0020 con 0x7f0b340ff5e0 2023-07-21T22:14:29.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.787+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34105f50 con 0x7f0b340ff5e0 2023-07-21T22:14:29.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.789+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0b1c004750 con 0x7f0b340ff5e0 2023-07-21T22:14:29.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.789+0000 7f0b317fa700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 0x7f0b200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:29.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.790+0000 7f0b337fe700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 0x7f0b200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:29.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.790+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b1c0cc6f0 con 0x7f0b340ff5e0 2023-07-21T22:14:29.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.790+0000 7f0b337fe700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 0x7f0b200a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b34067450 tx=0x7f0b2400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:29.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:29.793+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0b1c0962e0 con 0x7f0b340ff5e0 2023-07-21T22:14:29.878 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:29 smithi134 ceph-mds[122535]: -- 172.21.15.134:0/4140805301 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e060e8f1e0 con 0x55e060e64800 2023-07-21T22:14:29.878 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:29 smithi134 bash[122495]: 1982af2ee81e5adc0ba99e80374459de01c548dfa9d4584908da5e77e013f73d 2023-07-21T22:14:29.878 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: starting mds.e at 2023-07-21T22:14:29.878 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:14:29 smithi134 systemd[1]: Started Ceph mds.e for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.023+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b3402c850 con 0x7f0b340ff5e0 2023-07-21T22:14:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.023+0000 7f0b317fa700 1 -- 172.21.15.93:0/3399790256 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f0b1c016de0 con 0x7f0b340ff5e0 2023-07-21T22:14:30.068 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:30.068 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:14:30.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.025+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 msgr2=0x7f0b200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:30.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.025+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 0x7f0b200a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b34067450 tx=0x7f0b2400b040 comp rx=0 tx=0).stop 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 msgr2=0x7f0b341078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00d980 tx=0x7f0b1c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 shutdown_connections 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0b200a11a0 0x7f0b200a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b340fffe0 0x7f0b34107e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:30.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b340fec30 0x7f0b34109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:30.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 --2- 172.21.15.93:0/3399790256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b340ff5e0 0x7f0b341078c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:30.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 >> 172.21.15.93:0/3399790256 conn(0x7f0b340f9770 msgr2=0x7f0b34103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:30.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 shutdown_connections 2023-07-21T22:14:30.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:30.026+0000 7f0b3a0b3700 1 -- 172.21.15.93:0/3399790256 wait complete. 2023-07-21T22:14:30.072 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 5 2023-07-21T22:14:30.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:30.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: Deploying daemon mds.f on smithi194 2023-07-21T22:14:30.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3399790256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:30.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6834/3262717838,v1:172.21.15.134:6835/3262717838] up:boot 2023-07-21T22:14:30.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: fsmap 5 up:standby 2023-07-21T22:14:30.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:30 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:14:30.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:30.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:30.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: Deploying daemon mds.f on smithi194 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3399790256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6834/3262717838,v1:172.21.15.134:6835/3262717838] up:boot 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: fsmap 5 up:standby 2023-07-21T22:14:30.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:30 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:14:30.988 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:30.989 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:30.990 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: Deploying daemon mds.f on smithi194 2023-07-21T22:14:30.990 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3399790256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:30.990 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6834/3262717838,v1:172.21.15.134:6835/3262717838] up:boot 2023-07-21T22:14:30.990 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: fsmap 5 up:standby 2023-07-21T22:14:30.990 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:30 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:14:31.336 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 systemd[1]: Starting Ceph mds.f for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:31.806 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 podman[123868]: 2023-07-21T22:14:32.083 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:32.086 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 ceph-mds[123908]: -- 172.21.15.194:0/4104840681 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4fe3571e0 con 0x55c4fe32d000 2023-07-21T22:14:32.086 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 bash[123868]: dc59b8726a1159a3104973b371989931c682a9ce8c4746f3b78e7ad1c0100783 2023-07-21T22:14:32.086 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: starting mds.f at 2023-07-21T22:14:32.086 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:14:31 smithi194 systemd[1]: Started Ceph mds.f for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:32.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- 172.21.15.93:0/1841916872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c0ffbc0 msgr2=0x7faa0c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 --2- 172.21.15.93:0/1841916872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c100010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009990 tx=0x7fa9fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- 172.21.15.93:0/1841916872 shutdown_connections 2023-07-21T22:14:32.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 --2- 172.21.15.93:0/1841916872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c100550 0x7faa0c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 --2- 172.21.15.93:0/1841916872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0fe9d0 0x7faa0c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 --2- 172.21.15.93:0/1841916872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c100010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- 172.21.15.93:0/1841916872 >> 172.21.15.93:0/1841916872 conn(0x7faa0c0f9fd0 msgr2=0x7faa0c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- 172.21.15.93:0/1841916872 shutdown_connections 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- 172.21.15.93:0/1841916872 wait complete. 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 Processor -- start 2023-07-21T22:14:32.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.182+0000 7faa146e6700 1 -- start start 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c0fe9d0 0x7faa0c068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c19d2d0 con 0x7faa0c100550 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c19d440 con 0x7faa0c0fe9d0 2023-07-21T22:14:32.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c19d600 con 0x7faa0c0ffbc0 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41886/0 (socket says 172.21.15.93:41886) 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 -- 172.21.15.93:0/2555393759 learned_addr learned my addr 172.21.15.93:0/2555393759 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa11c81700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12482700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c0fe9d0 0x7faa0c068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 -- 172.21.15.93:0/2555393759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0ffbc0 msgr2=0x7faa0c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 -- 172.21.15.93:0/2555393759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c0fe9d0 msgr2=0x7faa0c068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c0fe9d0 0x7faa0c068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 -- 172.21.15.93:0/2555393759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0095f0 con 0x7faa0c100550 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa12c83700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faa04007d70 tx=0x7faa0400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0400ac90 con 0x7faa0c100550 2023-07-21T22:14:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0400adf0 con 0x7faa0c100550 2023-07-21T22:14:32.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c19d8e0 con 0x7faa0c100550 2023-07-21T22:14:32.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040115e0 con 0x7faa0c100550 2023-07-21T22:14:32.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.183+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c19ddf0 con 0x7faa0c100550 2023-07-21T22:14:32.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.184+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c105810 con 0x7faa0c100550 2023-07-21T22:14:32.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.186+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faa04012420 con 0x7faa0c100550 2023-07-21T22:14:32.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.186+0000 7faa037fe700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 0x7fa9f80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.186+0000 7faa12482700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 0x7fa9f80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.186+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa040cc9c0 con 0x7faa0c100550 2023-07-21T22:14:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.187+0000 7faa12482700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 0x7fa9f80a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0ffa20 tx=0x7faa08007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.189+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faa04096510 con 0x7faa0c100550 2023-07-21T22:14:32.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.422+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa0c02c850 con 0x7faa0c100550 2023-07-21T22:14:32.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.422+0000 7faa037fe700 1 -- 172.21.15.93:0/2555393759 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4294 (secure 0 0 0) 0x7faa04099b30 con 0x7faa0c100550 2023-07-21T22:14:32.466 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:32.466 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:14:32.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 msgr2=0x7fa9f80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 0x7fa9f80a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0ffa20 tx=0x7faa08007400 comp rx=0 tx=0).stop 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 msgr2=0x7faa0c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faa04007d70 tx=0x7faa0400a340 comp rx=0 tx=0).stop 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 shutdown_connections 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa0c100550 0x7faa0c19cb00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa9f80a1220 0x7fa9f80a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa0c0fe9d0 0x7faa0c068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 --2- 172.21.15.93:0/2555393759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa0c0ffbc0 0x7faa0c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 >> 172.21.15.93:0/2555393759 conn(0x7faa0c0f9fd0 msgr2=0x7faa0c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:32.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 shutdown_connections 2023-07-21T22:14:32.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:32.424+0000 7faa146e6700 1 -- 172.21.15.93:0/2555393759 wait complete. 2023-07-21T22:14:32.469 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 6 2023-07-21T22:14:32.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:32 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2555393759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:32.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:32.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:32 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2555393759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:32.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:32.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:32.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:32.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:32.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:32.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:32 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2555393759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:33.800 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:33 smithi194 ceph-mon[98288]: Deploying daemon mds.c on smithi194 2023-07-21T22:14:33.800 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:33 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:boot 2023-07-21T22:14:33.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:33 smithi194 ceph-mon[98288]: fsmap 6 up:standby 2023-07-21T22:14:33.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:33 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:14:33.801 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:14:33 smithi194 systemd[1]: Starting Ceph mds.c for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:33.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:33 smithi134 ceph-mon[97873]: Deploying daemon mds.c on smithi194 2023-07-21T22:14:33.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:33 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:boot 2023-07-21T22:14:33.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:33 smithi134 ceph-mon[97873]: fsmap 6 up:standby 2023-07-21T22:14:33.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:33 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:14:33.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:33 smithi093 ceph-mon[96579]: Deploying daemon mds.c on smithi194 2023-07-21T22:14:33.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:33 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:boot 2023-07-21T22:14:33.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:33 smithi093 ceph-mon[96579]: fsmap 6 up:standby 2023-07-21T22:14:33.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:33 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:14:34.086 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:14:33 smithi194 podman[124208]: 2023-07-21T22:14:34.480 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:34.586 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: starting mds.c at 2023-07-21T22:14:34.586 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:14:34 smithi194 bash[124208]: 701e8086c85668a9afbc059d288cb3e29ea206213da2e49a24b5b82328525779 2023-07-21T22:14:34.586 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:14:34 smithi194 systemd[1]: Started Ceph mds.c for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:34.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 -- 172.21.15.93:0/3175394002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0100590 msgr2=0x7f55b0102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:34.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 --2- 172.21.15.93:0/3175394002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0100590 0x7f55b0102970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009990 tx=0x7f55ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 -- 172.21.15.93:0/3175394002 shutdown_connections 2023-07-21T22:14:34.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 --2- 172.21.15.93:0/3175394002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 0x7f55b0107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 --2- 172.21.15.93:0/3175394002 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0100590 0x7f55b0102970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 --2- 172.21.15.93:0/3175394002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0102eb0 0x7f55b0105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 -- 172.21.15.93:0/3175394002 >> 172.21.15.93:0/3175394002 conn(0x7f55b00f9fd0 msgr2=0x7f55b00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:34.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 -- 172.21.15.93:0/3175394002 shutdown_connections 2023-07-21T22:14:34.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.574+0000 7f55b7594700 1 -- 172.21.15.93:0/3175394002 wait complete. 2023-07-21T22:14:34.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 Processor -- start 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 -- start start 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0100590 0x7f55b0195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 0x7f55b019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019d380 con 0x7f55b0102eb0 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019d4c0 con 0x7f55b01057d0 2023-07-21T22:14:34.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b7594700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019d650 con 0x7f55b0100590 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b4b2f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b4b2f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41896/0 (socket says 172.21.15.93:41896) 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b4b2f700 1 -- 172.21.15.93:0/2308926474 learned_addr learned my addr 172.21.15.93:0/2308926474 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b5330700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0100590 0x7f55b0195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.575+0000 7f55b5b31700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 0x7f55b019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 -- 172.21.15.93:0/2308926474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0100590 msgr2=0x7f55b0195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0100590 0x7f55b0195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 -- 172.21.15.93:0/2308926474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 msgr2=0x7f55b019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 0x7f55b019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 -- 172.21.15.93:0/2308926474 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac0095f0 con 0x7f55b0102eb0 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b4b2f700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f55a000d950 tx=0x7f55a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000f890 con 0x7f55b0102eb0 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a0010460 con 0x7f55b0102eb0 2023-07-21T22:14:34.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000e610 con 0x7f55b0102eb0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b019d930 con 0x7f55b0102eb0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.576+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b019ddf0 con 0x7f55b0102eb0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.577+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b00fbb40 con 0x7f55b0102eb0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.578+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f55a00108d0 con 0x7f55b0102eb0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.578+0000 7f55a67fc700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 0x7f559c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.579+0000 7f55b5330700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 0x7f559c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:34.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.579+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55a00cc250 con 0x7f55b0102eb0 2023-07-21T22:14:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.579+0000 7f55b5330700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 0x7f559c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00b5c0 tx=0x7f55ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.583+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55a0095e40 con 0x7f55b0102eb0 2023-07-21T22:14:34.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.817+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55b002c850 con 0x7f55b0102eb0 2023-07-21T22:14:34.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.817+0000 7f55a67fc700 1 -- 172.21.15.93:0/2308926474 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f55a0099460 con 0x7f55b0102eb0 2023-07-21T22:14:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:34.861 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-21T22:14:34.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 msgr2=0x7f559c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 0x7f559c0a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00b5c0 tx=0x7f55ac005fd0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 msgr2=0x7f55b0195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f55a000d950 tx=0x7f55a000dc60 comp rx=0 tx=0).stop 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 shutdown_connections 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f559c0a11a0 0x7f559c0a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b01057d0 0x7f55b019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55b0100590 0x7f55b0195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 --2- 172.21.15.93:0/2308926474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55b0102eb0 0x7f55b0195d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:34.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.819+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 >> 172.21.15.93:0/2308926474 conn(0x7f55b00f9fd0 msgr2=0x7f55b0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:34.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.820+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 shutdown_connections 2023-07-21T22:14:34.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:34.820+0000 7f55b7594700 1 -- 172.21.15.93:0/2308926474 wait complete. 2023-07-21T22:14:34.865 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 8 2023-07-21T22:14:34.873 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:34.874 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:34 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:34.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:34.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:34.891 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:34.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:34.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:34 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:35.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:35.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:35.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:35.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:35.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:35.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:35.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:34 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:35.764 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:35 smithi134 ceph-mon[97873]: Deploying daemon mds.h on smithi134 2023-07-21T22:14:35.764 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:35 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:boot 2023-07-21T22:14:35.764 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:35 smithi134 ceph-mon[97873]: fsmap 7 up:standby 2023-07-21T22:14:35.764 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:35 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:14:35.765 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:35 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2308926474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:35.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:35 smithi093 ceph-mon[96579]: Deploying daemon mds.h on smithi134 2023-07-21T22:14:35.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:35 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:boot 2023-07-21T22:14:35.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:35 smithi093 ceph-mon[96579]: fsmap 7 up:standby 2023-07-21T22:14:35.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:35 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:14:35.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:35 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2308926474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:36.054 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:35 smithi134 systemd[1]: Starting Ceph mds.h for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:36.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:35 smithi194 ceph-mon[98288]: Deploying daemon mds.h on smithi134 2023-07-21T22:14:36.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:35 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:boot 2023-07-21T22:14:36.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:35 smithi194 ceph-mon[98288]: fsmap 7 up:standby 2023-07-21T22:14:36.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:35 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:14:36.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:35 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2308926474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:36.307 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:36 smithi134 podman[122837]: 2023-07-21T22:14:36.628 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/2198225135 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5578c3bb11e0 con 0x5578c3b86800 2023-07-21T22:14:36.628 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:36 smithi134 bash[122837]: 4d441787899916c6b7c384c6c65cb707af6dfaf3a128cb2075279b4a7753ea09 2023-07-21T22:14:36.629 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:14:36.629 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:14:36 smithi134 systemd[1]: Started Ceph mds.h for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:36.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:36.907 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:36.907 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:36.908 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:36.908 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:36.908 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:36.908 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:36.908 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:36 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:37.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 -- 172.21.15.93:0/3380742329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b00ffbc0 msgr2=0x7fb3b0100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:37.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3380742329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0100010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009990 tx=0x7fb3a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 -- 172.21.15.93:0/3380742329 shutdown_connections 2023-07-21T22:14:37.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3380742329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b0100550 0x7fb3b0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3380742329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3380742329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0100010 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 -- 172.21.15.93:0/3380742329 >> 172.21.15.93:0/3380742329 conn(0x7fb3b00f9fd0 msgr2=0x7fb3b00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:37.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 -- 172.21.15.93:0/3380742329 shutdown_connections 2023-07-21T22:14:37.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 -- 172.21.15.93:0/3380742329 wait complete. 2023-07-21T22:14:37.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.972+0000 7fb3b6103700 1 Processor -- start 2023-07-21T22:14:37.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 -- start start 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b01946f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019c270 con 0x7fb3b0100550 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019c3b0 con 0x7fb3b00ffbc0 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3b6103700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b019c540 con 0x7fb3b00fe9d0 2023-07-21T22:14:37.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41912/0 (socket says 172.21.15.93:41912) 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 -- 172.21.15.93:0/3592196558 learned_addr learned my addr 172.21.15.93:0/3592196558 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3aeffd700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3af7fe700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b01946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 -- 172.21.15.93:0/3592196558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 msgr2=0x7fb3b01946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b01946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 -- 172.21.15.93:0/3592196558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b00ffbc0 msgr2=0x7fb3b0194c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0194c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3affff700 1 -- 172.21.15.93:0/3592196558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40095f0 con 0x7fb3b0100550 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.973+0000 7fb3af7fe700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b01946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3affff700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000d980 tx=0x7fb3a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0004a30 con 0x7fb3b0100550 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a000b5c0 con 0x7fb3b0100550 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b019c820 con 0x7fb3b0100550 2023-07-21T22:14:37.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0009b50 con 0x7fb3b0100550 2023-07-21T22:14:37.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.974+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b019cce0 con 0x7fb3b0100550 2023-07-21T22:14:37.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.975+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0105f20 con 0x7fb3b0100550 2023-07-21T22:14:37.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.976+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3a0009cb0 con 0x7fb3b0100550 2023-07-21T22:14:37.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.976+0000 7fb3acff9700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 0x7fb3980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:37.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.976+0000 7fb3af7fe700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 0x7fb3980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:37.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.976+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb3a00cc3a0 con 0x7fb3b0100550 2023-07-21T22:14:37.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.977+0000 7fb3af7fe700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 0x7fb3980a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00ffa20 tx=0x7fb3a800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:37.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:36.980+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb3a0095ef0 con 0x7fb3b0100550 2023-07-21T22:14:37.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:37.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:37.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:37.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:36 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:37.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:37.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:37.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:37.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:37.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:36 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:37.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.213+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3b0061120 con 0x7fb3b0100550 2023-07-21T22:14:37.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.213+0000 7fb3acff9700 1 -- 172.21.15.93:0/3592196558 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6565 (secure 0 0 0) 0x7fb3a000ee60 con 0x7fb3b0100550 2023-07-21T22:14:37.257 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-21T22:14:37.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 msgr2=0x7fb3980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 0x7fb3980a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00ffa20 tx=0x7fb3a800b040 comp rx=0 tx=0).stop 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 msgr2=0x7fb3b019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000d980 tx=0x7fb3a000dc90 comp rx=0 tx=0).stop 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 shutdown_connections 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b0100550 0x7fb3b019bca0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3b00fe9d0 0x7fb3b01946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb3980a1220 0x7fb3980a36d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 --2- 172.21.15.93:0/3592196558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3b00ffbc0 0x7fb3b0194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:37.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 >> 172.21.15.93:0/3592196558 conn(0x7fb3b00f9fd0 msgr2=0x7fb3b0103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:37.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 shutdown_connections 2023-07-21T22:14:37.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:37.215+0000 7fb3b6103700 1 -- 172.21.15.93:0/3592196558 wait complete. 2023-07-21T22:14:37.260 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 9 2023-07-21T22:14:37.769 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:37 smithi194 ceph-mon[98288]: Deploying daemon mds.i on smithi194 2023-07-21T22:14:37.770 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:37 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/117715569,v1:172.21.15.134:6837/117715569] up:boot 2023-07-21T22:14:37.770 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:37 smithi194 ceph-mon[98288]: fsmap 8 up:standby 2023-07-21T22:14:37.770 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:37 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:14:37.771 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:37 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3592196558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:37 smithi093 ceph-mon[96579]: Deploying daemon mds.i on smithi194 2023-07-21T22:14:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:37 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/117715569,v1:172.21.15.134:6837/117715569] up:boot 2023-07-21T22:14:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:37 smithi093 ceph-mon[96579]: fsmap 8 up:standby 2023-07-21T22:14:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:37 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:14:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:37 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3592196558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:38.019 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:37 smithi194 systemd[1]: Starting Ceph mds.i for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:38.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:37 smithi134 ceph-mon[97873]: Deploying daemon mds.i on smithi194 2023-07-21T22:14:38.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:37 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/117715569,v1:172.21.15.134:6837/117715569] up:boot 2023-07-21T22:14:38.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:37 smithi134 ceph-mon[97873]: fsmap 8 up:standby 2023-07-21T22:14:38.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:37 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:14:38.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:37 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3592196558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:38.585 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:38 smithi194 podman[124547]: 2023-07-21T22:14:38.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:38 smithi194 ceph-mon[98288]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:38.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:38 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:38.836 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:38 smithi194 ceph-mds[124587]: -- 172.21.15.194:0/806972146 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1648fb1e0 con 0x55f1648d0800 2023-07-21T22:14:38.837 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: starting mds.i at 2023-07-21T22:14:38.837 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:38 smithi194 bash[124547]: 7e9037a91fa8d73a4e573f3be83da8d1c637fd7183e3e63363313ead5d145dd3 2023-07-21T22:14:38.837 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:38 smithi194 systemd[1]: Started Ceph mds.i for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:38.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:38 smithi093 ceph-mon[96579]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:38.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:38 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:39.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:38 smithi134 ceph-mon[97873]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:39.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:38 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:39.271 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:39.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 -- 172.21.15.93:0/1883473948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 msgr2=0x7fe1600fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:39.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 --2- 172.21.15.93:0/1883473948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1600fede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe150009990 tx=0x7fe150009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 -- 172.21.15.93:0/1883473948 shutdown_connections 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 --2- 172.21.15.93:0/1883473948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 0x7fe160106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 --2- 172.21.15.93:0/1883473948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1600fede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 --2- 172.21.15.93:0/1883473948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 0x7fe160100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 -- 172.21.15.93:0/1883473948 >> 172.21.15.93:0/1883473948 conn(0x7fe1600f9fd0 msgr2=0x7fe1600fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 -- 172.21.15.93:0/1883473948 shutdown_connections 2023-07-21T22:14:39.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.357+0000 7fe1662c1700 1 -- 172.21.15.93:0/1883473948 wait complete. 2023-07-21T22:14:39.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 Processor -- start 2023-07-21T22:14:39.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 -- start start 2023-07-21T22:14:39.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 0x7fe160194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 0x7fe16019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16019c260 con 0x7fe1600fe9d0 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16019c3a0 con 0x7fe160100550 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe1662c1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16019c530 con 0x7fe1600ffbc0 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:39.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41922/0 (socket says 172.21.15.93:41922) 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 -- 172.21.15.93:0/3893990469 learned_addr learned my addr 172.21.15.93:0/3893990469 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe16485e700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 0x7fe16019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15f7fe700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 0x7fe160194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 -- 172.21.15.93:0/3893990469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 msgr2=0x7fe160194c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 0x7fe160194c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 -- 172.21.15.93:0/3893990469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 msgr2=0x7fe16019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 0x7fe16019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.358+0000 7fe15ffff700 1 -- 172.21.15.93:0/3893990469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500095f0 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe15ffff700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe1500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15001d070 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe150004060 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16019c7b0 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150022d30 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.359+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16019ccb0 con 0x7fe1600fe9d0 2023-07-21T22:14:39.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.360+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe160105f60 con 0x7fe1600fe9d0 2023-07-21T22:14:39.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.361+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe150003ba0 con 0x7fe1600fe9d0 2023-07-21T22:14:39.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.361+0000 7fe15d7fa700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 0x7fe1480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:39.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.362+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1500cdac0 con 0x7fe1600fe9d0 2023-07-21T22:14:39.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.362+0000 7fe15f7fe700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 0x7fe1480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:39.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.362+0000 7fe15f7fe700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 0x7fe1480a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe154005950 tx=0x7fe1540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:39.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.365+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe150097740 con 0x7fe1600fe9d0 2023-07-21T22:14:39.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.622+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe16002c850 con 0x7fe1600fe9d0 2023-07-21T22:14:39.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.623+0000 7fe15d7fa700 1 -- 172.21.15.93:0/3893990469 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7fe15009ad60 con 0x7fe1600fe9d0 2023-07-21T22:14:39.666 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:39.666 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:14:39.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 msgr2=0x7fe1480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:39.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 0x7fe1480a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe154005950 tx=0x7fe1540058e0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 msgr2=0x7fe1601946e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:39.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe1500047b0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 shutdown_connections 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe1480a1220 0x7fe1480a36d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe160100550 0x7fe16019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1600fe9d0 0x7fe1601946e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 --2- 172.21.15.93:0/3893990469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1600ffbc0 0x7fe160194c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 >> 172.21.15.93:0/3893990469 conn(0x7fe1600f9fd0 msgr2=0x7fe1601030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 shutdown_connections 2023-07-21T22:14:39.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:39.626+0000 7fe1662c1700 1 -- 172.21.15.93:0/3893990469 wait complete. 2023-07-21T22:14:39.671 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 10 2023-07-21T22:14:39.875 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:39.875 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:39.875 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:39.875 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:39.876 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:39.876 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: Deploying daemon mds.g on smithi093 2023-07-21T22:14:39.876 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:boot 2023-07-21T22:14:39.876 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: fsmap 9 up:standby 2023-07-21T22:14:39.876 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:39 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:14:40.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:40.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:40.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:40.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:40.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:40.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: Deploying daemon mds.g on smithi093 2023-07-21T22:14:40.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:boot 2023-07-21T22:14:40.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: fsmap 9 up:standby 2023-07-21T22:14:40.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:39 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:14:40.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:40.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: Deploying daemon mds.g on smithi093 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:boot 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: fsmap 9 up:standby 2023-07-21T22:14:40.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:39 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:14:40.441 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 systemd[1]: Starting Ceph mds.g for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:40.707 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 podman[130947]: 2023-07-21T22:14:40.708 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:40 smithi093 ceph-mon[96579]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:40.708 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:40 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3893990469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:41.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:40 smithi194 ceph-mon[98288]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:41.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:40 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3893990469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:41.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:40 smithi134 ceph-mon[97873]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:41.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:40 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3893990469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:41.190 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 bash[130947]: 4567bb4295c9ce28b0b52c601314b67b41702398065c963516c62d1d7096fff2 2023-07-21T22:14:41.190 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 ceph-mds[130987]: -- 172.21.15.93:0/881549038 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b298a231e0 con 0x55b2989f9000 2023-07-21T22:14:41.191 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: starting mds.g at 2023-07-21T22:14:41.191 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:14:40 smithi093 systemd[1]: Started Ceph mds.g for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:41.684 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:41.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 -- 172.21.15.93:0/462878299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe728100d60 msgr2=0x7fe7281011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:41.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 --2- 172.21.15.93:0/462878299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe728100d60 0x7fe7281011b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe718009990 tx=0x7fe718009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 -- 172.21.15.93:0/462878299 shutdown_connections 2023-07-21T22:14:41.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 --2- 172.21.15.93:0/462878299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7281016f0 0x7fe728107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 --2- 172.21.15.93:0/462878299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 0x7fe7280fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 --2- 172.21.15.93:0/462878299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe728100d60 0x7fe7281011b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 -- 172.21.15.93:0/462878299 >> 172.21.15.93:0/462878299 conn(0x7fe7280f9770 msgr2=0x7fe7280fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:41.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 -- 172.21.15.93:0/462878299 shutdown_connections 2023-07-21T22:14:41.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.771+0000 7fe72ce79700 1 -- 172.21.15.93:0/462878299 wait complete. 2023-07-21T22:14:41.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.772+0000 7fe72ce79700 1 Processor -- start 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.772+0000 7fe72ce79700 1 -- start start 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 0x7fe728195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe728100d60 0x7fe728195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819d310 con 0x7fe7281016f0 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819d450 con 0x7fe7280fdd90 2023-07-21T22:14:41.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819d5e0 con 0x7fe728100d60 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41954/0 (socket says 172.21.15.93:41954) 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 -- 172.21.15.93:0/1796112221 learned_addr learned my addr 172.21.15.93:0/1796112221 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72659c700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 0x7fe728195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe725d9b700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe728100d60 0x7fe728195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 -- 172.21.15.93:0/1796112221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe728100d60 msgr2=0x7fe728195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:41.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe728100d60 0x7fe728195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 -- 172.21.15.93:0/1796112221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 msgr2=0x7fe728195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 0x7fe728195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 -- 172.21.15.93:0/1796112221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180095f0 con 0x7fe7281016f0 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe726d9d700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe720007ee0 tx=0x7fe72000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720017070 con 0x7fe7281016f0 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72819d8c0 con 0x7fe7281016f0 2023-07-21T22:14:41.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe72000f040 con 0x7fe7281016f0 2023-07-21T22:14:41.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720005580 con 0x7fe7281016f0 2023-07-21T22:14:41.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.773+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72819dd80 con 0x7fe7281016f0 2023-07-21T22:14:41.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.774+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72810b2d0 con 0x7fe7281016f0 2023-07-21T22:14:41.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.775+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe72000db20 con 0x7fe7281016f0 2023-07-21T22:14:41.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.776+0000 7fe7177fe700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 0x7fe7100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:41.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.776+0000 7fe72659c700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 0x7fe7100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:41.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.776+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe7200cbba0 con 0x7fe7281016f0 2023-07-21T22:14:41.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.776+0000 7fe72659c700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 0x7fe7100a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe728067470 tx=0x7fe71c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:41.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:41.780+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe720095790 con 0x7fe7281016f0 2023-07-21T22:14:42.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: Deploying daemon mds.a on smithi093 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6838/3482994338,v1:172.21.15.93:6839/3482994338] up:boot 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: fsmap 10 up:standby 2023-07-21T22:14:42.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:41 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:14:42.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.033+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe72810b4c0 con 0x7fe7281016f0 2023-07-21T22:14:42.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.033+0000 7fe7177fe700 1 -- 172.21.15.93:0/1796112221 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8085 (secure 0 0 0) 0x7fe720098db0 con 0x7fe7281016f0 2023-07-21T22:14:42.077 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:42.078 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:14:42.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.035+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 msgr2=0x7fe7100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:42.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.035+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 0x7fe7100a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe728067470 tx=0x7fe71c00a380 comp rx=0 tx=0).stop 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.035+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 msgr2=0x7fe72819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.035+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe720007ee0 tx=0x7fe72000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 shutdown_connections 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7281016f0 0x7fe72819cd40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe7100a11a0 0x7fe7100a3650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7280fdd90 0x7fe728195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:42.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 --2- 172.21.15.93:0/1796112221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe728100d60 0x7fe728195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:42.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 >> 172.21.15.93:0/1796112221 conn(0x7fe7280f9770 msgr2=0x7fe7280fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:42.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 shutdown_connections 2023-07-21T22:14:42.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:42.036+0000 7fe72ce79700 1 -- 172.21.15.93:0/1796112221 wait complete. 2023-07-21T22:14:42.081 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 11 2023-07-21T22:14:42.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:42.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: Deploying daemon mds.a on smithi093 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6838/3482994338,v1:172.21.15.93:6839/3482994338] up:boot 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: fsmap 10 up:standby 2023-07-21T22:14:42.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:41 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:14:42.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: Deploying daemon mds.a on smithi093 2023-07-21T22:14:42.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:42.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6838/3482994338,v1:172.21.15.93:6839/3482994338] up:boot 2023-07-21T22:14:42.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: fsmap 10 up:standby 2023-07-21T22:14:42.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:41 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:14:42.440 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 systemd[1]: Starting Ceph mds.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:42.823 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 podman[131332]: 2023-07-21T22:14:42.824 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mds[131372]: -- 172.21.15.93:0/1124041628 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55691ff071e0 con 0x55691fedc800 2023-07-21T22:14:42.824 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: starting mds.a at 2023-07-21T22:14:43.190 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 bash[131332]: eec88a370793076103bbb6a2e29335b0c769de4c161b65dcaf6acbaec218cc36 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:14:42 smithi093 systemd[1]: Started Ceph mds.a for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1796112221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:43.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:43.192 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:42 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:43.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1796112221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:43.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:43.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:43.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:42 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:43.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1796112221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:43.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:43.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:14:43.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:14:43.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:42 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:44.092 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:44.206 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:43 smithi134 ceph-mon[97873]: Deploying daemon mds.b on smithi134 2023-07-21T22:14:44.206 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:43 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:boot 2023-07-21T22:14:44.206 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:43 smithi134 ceph-mon[97873]: fsmap 11 up:standby 2023-07-21T22:14:44.207 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:43 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:14:44.207 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:43 smithi134 ceph-mon[97873]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:44.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 -- 172.21.15.93:0/397936036 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0ffbe0 msgr2=0x7f840c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 --2- 172.21.15.93:0/397936036 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0ffbe0 0x7f840c100030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8400009990 tx=0x7f8400009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 -- 172.21.15.93:0/397936036 shutdown_connections 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 --2- 172.21.15.93:0/397936036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 0x7f840c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 --2- 172.21.15.93:0/397936036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0fe9f0 0x7f840c0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 --2- 172.21.15.93:0/397936036 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0ffbe0 0x7f840c100030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 -- 172.21.15.93:0/397936036 >> 172.21.15.93:0/397936036 conn(0x7f840c0f9fd0 msgr2=0x7f840c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:44.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 -- 172.21.15.93:0/397936036 shutdown_connections 2023-07-21T22:14:44.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.185+0000 7f8412f1c700 1 -- 172.21.15.93:0/397936036 wait complete. 2023-07-21T22:14:44.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.186+0000 7f8412f1c700 1 Processor -- start 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.186+0000 7f8412f1c700 1 -- start start 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.186+0000 7f8412f1c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0ffbe0 0x7f840c194c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 0x7f840c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19c250 con 0x7f840c0fe9f0 2023-07-21T22:14:44.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19c390 con 0x7f840c100570 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c19c520 con 0x7f840c0ffbe0 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41976/0 (socket says 172.21.15.93:41976) 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 -- 172.21.15.93:0/1968743724 learned_addr learned my addr 172.21.15.93:0/1968743724 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f84114b9700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 0x7f840c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f840bfff700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0ffbe0 0x7f840c194c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:44.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 -- 172.21.15.93:0/1968743724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0ffbe0 msgr2=0x7f840c194c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0ffbe0 0x7f840c194c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 -- 172.21.15.93:0/1968743724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 msgr2=0x7f840c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 0x7f840c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 -- 172.21.15.93:0/1968743724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000095f0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8410cb8700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cfb0 tx=0x7f83fc00e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011070 con 0x7f840c0fe9f0 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83fc007be0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c068700 con 0x7f840c0fe9f0 2023-07-21T22:14:44.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00a8f0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.187+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c068ce0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.188+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c10a240 con 0x7f840c0fe9f0 2023-07-21T22:14:44.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.189+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f83fc00ea60 con 0x7f840c0fe9f0 2023-07-21T22:14:44.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.190+0000 7f8409ffb700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 0x7f83f40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:44.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.190+0000 7f840bfff700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 0x7f83f40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:44.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.190+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83fc0cdc60 con 0x7f840c0fe9f0 2023-07-21T22:14:44.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.190+0000 7f840bfff700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 0x7f83f40a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:44.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.194+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f83fc0978e0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:43 smithi194 ceph-mon[98288]: Deploying daemon mds.b on smithi134 2023-07-21T22:14:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:43 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:boot 2023-07-21T22:14:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:43 smithi194 ceph-mon[98288]: fsmap 11 up:standby 2023-07-21T22:14:44.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:43 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:14:44.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:43 smithi194 ceph-mon[98288]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:43 smithi093 ceph-mon[96579]: Deploying daemon mds.b on smithi134 2023-07-21T22:14:44.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:43 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:boot 2023-07-21T22:14:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:43 smithi093 ceph-mon[96579]: fsmap 11 up:standby 2023-07-21T22:14:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:43 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:14:44.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:43 smithi093 ceph-mon[96579]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:44.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.425+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f840c105ce0 con 0x7f840c0fe9f0 2023-07-21T22:14:44.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.426+0000 7f8409ffb700 1 -- 172.21.15.93:0/1968743724 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8843 (secure 0 0 0) 0x7f83fc09af00 con 0x7f840c0fe9f0 2023-07-21T22:14:44.469 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:44.470 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:14:44.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 msgr2=0x7f83f40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:44.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 0x7f83f40a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005fb0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 msgr2=0x7f840c1946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cfb0 tx=0x7f83fc00e330 comp rx=0 tx=0).stop 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 shutdown_connections 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83f40a1220 0x7f83f40a36d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f840c100570 0x7f840c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f840c0fe9f0 0x7f840c1946d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 --2- 172.21.15.93:0/1968743724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f840c0ffbe0 0x7f840c194c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:44.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 >> 172.21.15.93:0/1968743724 conn(0x7f840c0f9fd0 msgr2=0x7f840c103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:44.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 shutdown_connections 2023-07-21T22:14:44.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:44.428+0000 7f8412f1c700 1 -- 172.21.15.93:0/1968743724 wait complete. 2023-07-21T22:14:44.473 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 12 2023-07-21T22:14:44.629 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 systemd[1]: Starting Ceph mds.b for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:14:44.956 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 podman[123177]: 2023-07-21T22:14:45.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:45 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1968743724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:45.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:45 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1968743724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:45.378 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 bash[123177]: 707b57e11fc9192fa4fa0b9ef019bbe296c221f30978e316e8fccc4b511bf614 2023-07-21T22:14:45.378 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 ceph-mds[123217]: -- 172.21.15.134:0/658146279 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a6076c91e0 con 0x55a60769ec00 2023-07-21T22:14:45.379 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: starting mds.b at 2023-07-21T22:14:45.379 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:14:44 smithi134 systemd[1]: Started Ceph mds.b for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:14:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:45 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1968743724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:boot 2023-07-21T22:14:46.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: fsmap 12 up:standby 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:46.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:46 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:boot 2023-07-21T22:14:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: fsmap 12 up:standby 2023-07-21T22:14:46.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:46.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:46 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:boot 2023-07-21T22:14:46.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: fsmap 12 up:standby 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:46.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:46.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:46 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:46.484 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:46.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 -- 172.21.15.93:0/3008150757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc428100030 msgr2=0x7fc428100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:46.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 --2- 172.21.15.93:0/3008150757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc428100030 0x7fc428100440 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc424009990 tx=0x7fc424009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 -- 172.21.15.93:0/3008150757 shutdown_connections 2023-07-21T22:14:46.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 --2- 172.21.15.93:0/3008150757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280ff6f0 0x7fc428106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 --2- 172.21.15.93:0/3008150757 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc428100030 0x7fc428100440 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 --2- 172.21.15.93:0/3008150757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4280fed60 0x7fc4280ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 -- 172.21.15.93:0/3008150757 >> 172.21.15.93:0/3008150757 conn(0x7fc4280f9820 msgr2=0x7fc4280fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 -- 172.21.15.93:0/3008150757 shutdown_connections 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.572+0000 7fc4302d4700 1 -- 172.21.15.93:0/3008150757 wait complete. 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 Processor -- start 2023-07-21T22:14:46.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 -- start start 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280fed60 0x7fc42810d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc428100030 0x7fc42810ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42810f340 con 0x7fc4280ff6f0 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42810f4b0 con 0x7fc428100030 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc4302d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42810f670 con 0x7fc4280fed60 2023-07-21T22:14:46.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41992/0 (socket says 172.21.15.93:41992) 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 -- 172.21.15.93:0/426368260 learned_addr learned my addr 172.21.15.93:0/426368260 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42e070700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280fed60 0x7fc42810d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42e871700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc428100030 0x7fc42810ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 -- 172.21.15.93:0/426368260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280fed60 msgr2=0x7fc42810d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280fed60 0x7fc42810d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 -- 172.21.15.93:0/426368260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc428100030 msgr2=0x7fc42810ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.573+0000 7fc42d86f700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc428100030 0x7fc42810ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc42d86f700 1 -- 172.21.15.93:0/426368260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240095f0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc42d86f700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b530 tx=0x7fc41800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418011830 con 0x7fc4280ff6f0 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc418011e50 con 0x7fc4280ff6f0 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42810f8f0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41800f2e0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.574+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42810fe00 con 0x7fc4280ff6f0 2023-07-21T22:14:46.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.575+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc428106140 con 0x7fc4280ff6f0 2023-07-21T22:14:46.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.576+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc4180105a0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.577+0000 7fc41f7fe700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 0x7fc4140a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:46.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.577+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4180cc450 con 0x7fc4280ff6f0 2023-07-21T22:14:46.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.577+0000 7fc42e070700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 0x7fc4140a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:46.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.577+0000 7fc42e070700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 0x7fc4140a36b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc42400b2e0 tx=0x7fc424005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:46.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.580+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc418095fa0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.826+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc428067750 con 0x7fc4280ff6f0 2023-07-21T22:14:46.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.826+0000 7fc41f7fe700 1 -- 172.21.15.93:0/426368260 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9604 (secure 0 0 0) 0x7fc4180995c0 con 0x7fc4280ff6f0 2023-07-21T22:14:46.870 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:46.870 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:14:46.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.828+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 msgr2=0x7fc4140a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:46.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 0x7fc4140a36b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc42400b2e0 tx=0x7fc424005fb0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 msgr2=0x7fc42810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:46.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b530 tx=0x7fc41800b840 comp rx=0 tx=0).stop 2023-07-21T22:14:46.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 shutdown_connections 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4140a1200 0x7fc4140a36b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc428100030 0x7fc42810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4280fed60 0x7fc42810d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 --2- 172.21.15.93:0/426368260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4280ff6f0 0x7fc42810dd30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 >> 172.21.15.93:0/426368260 conn(0x7fc4280f9820 msgr2=0x7fc428103280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 shutdown_connections 2023-07-21T22:14:46.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.829+0000 7fc4302d4700 1 -- 172.21.15.93:0/426368260 wait complete. 2023-07-21T22:14:46.874 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 13 2023-07-21T22:14:46.884 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:14:47.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 -- 172.21.15.93:0/2223523781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc0ffb00 msgr2=0x7f73cc0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/2223523781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc0ffb00 0x7f73cc0fff10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009990 tx=0x7f73b4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 -- 172.21.15.93:0/2223523781 shutdown_connections 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/2223523781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 0x7f73cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/2223523781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc0ffb00 0x7f73cc0fff10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/2223523781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc100cf0 0x7f73cc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 -- 172.21.15.93:0/2223523781 >> 172.21.15.93:0/2223523781 conn(0x7f73cc0fb0e0 msgr2=0x7f73cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 -- 172.21.15.93:0/2223523781 shutdown_connections 2023-07-21T22:14:47.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.971+0000 7f73d0e5d700 1 -- 172.21.15.93:0/2223523781 wait complete. 2023-07-21T22:14:47.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 Processor -- start 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 -- start start 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc0ffb00 0x7f73cc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 0x7f73cc19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19f680 con 0x7f73cc100cf0 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19f7c0 con 0x7f73cc101680 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73d0e5d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19f950 con 0x7f73cc0ffb00 2023-07-21T22:14:47.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42006/0 (socket says 172.21.15.93:42006) 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 -- 172.21.15.93:0/1468894997 learned_addr learned my addr 172.21.15.93:0/1468894997 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73ca59c700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc0ffb00 0x7f73cc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73cad9d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 0x7f73cc19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 -- 172.21.15.93:0/1468894997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc0ffb00 msgr2=0x7f73cc197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc0ffb00 0x7f73cc197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 -- 172.21.15.93:0/1468894997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 msgr2=0x7f73cc19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 0x7f73cc19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.972+0000 7f73c9d9b700 1 -- 172.21.15.93:0/1468894997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b40095f0 con 0x7f73cc100cf0 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73c9d9b700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00b530 tx=0x7f73bc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc011830 con 0x7f73cc100cf0 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73bc011e50 con 0x7f73cc100cf0 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc19fc30 con 0x7f73cc100cf0 2023-07-21T22:14:47.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00f2e0 con 0x7f73cc100cf0 2023-07-21T22:14:47.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.973+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a0190 con 0x7f73cc100cf0 2023-07-21T22:14:47.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.974+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc192210 con 0x7f73cc100cf0 2023-07-21T22:14:47.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.975+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f73bc0105a0 con 0x7f73cc100cf0 2023-07-21T22:14:47.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.975+0000 7f73c37fe700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 0x7f73b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.975+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73bc0cbda0 con 0x7f73cc100cf0 2023-07-21T22:14:47.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.975+0000 7f73ca59c700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 0x7f73b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.976+0000 7f73ca59c700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 0x7f73b80a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73b400b5c0 tx=0x7f73b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:46.980+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f73bc095990 con 0x7f73cc100cf0 2023-07-21T22:14:47.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.181+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73cc02caa0 con 0x7f73cc100cf0 2023-07-21T22:14:47.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.182+0000 7f73c37fe700 1 -- 172.21.15.93:0/1468894997 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f73bc098fb0 con 0x7f73cc100cf0 2023-07-21T22:14:47.226 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:47.226 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":78,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:10.572001+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:47.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 msgr2=0x7f73b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 0x7f73b80a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73b400b5c0 tx=0x7f73b4005960 comp rx=0 tx=0).stop 2023-07-21T22:14:47.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 msgr2=0x7f73cc198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00b530 tx=0x7f73bc00b840 comp rx=0 tx=0).stop 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 shutdown_connections 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f73b80a1220 0x7f73b80a36d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73cc101680 0x7f73cc19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73cc0ffb00 0x7f73cc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 --2- 172.21.15.93:0/1468894997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73cc100cf0 0x7f73cc198040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 >> 172.21.15.93:0/1468894997 conn(0x7f73cc0fb0e0 msgr2=0x7f73cc1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 shutdown_connections 2023-07-21T22:14:47.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.186+0000 7f73d0e5d700 1 -- 172.21.15.93:0/1468894997 wait complete. 2023-07-21T22:14:47.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:14:47.242 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:14:47.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 -- 172.21.15.93:0/1126047368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 msgr2=0x7f90f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 --2- 172.21.15.93:0/1126047368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f0106e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b0a0 tx=0x7f90e800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 -- 172.21.15.93:0/1126047368 shutdown_connections 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 --2- 172.21.15.93:0/1126047368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f0106e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 --2- 172.21.15.93:0/1126047368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00fea20 0x7f90f00fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 --2- 172.21.15.93:0/1126047368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00ffc10 0x7f90f0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 -- 172.21.15.93:0/1126047368 >> 172.21.15.93:0/1126047368 conn(0x7f90f00fa020 msgr2=0x7f90f00fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 -- 172.21.15.93:0/1126047368 shutdown_connections 2023-07-21T22:14:47.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.327+0000 7f90f879c700 1 -- 172.21.15.93:0/1126047368 wait complete. 2023-07-21T22:14:47.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 Processor -- start 2023-07-21T22:14:47.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 -- start start 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00fea20 0x7f90f0194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00ffc10 0x7f90f0194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019c310 con 0x7f90f01005a0 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019c450 con 0x7f90f00ffc10 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f879c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019c5e0 con 0x7f90f00fea20 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f6d39700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f6d39700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42022/0 (socket says 172.21.15.93:42022) 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f6d39700 1 -- 172.21.15.93:0/3181797590 learned_addr learned my addr 172.21.15.93:0/3181797590 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f5d37700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00ffc10 0x7f90f0194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.328+0000 7f90f6538700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00fea20 0x7f90f0194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 -- 172.21.15.93:0/3181797590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00fea20 msgr2=0x7f90f0194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00fea20 0x7f90f0194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 -- 172.21.15.93:0/3181797590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00ffc10 msgr2=0x7f90f0194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00ffc10 0x7f90f0194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 -- 172.21.15.93:0/3181797590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e8009d00 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f6d39700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b830 tx=0x7f90e8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800e040 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90e801ab90 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f0068600 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80056f0 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.329+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f0068b10 con 0x7f90f01005a0 2023-07-21T22:14:47.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.330+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f010a200 con 0x7f90f01005a0 2023-07-21T22:14:47.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.332+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f90e801ad00 con 0x7f90f01005a0 2023-07-21T22:14:47.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.333+0000 7f90e77fe700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 0x7f90dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.333+0000 7f90f6538700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 0x7f90dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.333+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90e80cf8c0 con 0x7f90f01005a0 2023-07-21T22:14:47.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.333+0000 7f90f6538700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 0x7f90dc0a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f90f00ffa70 tx=0x7f90ec007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.336+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f90e80995d0 con 0x7f90f01005a0 2023-07-21T22:14:47.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:47 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:47.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:47 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/426368260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:47 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1468894997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.548+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90f0069400 con 0x7f90f01005a0 2023-07-21T22:14:47.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.549+0000 7f90e77fe700 1 -- 172.21.15.93:0/3181797590 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90e809cbf0 con 0x7f90f01005a0 2023-07-21T22:14:47.592 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 msgr2=0x7f90dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 0x7f90dc0a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f90f00ffa70 tx=0x7f90ec007400 comp rx=0 tx=0).stop 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 msgr2=0x7f90f019bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b830 tx=0x7f90e8004c80 comp rx=0 tx=0).stop 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 shutdown_connections 2023-07-21T22:14:47.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90f01005a0 0x7f90f019bd40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f00fea20 0x7f90f0194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90dc0a1150 0x7f90dc0a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 --2- 172.21.15.93:0/3181797590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f00ffc10 0x7f90f0194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 >> 172.21.15.93:0/3181797590 conn(0x7f90f00fa020 msgr2=0x7f90f00fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 shutdown_connections 2023-07-21T22:14:47.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.551+0000 7f90f879c700 1 -- 172.21.15.93:0/3181797590 wait complete. 2023-07-21T22:14:47.604 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:47 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:47.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:47 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/426368260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:47 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1468894997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.606 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-21T22:14:47.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:47 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:47.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:47 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/426368260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:47 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1468894997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1077291187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0780ffbf0 msgr2=0x7fb078100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1077291187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0780ffbf0 0x7fb078100040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1077291187 shutdown_connections 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1077291187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb078100580 0x7fb078106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1077291187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780fea00 0x7fb0780fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1077291187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0780ffbf0 0x7fb078100040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1077291187 >> 172.21.15.93:0/1077291187 conn(0x7fb0780fa000 msgr2=0x7fb0780fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1077291187 shutdown_connections 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1077291187 wait complete. 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 Processor -- start 2023-07-21T22:14:47.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.689+0000 7fb07fcfa700 1 -- start start 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0780fea00 0x7fb078069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 0x7fb07819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819d250 con 0x7fb078100580 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819d3c0 con 0x7fb0780ffbf0 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819d580 con 0x7fb0780fea00 2023-07-21T22:14:47.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42036/0 (socket says 172.21.15.93:42036) 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07da96700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0780fea00 0x7fb078069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 -- 172.21.15.93:0/1921893869 learned_addr learned my addr 172.21.15.93:0/1921893869 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07d295700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 0x7fb07819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 -- 172.21.15.93:0/1921893869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0780fea00 msgr2=0x7fb078069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0780fea00 0x7fb078069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 -- 172.21.15.93:0/1921893869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 msgr2=0x7fb07819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.739 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 0x7fb07819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 -- 172.21.15.93:0/1921893869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0095f0 con 0x7fb078100580 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07d295700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 0x7fb07819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07e297700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e940 tx=0x7fb07000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700097b0 con 0x7fb078100580 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb07000ee90 con 0x7fb078100580 2023-07-21T22:14:47.740 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07819d860 con 0x7fb078100580 2023-07-21T22:14:47.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070005520 con 0x7fb078100580 2023-07-21T22:14:47.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.690+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07819ddc0 con 0x7fb078100580 2023-07-21T22:14:47.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.691+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078105ea0 con 0x7fb078100580 2023-07-21T22:14:47.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.692+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb070004750 con 0x7fb078100580 2023-07-21T22:14:47.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.693+0000 7fb076ffd700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 0x7fb0600a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:47.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.693+0000 7fb07da96700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 0x7fb0600a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:47.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.693+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb070014070 con 0x7fb078100580 2023-07-21T22:14:47.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.693+0000 7fb07da96700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 0x7fb0600a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0780ffa50 tx=0x7fb068007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:47.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.697+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb0700963f0 con 0x7fb078100580 2023-07-21T22:14:47.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.894+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb07802c850 con 0x7fb078100580 2023-07-21T22:14:47.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.895+0000 7fb076ffd700 1 -- 172.21.15.93:0/1921893869 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb070099a10 con 0x7fb078100580 2023-07-21T22:14:47.938 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:47.938 INFO:teuthology.orchestra.run.smithi093.stdout:[ 2023-07-21T22:14:47.939 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:14:47.939 INFO:teuthology.orchestra.run.smithi093.stdout: "poolnum": 1, 2023-07-21T22:14:47.939 INFO:teuthology.orchestra.run.smithi093.stdout: "poolname": ".mgr" 2023-07-21T22:14:47.939 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:14:47.939 INFO:teuthology.orchestra.run.smithi093.stdout:] 2023-07-21T22:14:47.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 msgr2=0x7fb0600a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 0x7fb0600a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb0780ffa50 tx=0x7fb068007a30 comp rx=0 tx=0).stop 2023-07-21T22:14:47.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 msgr2=0x7fb07819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:47.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb07000e940 tx=0x7fb07000ec50 comp rx=0 tx=0).stop 2023-07-21T22:14:47.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 shutdown_connections 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb078100580 0x7fb07819cab0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0780fea00 0x7fb078069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb0600a1220 0x7fb0600a36d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 --2- 172.21.15.93:0/1921893869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0780ffbf0 0x7fb07819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 >> 172.21.15.93:0/1921893869 conn(0x7fb0780fa000 msgr2=0x7fb078102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:47.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 shutdown_connections 2023-07-21T22:14:47.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:47.897+0000 7fb07fcfa700 1 -- 172.21.15.93:0/1921893869 wait complete. 2023-07-21T22:14:47.953 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-21T22:14:47.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-21T22:14:48.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 -- 172.21.15.93:0/494128619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 msgr2=0x7ff2081000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:48.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 --2- 172.21.15.93:0/494128619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff2081000b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009990 tx=0x7ff1f8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 -- 172.21.15.93:0/494128619 shutdown_connections 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 --2- 172.21.15.93:0/494128619 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 0x7ff208106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 --2- 172.21.15.93:0/494128619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 --2- 172.21.15.93:0/494128619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff2081000b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 -- 172.21.15.93:0/494128619 >> 172.21.15.93:0/494128619 conn(0x7ff2080f97b0 msgr2=0x7ff2080fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 -- 172.21.15.93:0/494128619 shutdown_connections 2023-07-21T22:14:48.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.055+0000 7ff20d135700 1 -- 172.21.15.93:0/494128619 wait complete. 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 Processor -- start 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 -- start start 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080fe610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 0x7ff20819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:48.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20819d350 con 0x7ff2080ffc40 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20819d4c0 con 0x7ff2080fef30 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20d135700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20819d680 con 0x7ff2081005f0 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42048/0 (socket says 172.21.15.93:42048) 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 -- 172.21.15.93:0/1261120966 learned_addr learned my addr 172.21.15.93:0/1261120966 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff206d9d700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:48.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20759e700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 0x7ff20819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 -- 172.21.15.93:0/1261120966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 msgr2=0x7ff20819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 0x7ff20819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 -- 172.21.15.93:0/1261120966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 msgr2=0x7ff2080fe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080fe610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff20659c700 1 -- 172.21.15.93:0/1261120966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80095f0 con 0x7ff2080ffc40 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.056+0000 7ff206d9d700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:14:48.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff20659c700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8004710 tx=0x7ff1f8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f801d070 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20819d900 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1f8021410 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8017650 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.057+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20819dd80 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.058+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20810a1f0 con 0x7ff2080ffc40 2023-07-21T22:14:48.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.059+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff1f8021580 con 0x7ff2080ffc40 2023-07-21T22:14:48.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.059+0000 7ff1effff700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 0x7ff1f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:48.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.059+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff1f80cd790 con 0x7ff2080ffc40 2023-07-21T22:14:48.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.059+0000 7ff206d9d700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 0x7ff1f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:48.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.060+0000 7ff206d9d700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 0x7ff1f00a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2080683d0 tx=0x7ff1fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:48.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.063+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff1f8097410 con 0x7ff2080ffc40 2023-07-21T22:14:48.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:48.260+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff2081058e0 con 0x7ff2080ffc40 2023-07-21T22:14:48.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:48.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3181797590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:48.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1921893869' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:14:48.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:48 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3181797590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1921893869' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:48 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3181797590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1921893869' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:48.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:48 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:49.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.213+0000 7ff1effff700 1 -- 172.21.15.93:0/1261120966 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff1f80218a0 con 0x7ff2080ffc40 2023-07-21T22:14:49.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.217+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 msgr2=0x7ff1f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:49.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.217+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 0x7ff1f00a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2080683d0 tx=0x7ff1fc009260 comp rx=0 tx=0).stop 2023-07-21T22:14:49.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.217+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 msgr2=0x7ff20819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:49.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.217+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8004710 tx=0x7ff1f8009580 comp rx=0 tx=0).stop 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 shutdown_connections 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2081005f0 0x7ff20819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff1f00a1150 0x7ff1f00a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2080fef30 0x7ff2080fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 --2- 172.21.15.93:0/1261120966 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2080ffc40 0x7ff20819a530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 >> 172.21.15.93:0/1261120966 conn(0x7ff2080f97b0 msgr2=0x7ff2080fb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:49.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 shutdown_connections 2023-07-21T22:14:49.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.218+0000 7ff20d135700 1 -- 172.21.15.93:0/1261120966 wait complete. 2023-07-21T22:14:49.262 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' created 2023-07-21T22:14:49.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-21T22:14:49.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 -- 172.21.15.93:0/50557555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 msgr2=0x7f193c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:49.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 --2- 172.21.15.93:0/50557555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c101120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1938009990 tx=0x7f1938009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 -- 172.21.15.93:0/50557555 shutdown_connections 2023-07-21T22:14:49.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 --2- 172.21.15.93:0/50557555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 0x7f193c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 --2- 172.21.15.93:0/50557555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 0x7f193c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 --2- 172.21.15.93:0/50557555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c101120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.364+0000 7f19434b9700 1 -- 172.21.15.93:0/50557555 >> 172.21.15.93:0/50557555 conn(0x7f193c0fb120 msgr2=0x7f193c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- 172.21.15.93:0/50557555 shutdown_connections 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- 172.21.15.93:0/50557555 wait complete. 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 Processor -- start 2023-07-21T22:14:49.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- start start 2023-07-21T22:14:49.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 0x7f193c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:49.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:49.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 0x7f193c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c0735e0 con 0x7f193c100cd0 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c073750 con 0x7f193c101660 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f19434b9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c19f900 con 0x7f193c0ffae0 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f1940a54700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f1940a54700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42062/0 (socket says 172.21.15.93:42062) 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f1940a54700 1 -- 172.21.15.93:0/3988181017 learned_addr learned my addr 172.21.15.93:0/3988181017 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:49.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f1941a56700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 0x7f193c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.365+0000 7f1941255700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 0x7f193c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 -- 172.21.15.93:0/3988181017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 msgr2=0x7f193c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 0x7f193c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 -- 172.21.15.93:0/3988181017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 msgr2=0x7f193c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 0x7f193c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 -- 172.21.15.93:0/3988181017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380095f0 con 0x7f193c100cd0 2023-07-21T22:14:49.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f1940a54700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1938005740 tx=0x7f1938005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193801d070 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c19fb80 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1938021410 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938017650 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.366+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c1a0030 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.367+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c107070 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.368+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1938021580 con 0x7f193c100cd0 2023-07-21T22:14:49.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.368+0000 7f19327fc700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 0x7f19280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:49.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.369+0000 7f1941255700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 0x7f19280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:49.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.369+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f19380cdf60 con 0x7f193c100cd0 2023-07-21T22:14:49.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.369+0000 7f1941255700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 0x7f19280a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f193c100b30 tx=0x7f192c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:49.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.372+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1938097a70 con 0x7f193c100cd0 2023-07-21T22:14:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:49 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:14:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:49.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:49 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:49.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:49.575+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f193c02c850 con 0x7f193c100cd0 2023-07-21T22:14:49.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:49 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:14:49.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:49 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:49.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:49 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:14:49.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:49.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:49.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:49.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:49 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:50.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.221+0000 7f19327fc700 1 -- 172.21.15.93:0/3988181017 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f193809b090 con 0x7f193c100cd0 2023-07-21T22:14:50.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.225+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 msgr2=0x7f19280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:50.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.225+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 0x7f19280a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f193c100b30 tx=0x7f192c009260 comp rx=0 tx=0).stop 2023-07-21T22:14:50.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.225+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 msgr2=0x7f193c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:50.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.225+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1938005740 tx=0x7f1938005770 comp rx=0 tx=0).stop 2023-07-21T22:14:50.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 shutdown_connections 2023-07-21T22:14:50.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f19280a1150 0x7f19280a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f193c101660 0x7f193c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c0ffae0 0x7f193c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 --2- 172.21.15.93:0/3988181017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f193c100cd0 0x7f193c072900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.227+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 >> 172.21.15.93:0/3988181017 conn(0x7f193c0fb120 msgr2=0x7f193c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:50.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.232+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 shutdown_connections 2023-07-21T22:14:50.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.233+0000 7f19434b9700 1 -- 172.21.15.93:0/3988181017 wait complete. 2023-07-21T22:14:50.281 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' created 2023-07-21T22:14:50.292 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-21T22:14:50.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 -- 172.21.15.93:0/3046179129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8114105800 msgr2=0x7f8114107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:50.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 --2- 172.21.15.93:0/3046179129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8114105800 0x7f8114107c60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f810c00b0a0 tx=0x7f810c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 -- 172.21.15.93:0/3046179129 shutdown_connections 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 --2- 172.21.15.93:0/3046179129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8114105800 0x7f8114107c60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 --2- 172.21.15.93:0/3046179129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81141005c0 0x7f81141029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 --2- 172.21.15.93:0/3046179129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114102ee0 0x7f81141052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 -- 172.21.15.93:0/3046179129 >> 172.21.15.93:0/3046179129 conn(0x7f81140fa0a0 msgr2=0x7f81140fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:50.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 -- 172.21.15.93:0/3046179129 shutdown_connections 2023-07-21T22:14:50.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 -- 172.21.15.93:0/3046179129 wait complete. 2023-07-21T22:14:50.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.385+0000 7f811a437700 1 Processor -- start 2023-07-21T22:14:50.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 -- start start 2023-07-21T22:14:50.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:50.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8114102ee0 0x7f8114195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:50.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114105800 0x7f811419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:50.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419d450 con 0x7f81141005c0 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419d590 con 0x7f8114102ee0 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f811a437700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811419d720 con 0x7f8114105800 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42078/0 (socket says 172.21.15.93:42078) 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 -- 172.21.15.93:0/2549037558 learned_addr learned my addr 172.21.15.93:0/2549037558 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f81137fe700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8114102ee0 0x7f8114195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:50.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f81189d4700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114105800 0x7f811419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 -- 172.21.15.93:0/2549037558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114105800 msgr2=0x7f811419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114105800 0x7f811419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 -- 172.21.15.93:0/2549037558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8114102ee0 msgr2=0x7f8114195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8114102ee0 0x7f8114195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 -- 172.21.15.93:0/2549037558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c009d00 con 0x7f81141005c0 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.386+0000 7f8113fff700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:50.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.387+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108009940 con 0x7f81141005c0 2023-07-21T22:14:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.387+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8108010460 con 0x7f81141005c0 2023-07-21T22:14:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.387+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811419da00 con 0x7f81141005c0 2023-07-21T22:14:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.387+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800f620 con 0x7f81141005c0 2023-07-21T22:14:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.387+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811419df10 con 0x7f81141005c0 2023-07-21T22:14:50.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.388+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81140fcc20 con 0x7f81141005c0 2023-07-21T22:14:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.389+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f81080108d0 con 0x7f81141005c0 2023-07-21T22:14:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.389+0000 7f81117fa700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 0x7f80fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:50.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.389+0000 7f81137fe700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 0x7f80fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.390+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f81080cc5f0 con 0x7f81141005c0 2023-07-21T22:14:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.390+0000 7f81137fe700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 0x7f80fc0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8104007be0 tx=0x7f81040073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:50.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.393+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8108095ef0 con 0x7f81141005c0 2023-07-21T22:14:50.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:50 smithi194 ceph-mon[98288]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:50.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:50 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:14:50.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:50 smithi194 ceph-mon[98288]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:14:50.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:50 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:14:50.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:50 smithi134 ceph-mon[97873]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:50.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:50 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:14:50.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:50 smithi134 ceph-mon[97873]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:14:50.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:50 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:14:50.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:50.633+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f811402ca60 con 0x7f81141005c0 2023-07-21T22:14:50.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:50 smithi093 ceph-mon[96579]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:50.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:50 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1261120966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:14:50.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:50 smithi093 ceph-mon[96579]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:14:50.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:50 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:14:51.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.244+0000 7f81117fa700 1 -- 172.21.15.93:0/2549037558 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f81080160c0 con 0x7f81141005c0 2023-07-21T22:14:51.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.251+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 msgr2=0x7f80fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.251+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 0x7f80fc0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8104007be0 tx=0x7f81040073d0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.251+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 msgr2=0x7f81141958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.251+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800dc60 comp rx=0 tx=0).stop 2023-07-21T22:14:51.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 shutdown_connections 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8114105800 0x7f811419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81141005c0 0x7f81141958d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f80fc0a1150 0x7f80fc0a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 --2- 172.21.15.93:0/2549037558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8114102ee0 0x7f8114195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 >> 172.21.15.93:0/2549037558 conn(0x7f81140fa0a0 msgr2=0x7f8114101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.252+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 shutdown_connections 2023-07-21T22:14:51.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.253+0000 7f811a437700 1 -- 172.21.15.93:0/2549037558 wait complete. 2023-07-21T22:14:51.303 INFO:teuthology.orchestra.run.smithi093.stderr:new fs with metadata pool 2 and data pool 3 2023-07-21T22:14:51.313 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:14:51.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 -- 172.21.15.93:0/2623589084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 msgr2=0x7f5b08101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/2623589084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b08101270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009990 tx=0x7f5af4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 -- 172.21.15.93:0/2623589084 shutdown_connections 2023-07-21T22:14:51.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/2623589084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b081017b0 0x7f5b08108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/2623589084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b080fddb0 0x7f5b080fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/2623589084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b08101270 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 -- 172.21.15.93:0/2623589084 >> 172.21.15.93:0/2623589084 conn(0x7f5b080f9770 msgr2=0x7f5b080fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:51.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 -- 172.21.15.93:0/2623589084 shutdown_connections 2023-07-21T22:14:51.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.395+0000 7f5b0ebed700 1 -- 172.21.15.93:0/2623589084 wait complete. 2023-07-21T22:14:51.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 Processor -- start 2023-07-21T22:14:51.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 -- start start 2023-07-21T22:14:51.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b080fddb0 0x7f5b08100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 0x7f5b080ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080ffab0 con 0x7f5b08100e20 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080ffc20 con 0x7f5b081017b0 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b0ebed700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819f820 con 0x7f5b080fddb0 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b07fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b07fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42098/0 (socket says 172.21.15.93:42098) 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.396+0000 7f5b07fff700 1 -- 172.21.15.93:0/3890238678 learned_addr learned my addr 172.21.15.93:0/3890238678 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:51.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b0c989700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b080fddb0 0x7f5b08100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b0d18a700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 0x7f5b080ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 -- 172.21.15.93:0/3890238678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b080fddb0 msgr2=0x7f5b08100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b080fddb0 0x7f5b08100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 -- 172.21.15.93:0/3890238678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 msgr2=0x7f5b080ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 0x7f5b080ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 -- 172.21.15.93:0/3890238678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40095f0 con 0x7f5b08100e20 2023-07-21T22:14:51.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b0d18a700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 0x7f5b080ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b07fff700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af4000c00 tx=0x7f5af400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af401d070 con 0x7f5b08100e20 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0819faa0 con 0x7f5b08100e20 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5af400fb50 con 0x7f5b08100e20 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4017650 con 0x7f5b08100e20 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.397+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0819ffa0 con 0x7f5b08100e20 2023-07-21T22:14:51.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.398+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0810b290 con 0x7f5b08100e20 2023-07-21T22:14:51.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.400+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5af400fcc0 con 0x7f5b08100e20 2023-07-21T22:14:51.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.400+0000 7f5b05ffb700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 0x7f5af80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.401+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5af40cde80 con 0x7f5b08100e20 2023-07-21T22:14:51.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.401+0000 7f5b0c989700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 0x7f5af80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.401+0000 7f5b0c989700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 0x7f5af80a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b080673b0 tx=0x7f5afc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:51.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.404+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5af40977b0 con 0x7f5b08100e20 2023-07-21T22:14:51.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:51 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:14:51.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:51 smithi194 ceph-mon[98288]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:14:51.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:51 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:14:51.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:51 smithi194 ceph-mon[98288]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-21T22:14:51.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:51 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:14:51.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:51 smithi134 ceph-mon[97873]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:14:51.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:51 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:14:51.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:51 smithi134 ceph-mon[97873]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-21T22:14:51.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.598+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b0810b450 con 0x7f5b08100e20 2023-07-21T22:14:51.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.600+0000 7f5b05ffb700 1 -- 172.21.15.93:0/3890238678 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22097 (secure 0 0 0) 0x7f5af409add0 con 0x7f5b08100e20 2023-07-21T22:14:51.643 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:51.643 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":81,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:50.634941+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:51.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 msgr2=0x7f5af80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 0x7f5af80a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b080673b0 tx=0x7f5afc00b410 comp rx=0 tx=0).stop 2023-07-21T22:14:51.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 msgr2=0x7f5b080fedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af4000c00 tx=0x7f5af400f710 comp rx=0 tx=0).stop 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 shutdown_connections 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5af80a11a0 0x7f5af80a3650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b081017b0 0x7f5b080ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b080fddb0 0x7f5b08100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 --2- 172.21.15.93:0/3890238678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b08100e20 0x7f5b080fedd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 >> 172.21.15.93:0/3890238678 conn(0x7f5b080f9770 msgr2=0x7f5b080fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:51.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 shutdown_connections 2023-07-21T22:14:51.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.602+0000 7f5b0ebed700 1 -- 172.21.15.93:0/3890238678 wait complete. 2023-07-21T22:14:51.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:51 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3988181017' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:14:51.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:51 smithi093 ceph-mon[96579]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:14:51.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:51 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:14:51.656 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:51 smithi093 ceph-mon[96579]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-07-21T22:14:51.658 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:14:51.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 -- 172.21.15.93:0/1245477492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2180ffb00 msgr2=0x7fe2180fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/1245477492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2180ffb00 0x7fe2180fff10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe208009990 tx=0x7fe208009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 -- 172.21.15.93:0/1245477492 shutdown_connections 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/1245477492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218101680 0x7fe218107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/1245477492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2180ffb00 0x7fe2180fff10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/1245477492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe218100cf0 0x7fe218101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 -- 172.21.15.93:0/1245477492 >> 172.21.15.93:0/1245477492 conn(0x7fe2180fb0e0 msgr2=0x7fe2180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:51.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 -- 172.21.15.93:0/1245477492 shutdown_connections 2023-07-21T22:14:51.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 -- 172.21.15.93:0/1245477492 wait complete. 2023-07-21T22:14:51.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.735+0000 7fe21ed1f700 1 Processor -- start 2023-07-21T22:14:51.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 -- start start 2023-07-21T22:14:51.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180ffb00 0x7fe2180742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218100cf0 0x7fe218072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180735e0 con 0x7fe218101680 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe218073750 con 0x7fe2180ffb00 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21819f8a0 con 0x7fe218100cf0 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42114/0 (socket says 172.21.15.93:42114) 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 -- 172.21.15.93:0/3749298154 learned_addr learned my addr 172.21.15.93:0/3749298154 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21cabb700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180ffb00 0x7fe2180742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe217fff700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218100cf0 0x7fe218072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 -- 172.21.15.93:0/3749298154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218100cf0 msgr2=0x7fe218072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218100cf0 0x7fe218072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 -- 172.21.15.93:0/3749298154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180ffb00 msgr2=0x7fe2180742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180ffb00 0x7fe2180742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 -- 172.21.15.93:0/3749298154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080095f0 con 0x7fe218101680 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21d2bc700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe210007ee0 tx=0x7fe21000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21000dd20 con 0x7fe218101680 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21000f040 con 0x7fe218101680 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21819fb80 con 0x7fe218101680 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.736+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100048a0 con 0x7fe218101680 2023-07-21T22:14:51.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.737+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2181a0090 con 0x7fe218101680 2023-07-21T22:14:51.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.738+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21810b320 con 0x7fe218101680 2023-07-21T22:14:51.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.739+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe210003a00 con 0x7fe218101680 2023-07-21T22:14:51.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.739+0000 7fe215ffb700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 0x7fe2040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:51.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.739+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2100cbef0 con 0x7fe218101680 2023-07-21T22:14:51.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.739+0000 7fe21cabb700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 0x7fe2040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:51.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.740+0000 7fe21cabb700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 0x7fe2040a36d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b5c0 tx=0x7fe208005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:51.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.743+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe21004e770 con 0x7fe218101680 2023-07-21T22:14:51.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.933+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2181069a0 con 0x7fe218101680 2023-07-21T22:14:51.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.934+0000 7fe215ffb700 1 -- 172.21.15.93:0/3749298154 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22097 (secure 0 0 0) 0x7fe210095790 con 0x7fe218101680 2023-07-21T22:14:51.978 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:51.978 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":81,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:50.634941+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:51.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.936+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 msgr2=0x7fe2040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.936+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 0x7fe2040a36d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b5c0 tx=0x7fe208005fb0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.936+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 msgr2=0x7fe218072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:51.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.936+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe210007ee0 tx=0x7fe21000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 shutdown_connections 2023-07-21T22:14:51.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe218101680 0x7fe218072e40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe2040a1220 0x7fe2040a36d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180ffb00 0x7fe2180742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 --2- 172.21.15.93:0/3749298154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe218100cf0 0x7fe218072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:51.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 >> 172.21.15.93:0/3749298154 conn(0x7fe2180fb0e0 msgr2=0x7fe2180fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:51.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 shutdown_connections 2023-07-21T22:14:51.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:51.937+0000 7fe21ed1f700 1 -- 172.21.15.93:0/3749298154 wait complete. 2023-07-21T22:14:51.992 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-21T22:14:52.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 -- 172.21.15.93:0/2399932564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6744100590 msgr2=0x7f6744106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 --2- 172.21.15.93:0/2399932564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6744100590 0x7f6744106e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b0a0 tx=0x7f673c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 -- 172.21.15.93:0/2399932564 shutdown_connections 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 --2- 172.21.15.93:0/2399932564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6744100590 0x7f6744106e50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 --2- 172.21.15.93:0/2399932564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f67440fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 --2- 172.21.15.93:0/2399932564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67440ffc00 0x7f6744100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 -- 172.21.15.93:0/2399932564 >> 172.21.15.93:0/2399932564 conn(0x7f67440fa030 msgr2=0x7f67440fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:52.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 -- 172.21.15.93:0/2399932564 shutdown_connections 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.073+0000 7f674c1c0700 1 -- 172.21.15.93:0/2399932564 wait complete. 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 Processor -- start 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 -- start start 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:52.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6744100590 0x7f674419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419c3d0 con 0x7f67440ffc00 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419c510 con 0x7f67440fea10 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674c1c0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674419c6a0 con 0x7f6744100590 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674975b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f6749f5c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:52.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674975b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42126/0 (socket says 172.21.15.93:42126) 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f6749f5c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:53286/0 (socket says 172.21.15.93:53286) 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.074+0000 7f674975b700 1 -- 172.21.15.93:0/3675459031 learned_addr learned my addr 172.21.15.93:0/3675459031 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674a75d700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6744100590 0x7f674419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 -- 172.21.15.93:0/3675459031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6744100590 msgr2=0x7f674419be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6744100590 0x7f674419be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 -- 172.21.15.93:0/3675459031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 msgr2=0x7f6744194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:52.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 -- 172.21.15.93:0/3675459031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c009d00 con 0x7f67440ffc00 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f6749f5c700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674975b700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f673400d980 tx=0x7f673400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734004220 con 0x7f67440ffc00 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6734007640 con 0x7f67440ffc00 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674419c980 con 0x7f67440ffc00 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734005610 con 0x7f67440ffc00 2023-07-21T22:14:52.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.075+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674419cee0 con 0x7f67440ffc00 2023-07-21T22:14:52.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.076+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674410a240 con 0x7f67440ffc00 2023-07-21T22:14:52.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.077+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6734004750 con 0x7f67440ffc00 2023-07-21T22:14:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.078+0000 7f673affd700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 0x7f67300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.078+0000 7f6749f5c700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 0x7f67300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.078+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f67340cc570 con 0x7f67440ffc00 2023-07-21T22:14:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.078+0000 7f6749f5c700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 0x7f67300a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67440ffa60 tx=0x7f6740009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:52.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.082+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6734095e10 con 0x7f67440ffc00 2023-07-21T22:14:52.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:52.319+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f67441058a0 con 0x7f67440ffc00 2023-07-21T22:14:52.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: pgmap v250: 97 pgs: 23 creating+peering, 73 unknown, 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:52.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3890238678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3749298154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:active 2023-07-21T22:14:52.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:52 smithi194 ceph-mon[98288]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:52.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: pgmap v250: 97 pgs: 23 creating+peering, 73 unknown, 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3890238678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3749298154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:active 2023-07-21T22:14:52.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:52 smithi134 ceph-mon[97873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:52.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: pgmap v250: 97 pgs: 23 creating+peering, 73 unknown, 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2549037558' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:14:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3890238678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3749298154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] up:active 2023-07-21T22:14:52.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:52 smithi093 ceph-mon[96579]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:53.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.254+0000 7f673affd700 1 -- 172.21.15.93:0/3675459031 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f67340160b0 con 0x7f67440ffc00 2023-07-21T22:14:53.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 msgr2=0x7f67300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:53.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 0x7f67300a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67440ffa60 tx=0x7f6740009260 comp rx=0 tx=0).stop 2023-07-21T22:14:53.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 msgr2=0x7f6744194d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f673400d980 tx=0x7f673400dc90 comp rx=0 tx=0).stop 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 shutdown_connections 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6744100590 0x7f674419be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67300a11d0 0x7f67300a3680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67440fea10 0x7f6744194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 --2- 172.21.15.93:0/3675459031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f67440ffc00 0x7f6744194d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 >> 172.21.15.93:0/3675459031 conn(0x7f67440fa030 msgr2=0x7f67440fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:53.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 shutdown_connections 2023-07-21T22:14:53.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.257+0000 7f674c1c0700 1 -- 172.21.15.93:0/3675459031 wait complete. 2023-07-21T22:14:53.315 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-07-21T22:14:53.316 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-21T22:14:53.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- 172.21.15.93:0/2343116101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c1000a0 msgr2=0x7f652c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 --2- 172.21.15.93:0/2343116101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c1000a0 0x7f652c1004b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f651c009990 tx=0x7f651c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- 172.21.15.93:0/2343116101 shutdown_connections 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 --2- 172.21.15.93:0/2343116101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0ff6a0 0x7f652c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 --2- 172.21.15.93:0/2343116101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c1000a0 0x7f652c1004b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 --2- 172.21.15.93:0/2343116101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c0fecf0 0x7f652c0ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- 172.21.15.93:0/2343116101 >> 172.21.15.93:0/2343116101 conn(0x7f652c0f9850 msgr2=0x7f652c0fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:53.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- 172.21.15.93:0/2343116101 shutdown_connections 2023-07-21T22:14:53.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- 172.21.15.93:0/2343116101 wait complete. 2023-07-21T22:14:53.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 Processor -- start 2023-07-21T22:14:53.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.403+0000 7f6532953700 1 -- start start 2023-07-21T22:14:53.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0fecf0 0x7f652c194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:53.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:53.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c1000a0 0x7f652c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:53.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c19c310 con 0x7f652c0ff6a0 2023-07-21T22:14:53.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c19c450 con 0x7f652c0fecf0 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6532953700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c19c5e0 con 0x7f652c1000a0 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42140/0 (socket says 172.21.15.93:42140) 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 -- 172.21.15.93:0/1890691421 learned_addr learned my addr 172.21.15.93:0/1890691421 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652bfff700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0fecf0 0x7f652c194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:53.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f6530ef0700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c1000a0 0x7f652c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:53.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 -- 172.21.15.93:0/1890691421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c1000a0 msgr2=0x7f652c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:53.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c1000a0 0x7f652c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 -- 172.21.15.93:0/1890691421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0fecf0 msgr2=0x7f652c194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:53.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0fecf0 0x7f652c194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:53.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 -- 172.21.15.93:0/1890691421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0095f0 con 0x7f652c0ff6a0 2023-07-21T22:14:53.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f652b7fe700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f652000d9e0 tx=0x7f652000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:53.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.404+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000eb70 con 0x7f652c0ff6a0 2023-07-21T22:14:53.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.405+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f652000ecd0 con 0x7f652c0ff6a0 2023-07-21T22:14:53.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.405+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c0fdc40 con 0x7f652c0ff6a0 2023-07-21T22:14:53.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.405+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200146c0 con 0x7f652c0ff6a0 2023-07-21T22:14:53.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.405+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c0fe150 con 0x7f652c0ff6a0 2023-07-21T22:14:53.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.405+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c10a2e0 con 0x7f652c0ff6a0 2023-07-21T22:14:53.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.407+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6520014820 con 0x7f652c0ff6a0 2023-07-21T22:14:53.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.407+0000 7f65297fa700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 0x7f65140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:53.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.407+0000 7f652bfff700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 0x7f65140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:53.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.407+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f65200ce460 con 0x7f652c0ff6a0 2023-07-21T22:14:53.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.408+0000 7f652bfff700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 0x7f65140a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f651c00b5c0 tx=0x7f651c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:53.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.410+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6520097d00 con 0x7f652c0ff6a0 2023-07-21T22:14:53.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:53 smithi194 ceph-mon[98288]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:14:53.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:53 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:14:53.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:53 smithi134 ceph-mon[97873]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:14:53.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:53 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:14:53.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:53.643+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f652c105ea0 con 0x7f652c0ff6a0 2023-07-21T22:14:53.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:53 smithi093 ceph-mon[96579]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:14:53.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:53 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:14:54.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.255+0000 7f65297fa700 1 -- 172.21.15.93:0/1890691421 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7f652009b320 con 0x7f652c0ff6a0 2023-07-21T22:14:54.300 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: 2023-07-21T22:14:54.002+0000 7f3e7bb83700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-21T22:14:54.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.257+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 msgr2=0x7f65140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.257+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 0x7f65140a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f651c00b5c0 tx=0x7f651c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.257+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 msgr2=0x7f652c194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.257+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f652000d9e0 tx=0x7f652000dcf0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 shutdown_connections 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f652c1000a0 0x7f652c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65140a1150 0x7f65140a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f652c0fecf0 0x7f652c194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 --2- 172.21.15.93:0/1890691421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f652c0ff6a0 0x7f652c194cd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 >> 172.21.15.93:0/1890691421 conn(0x7f652c0f9850 msgr2=0x7f652c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:54.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 shutdown_connections 2023-07-21T22:14:54.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.258+0000 7f6532953700 1 -- 172.21.15.93:0/1890691421 wait complete. 2023-07-21T22:14:54.316 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-21T22:14:54.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- 172.21.15.93:0/4079214396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc0ffae0 msgr2=0x7f55fc0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 --2- 172.21.15.93:0/4079214396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc0ffae0 0x7f55fc0ffef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009990 tx=0x7f55ec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- 172.21.15.93:0/4079214396 shutdown_connections 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 --2- 172.21.15.93:0/4079214396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 --2- 172.21.15.93:0/4079214396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc0ffae0 0x7f55fc0ffef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 --2- 172.21.15.93:0/4079214396 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc100cd0 0x7f55fc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- 172.21.15.93:0/4079214396 >> 172.21.15.93:0/4079214396 conn(0x7f55fc0fb120 msgr2=0x7f55fc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:54.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- 172.21.15.93:0/4079214396 shutdown_connections 2023-07-21T22:14:54.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- 172.21.15.93:0/4079214396 wait complete. 2023-07-21T22:14:54.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 Processor -- start 2023-07-21T22:14:54.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.398+0000 7f560409c700 1 -- start start 2023-07-21T22:14:54.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 0x7f55fc1958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc19d430 con 0x7f55fc100cd0 2023-07-21T22:14:54.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc19d570 con 0x7f55fc101660 2023-07-21T22:14:54.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc19d700 con 0x7f55fc0ffae0 2023-07-21T22:14:54.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42152/0 (socket says 172.21.15.93:42152) 2023-07-21T22:14:54.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 -- 172.21.15.93:0/4133764289 learned_addr learned my addr 172.21.15.93:0/4133764289 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:54.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601e38700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 0x7f55fc1958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5602639700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 -- 172.21.15.93:0/4133764289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 msgr2=0x7f55fc1958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 0x7f55fc1958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 -- 172.21.15.93:0/4133764289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 msgr2=0x7f55fc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 -- 172.21.15.93:0/4133764289 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0095f0 con 0x7f55fc100cd0 2023-07-21T22:14:54.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601e38700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 0x7f55fc1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:54.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5602639700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:54.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f5601637700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b530 tx=0x7f55f800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8011830 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55f8011e50 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc19d9e0 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800f2e0 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.399+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc19df70 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.400+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc10b380 con 0x7f55fc100cd0 2023-07-21T22:14:54.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.401+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f55f8010580 con 0x7f55fc100cd0 2023-07-21T22:14:54.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.402+0000 7f55f2ffd700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 0x7f55e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.402+0000 7f5601e38700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 0x7f55e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.402+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55f80cc860 con 0x7f55fc100cd0 2023-07-21T22:14:54.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.403+0000 7f5601e38700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 0x7f55e80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b5c0 tx=0x7f55ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:54.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.406+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55f80960e0 con 0x7f55fc100cd0 2023-07-21T22:14:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:54 smithi194 ceph-mon[98288]: pgmap v253: 97 pgs: 29 creating+peering, 22 unknown, 46 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-21T22:14:54.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:54 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:14:54.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:54 smithi194 ceph-mon[98288]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:54.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:54 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:14:54.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:54 smithi134 ceph-mon[97873]: pgmap v253: 97 pgs: 29 creating+peering, 22 unknown, 46 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-21T22:14:54.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:54 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:14:54.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:54 smithi134 ceph-mon[97873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:54.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:54 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:14:54.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:54.638+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f55fc19e5e0 con 0x7f55fc100cd0 2023-07-21T22:14:54.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:54 smithi093 ceph-mon[96579]: pgmap v253: 97 pgs: 29 creating+peering, 22 unknown, 46 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-07-21T22:14:54.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:54 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3675459031' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:14:54.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:54 smithi093 ceph-mon[96579]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:54.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:54 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:14:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.266+0000 7f55f2ffd700 1 -- 172.21.15.93:0/4133764289 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f55f8099700 con 0x7f55fc100cd0 2023-07-21T22:14:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 msgr2=0x7f55e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 0x7f55e80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b5c0 tx=0x7f55ec005fb0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 msgr2=0x7f55fc195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b530 tx=0x7f55f800b8f0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 shutdown_connections 2023-07-21T22:14:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55e80a1150 0x7f55e80a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc101660 0x7f55fc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55fc0ffae0 0x7f55fc1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 --2- 172.21.15.93:0/4133764289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55fc100cd0 0x7f55fc195df0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 >> 172.21.15.93:0/4133764289 conn(0x7f55fc0fb120 msgr2=0x7f55fc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 shutdown_connections 2023-07-21T22:14:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.269+0000 7f560409c700 1 -- 172.21.15.93:0/4133764289 wait complete. 2023-07-21T22:14:55.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-21T22:14:55.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 -- 172.21.15.93:0/2384655704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 msgr2=0x7f2798100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:55.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 --2- 172.21.15.93:0/2384655704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f2798100030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2784009990 tx=0x7f2784009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 -- 172.21.15.93:0/2384655704 shutdown_connections 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 --2- 172.21.15.93:0/2384655704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798100570 0x7f2798106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 --2- 172.21.15.93:0/2384655704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27980fe9f0 0x7f27980fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 --2- 172.21.15.93:0/2384655704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f2798100030 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 -- 172.21.15.93:0/2384655704 >> 172.21.15.93:0/2384655704 conn(0x7f27980fa030 msgr2=0x7f27980fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 -- 172.21.15.93:0/2384655704 shutdown_connections 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.417+0000 7f279ea88700 1 -- 172.21.15.93:0/2384655704 wait complete. 2023-07-21T22:14:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 Processor -- start 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 -- start start 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 0x7f2798069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100570 0x7f279819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819d4e0 con 0x7f27980ffbe0 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819d650 con 0x7f2798100570 2023-07-21T22:14:55.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819d810 con 0x7f27980fe9f0 2023-07-21T22:14:55.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:55.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42166/0 (socket says 172.21.15.93:42166) 2023-07-21T22:14:55.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 -- 172.21.15.93:0/2697038750 learned_addr learned my addr 172.21.15.93:0/2697038750 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:55.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279c824700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 0x7f2798069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:55.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279d025700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100570 0x7f279819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 -- 172.21.15.93:0/2697038750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 msgr2=0x7f2798069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 0x7f2798069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 -- 172.21.15.93:0/2697038750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100570 msgr2=0x7f279819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100570 0x7f279819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 -- 172.21.15.93:0/2697038750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840095f0 con 0x7f27980ffbe0 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279c824700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 0x7f2798069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2797fff700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2784004760 tx=0x7f2784003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278401d070 con 0x7f27980ffbe0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279819da90 con 0x7f27980ffbe0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2784005030 con 0x7f27980ffbe0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.418+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784004ad0 con 0x7f27980ffbe0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.419+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27981946f0 con 0x7f27980ffbe0 2023-07-21T22:14:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.420+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279810a2e0 con 0x7f27980ffbe0 2023-07-21T22:14:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.421+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2784004c30 con 0x7f27980ffbe0 2023-07-21T22:14:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.422+0000 7f2795ffb700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 0x7f27880a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.422+0000 7f279c824700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 0x7f27880a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.422+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f27840cde00 con 0x7f27980ffbe0 2023-07-21T22:14:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.422+0000 7f279c824700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 0x7f27880a7a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27980ffa40 tx=0x7f278c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.425+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2784097690 con 0x7f27980ffbe0 2023-07-21T22:14:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:14:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:14:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:14:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-21T22:14:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-21T22:14:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:55 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:14:55.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:14:55.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:14:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:14:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-21T22:14:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-21T22:14:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:55 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:14:55.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:14:55.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1890691421' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:14:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:14:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-07-21T22:14:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-07-21T22:14:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-07-21T22:14:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:55 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:14:55.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:55.660+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f279810a470 con 0x7f27980ffbe0 2023-07-21T22:14:56.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.271+0000 7f2795ffb700 1 -- 172.21.15.93:0/2697038750 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f278409acb0 con 0x7f27980ffbe0 2023-07-21T22:14:56.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 msgr2=0x7f27880a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:56.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 0x7f27880a7a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f27980ffa40 tx=0x7f278c008040 comp rx=0 tx=0).stop 2023-07-21T22:14:56.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 msgr2=0x7f279819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:56.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2784004760 tx=0x7f2784003940 comp rx=0 tx=0).stop 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 shutdown_connections 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27880a5560 0x7f27880a7a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100570 0x7f279819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27980fe9f0 0x7f2798069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 --2- 172.21.15.93:0/2697038750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27980ffbe0 0x7f279819a6c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 >> 172.21.15.93:0/2697038750 conn(0x7f27980fa030 msgr2=0x7f27980fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:56.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 shutdown_connections 2023-07-21T22:14:56.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.274+0000 7f279ea88700 1 -- 172.21.15.93:0/2697038750 wait complete. 2023-07-21T22:14:56.333 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-07-21T22:14:56.334 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-21T22:14:56.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 -- 172.21.15.93:0/1658506674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b50100d60 msgr2=0x7f9b501011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 --2- 172.21.15.93:0/1658506674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b50100d60 0x7f9b501011b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009990 tx=0x7f9b40009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 -- 172.21.15.93:0/1658506674 shutdown_connections 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 --2- 172.21.15.93:0/1658506674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b501016f0 0x7f9b50107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 --2- 172.21.15.93:0/1658506674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 0x7f9b500fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 --2- 172.21.15.93:0/1658506674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b50100d60 0x7f9b501011b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 -- 172.21.15.93:0/1658506674 >> 172.21.15.93:0/1658506674 conn(0x7f9b500f9770 msgr2=0x7f9b500fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:56.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 -- 172.21.15.93:0/1658506674 shutdown_connections 2023-07-21T22:14:56.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.417+0000 7f9b54906700 1 -- 172.21.15.93:0/1658506674 wait complete. 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 Processor -- start 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 -- start start 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 0x7f9b501006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 0x7f9b500fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:56.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500ff9f0 con 0x7f9b501016f0 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500ffb60 con 0x7f9b50100d60 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b54906700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5019f950 con 0x7f9b500fdd90 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42182/0 (socket says 172.21.15.93:42182) 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 -- 172.21.15.93:0/2131068985 learned_addr learned my addr 172.21.15.93:0/2131068985 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4dd9b700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 0x7f9b500fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:56.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4e59c700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 0x7f9b501006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 -- 172.21.15.93:0/2131068985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 msgr2=0x7f9b501006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 0x7f9b501006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 -- 172.21.15.93:0/2131068985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 msgr2=0x7f9b500fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 0x7f9b500fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 -- 172.21.15.93:0/2131068985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400095f0 con 0x7f9b501016f0 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4dd9b700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 0x7f9b500fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:56.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b4ed9d700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007ee0 tx=0x7f9b4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.418+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48017070 con 0x7f9b501016f0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.419+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b4800f040 con 0x7f9b501016f0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.419+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5019fc30 con 0x7f9b501016f0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.419+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48005740 con 0x7f9b501016f0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.419+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b501a0190 con 0x7f9b501016f0 2023-07-21T22:14:56.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.420+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5010b390 con 0x7f9b501016f0 2023-07-21T22:14:56.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.421+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b480058a0 con 0x7f9b501016f0 2023-07-21T22:14:56.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.421+0000 7f9b3f7fe700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 0x7f9b380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:56.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.421+0000 7f9b4e59c700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 0x7f9b380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:56.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.421+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b480cc600 con 0x7f9b501016f0 2023-07-21T22:14:56.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.422+0000 7f9b4e59c700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 0x7f9b380a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:56.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.425+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b48095ea0 con 0x7f9b501016f0 2023-07-21T22:14:56.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: pgmap v254: 97 pgs: 21 creating+peering, 76 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-07-21T22:14:56.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:active 2023-07-21T22:14:56.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:14:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:14:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-21T22:14:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-21T22:14:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-21T22:14:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:56 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:14:56.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: pgmap v254: 97 pgs: 21 creating+peering, 76 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-07-21T22:14:56.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:active 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-21T22:14:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:56 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:14:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:56.647+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b50060950 con 0x7f9b380a11d0 2023-07-21T22:14:56.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: pgmap v254: 97 pgs: 21 creating+peering, 76 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-07-21T22:14:56.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] up:active 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4133764289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-07-21T22:14:56.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:14:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.023+0000 7f9b3f7fe700 1 -- 172.21.15.93:0/2131068985 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9b50060950 con 0x7f9b380a11d0 2023-07-21T22:14:57.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.025+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 msgr2=0x7f9b380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.025+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 0x7f9b380a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44008040 comp rx=0 tx=0).stop 2023-07-21T22:14:57.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.025+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 msgr2=0x7f9b500ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.025+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b48007ee0 tx=0x7f9b4800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 shutdown_connections 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b501016f0 0x7f9b500ff250 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b500fdd90 0x7f9b501006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9b380a11d0 0x7f9b380a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 --2- 172.21.15.93:0/2131068985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50100d60 0x7f9b500fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 >> 172.21.15.93:0/2131068985 conn(0x7f9b500f9770 msgr2=0x7f9b500fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 shutdown_connections 2023-07-21T22:14:57.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.026+0000 7f9b54906700 1 -- 172.21.15.93:0/2131068985 wait complete. 2023-07-21T22:14:57.080 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:14:56.651+0000 7fecf3a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:14:57.080 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:14:56.651+0000 7fecf3a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:14:57.080 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:14:56.651+0000 7fecf3a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:14:57.081 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:14:56.651+0000 7fecf3a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:14:57.081 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:14:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:14:56.651+0000 7fecf3a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:14:57.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-21T22:14:57.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 -- 172.21.15.93:0/10744630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40fe150 msgr2=0x7f6ee40fe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 --2- 172.21.15.93:0/10744630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40fe150 0x7f6ee40fe560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009990 tx=0x7f6ed4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 -- 172.21.15.93:0/10744630 shutdown_connections 2023-07-21T22:14:57.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 --2- 172.21.15.93:0/10744630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee4106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 --2- 172.21.15.93:0/10744630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40fe150 0x7f6ee40fe560 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 --2- 172.21.15.93:0/10744630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ff340 0x7f6ee40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 -- 172.21.15.93:0/10744630 >> 172.21.15.93:0/10744630 conn(0x7f6ee40f9770 msgr2=0x7f6ee40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 -- 172.21.15.93:0/10744630 shutdown_connections 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.163+0000 7f6ee9370700 1 -- 172.21.15.93:0/10744630 wait complete. 2023-07-21T22:14:57.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 Processor -- start 2023-07-21T22:14:57.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 -- start start 2023-07-21T22:14:57.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40fe150 0x7f6ee4194640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419c1c0 con 0x7f6ee40ff340 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419c300 con 0x7f6ee40ffcd0 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee9370700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee419c490 con 0x7f6ee40fe150 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee27fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee27fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42228/0 (socket says 172.21.15.93:42228) 2023-07-21T22:14:57.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.164+0000 7f6ee27fc700 1 -- 172.21.15.93:0/3844311913 learned_addr learned my addr 172.21.15.93:0/3844311913 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:57.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee37fe700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee2ffd700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40fe150 0x7f6ee4194640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 -- 172.21.15.93:0/3844311913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40fe150 msgr2=0x7f6ee4194640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40fe150 0x7f6ee4194640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 -- 172.21.15.93:0/3844311913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 msgr2=0x7f6ee419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 -- 172.21.15.93:0/3844311913 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40095f0 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee37fe700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee27fc700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c9b0 tx=0x7f6ed800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8007b00 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed8007c60 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee419c770 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8018660 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.165+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee419cc80 con 0x7f6ee40ff340 2023-07-21T22:14:57.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.166+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee4104fe0 con 0x7f6ee40ff340 2023-07-21T22:14:57.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.167+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ed8018880 con 0x7f6ee40ff340 2023-07-21T22:14:57.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.167+0000 7f6ecbfff700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 0x7f6ecc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.168+0000 7f6ee2ffd700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 0x7f6ecc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.168+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ed80ccc00 con 0x7f6ee40ff340 2023-07-21T22:14:57.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.168+0000 7f6ee2ffd700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 0x7f6ecc0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400b5c0 tx=0x7f6ed4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.171+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6ed8096490 con 0x7f6ee40ff340 2023-07-21T22:14:57.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.394+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ee4060950 con 0x7f6ecc0a1150 2023-07-21T22:14:57.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:14:57.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:14:57.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:14:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:14:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: Cluster is now healthy 2023-07-21T22:14:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T22:14:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T22:14:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:57 smithi093 ceph-mon[96579]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T22:14:57.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.399+0000 7f6ecbfff700 1 -- 172.21.15.93:0/3844311913 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6ee4060950 con 0x7f6ecc0a1150 2023-07-21T22:14:57.443 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_0/20362041-3dbd-44e9-a19d-7e7e3951dd2d 2023-07-21T22:14:57.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 msgr2=0x7f6ecc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 0x7f6ecc0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400b5c0 tx=0x7f6ed4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 msgr2=0x7f6ee4194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800c9b0 tx=0x7f6ed800cd70 comp rx=0 tx=0).stop 2023-07-21T22:14:57.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 shutdown_connections 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ecc0a1150 0x7f6ecc0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ee40ffcd0 0x7f6ee419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee40fe150 0x7f6ee4194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 --2- 172.21.15.93:0/3844311913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40ff340 0x7f6ee4194b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 >> 172.21.15.93:0/3844311913 conn(0x7f6ee40f9770 msgr2=0x7f6ee40fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 shutdown_connections 2023-07-21T22:14:57.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.401+0000 7f6ee9370700 1 -- 172.21.15.93:0/3844311913 wait complete. 2023-07-21T22:14:57.456 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-21T22:14:57.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 -- 172.21.15.93:0/2497243636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 msgr2=0x7f9aa00ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/2497243636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa00ff340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009990 tx=0x7f9a94009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 -- 172.21.15.93:0/2497243636 shutdown_connections 2023-07-21T22:14:57.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/2497243636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01005f0 0x7f9aa0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/2497243636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa00ff340 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/2497243636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 0x7f9aa01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 -- 172.21.15.93:0/2497243636 >> 172.21.15.93:0/2497243636 conn(0x7f9aa00f97b0 msgr2=0x7f9aa00fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 -- 172.21.15.93:0/2497243636 shutdown_connections 2023-07-21T22:14:57.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.537+0000 7f9aa6f90700 1 -- 172.21.15.93:0/2497243636 wait complete. 2023-07-21T22:14:57.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 Processor -- start 2023-07-21T22:14:57.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 -- start start 2023-07-21T22:14:57.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 0x7f9aa0194be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01005f0 0x7f9aa019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa019c220 con 0x7f9aa00fef30 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa019c360 con 0x7f9aa00ffc40 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa6f90700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa019c4f0 con 0x7f9aa01005f0 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa4d2c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa4d2c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42240/0 (socket says 172.21.15.93:42240) 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa4d2c700 1 -- 172.21.15.93:0/1863194900 learned_addr learned my addr 172.21.15.93:0/1863194900 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:57.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9a9ffff700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 0x7f9aa0194be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa4d2c700 1 -- 172.21.15.93:0/1863194900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01005f0 msgr2=0x7f9aa019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.538+0000 7f9aa4d2c700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01005f0 0x7f9aa019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa4d2c700 1 -- 172.21.15.93:0/1863194900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 msgr2=0x7f9aa0194be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa4d2c700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 0x7f9aa0194be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa4d2c700 1 -- 172.21.15.93:0/1863194900 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940095f0 con 0x7f9aa00fef30 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa4d2c700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098e0 tx=0x7f9a9400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401d070 con 0x7f9aa00fef30 2023-07-21T22:14:57.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9400fb70 con 0x7f9aa00fef30 2023-07-21T22:14:57.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa019c770 con 0x7f9aa00fef30 2023-07-21T22:14:57.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94017650 con 0x7f9aa00fef30 2023-07-21T22:14:57.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.539+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa019cb90 con 0x7f9aa00fef30 2023-07-21T22:14:57.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.540+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa0105f70 con 0x7f9aa00fef30 2023-07-21T22:14:57.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.541+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9a94022470 con 0x7f9aa00fef30 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-07-21T22:14:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: Cluster is now healthy 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T22:14:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T22:14:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:57 smithi194 ceph-mon[98288]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T22:14:57.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.541+0000 7f9a9dffb700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 0x7f9a900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.542+0000 7f9a9ffff700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 0x7f9a900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.542+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a940cdc90 con 0x7f9aa00fef30 2023-07-21T22:14:57.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.542+0000 7f9a9ffff700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 0x7f9a900a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005950 tx=0x7f9a8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.545+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9a94097520 con 0x7f9aa00fef30 2023-07-21T22:14:57.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2697038750' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-21T22:14:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: Cluster is now healthy 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-07-21T22:14:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:57 smithi134 ceph-mon[97873]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-07-21T22:14:57.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.764+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f9aa00609e0 con 0x7f9a900a11d0 2023-07-21T22:14:57.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.788+0000 7f9a9dffb700 1 -- 172.21.15.93:0/1863194900 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9aa00609e0 con 0x7f9a900a11d0 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.790+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 msgr2=0x7f9a900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.790+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 0x7f9a900a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005950 tx=0x7f9a8c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.790+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 msgr2=0x7f9aa01946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.790+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098e0 tx=0x7f9a9400f740 comp rx=0 tx=0).stop 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 shutdown_connections 2023-07-21T22:14:57.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aa01005f0 0x7f9aa019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9aa00fef30 0x7f9aa01946a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9a900a11d0 0x7f9a900a3680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 --2- 172.21.15.93:0/1863194900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9aa00ffc40 0x7f9aa0194be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 >> 172.21.15.93:0/1863194900 conn(0x7f9aa00f97b0 msgr2=0x7f9aa01030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 shutdown_connections 2023-07-21T22:14:57.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.791+0000 7f9aa6f90700 1 -- 172.21.15.93:0/1863194900 wait complete. 2023-07-21T22:14:57.846 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-21T22:14:57.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.925+0000 7feb95703700 1 -- 172.21.15.93:0/3234320101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb90100d40 msgr2=0x7feb90101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.925+0000 7feb95703700 1 --2- 172.21.15.93:0/3234320101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb90100d40 0x7feb90101190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb80009990 tx=0x7feb80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- 172.21.15.93:0/3234320101 shutdown_connections 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- 172.21.15.93:0/3234320101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb901016d0 0x7feb90107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- 172.21.15.93:0/3234320101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb900ffb50 0x7feb900fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- 172.21.15.93:0/3234320101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb90100d40 0x7feb90101190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- 172.21.15.93:0/3234320101 >> 172.21.15.93:0/3234320101 conn(0x7feb900fb170 msgr2=0x7feb900fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- 172.21.15.93:0/3234320101 shutdown_connections 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- 172.21.15.93:0/3234320101 wait complete. 2023-07-21T22:14:57.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 Processor -- start 2023-07-21T22:14:57.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- start start 2023-07-21T22:14:57.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900ffb50 0x7feb90195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 0x7feb90195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9019d3f0 con 0x7feb901016d0 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9019d530 con 0x7feb90100d40 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.926+0000 7feb95703700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9019d6c0 con 0x7feb900ffb50 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42260/0 (socket says 172.21.15.93:42260) 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 -- 172.21.15.93:0/4194325115 learned_addr learned my addr 172.21.15.93:0/4194325115 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8e7fc700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 0x7feb90195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8effd700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900ffb50 0x7feb90195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 -- 172.21.15.93:0/4194325115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900ffb50 msgr2=0x7feb90195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900ffb50 0x7feb90195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 -- 172.21.15.93:0/4194325115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 msgr2=0x7feb90195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 0x7feb90195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 -- 172.21.15.93:0/4194325115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800095f0 con 0x7feb901016d0 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8e7fc700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 0x7feb90195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:57.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb8f7fe700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feb88007ee0 tx=0x7feb8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88017070 con 0x7feb901016d0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9019d9a0 con 0x7feb901016d0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb8800f040 con 0x7feb901016d0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88005740 con 0x7feb901016d0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.927+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9019de60 con 0x7feb901016d0 2023-07-21T22:14:57.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.928+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb901070e0 con 0x7feb901016d0 2023-07-21T22:14:57.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.929+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feb880058a0 con 0x7feb901016d0 2023-07-21T22:14:57.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.930+0000 7feb77fff700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 0x7feb780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:57.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.930+0000 7feb8effd700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 0x7feb780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:57.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.930+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feb880cc630 con 0x7feb901016d0 2023-07-21T22:14:57.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.930+0000 7feb8effd700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 0x7feb780a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feb84005950 tx=0x7feb840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:57.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:57.933+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feb88095ed0 con 0x7feb901016d0 2023-07-21T22:14:58.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.010+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feb880962a0 con 0x7feb901016d0 2023-07-21T22:14:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.166+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7feb90060950 con 0x7feb780a1150 2023-07-21T22:14:58.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.169+0000 7feb77fff700 1 -- 172.21.15.93:0/4194325115 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7feb90060950 con 0x7feb780a1150 2023-07-21T22:14:58.212 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880 2023-07-21T22:14:58.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 msgr2=0x7feb780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 0x7feb780a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feb84005950 tx=0x7feb840058e0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 msgr2=0x7feb9019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feb88007ee0 tx=0x7feb8800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 shutdown_connections 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb901016d0 0x7feb9019ce20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb900ffb50 0x7feb90195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feb780a1150 0x7feb780a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 --2- 172.21.15.93:0/4194325115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb90100d40 0x7feb90195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 >> 172.21.15.93:0/4194325115 conn(0x7feb900fb170 msgr2=0x7feb90104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:58.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 shutdown_connections 2023-07-21T22:14:58.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.171+0000 7feb95703700 1 -- 172.21.15.93:0/4194325115 wait complete. 2023-07-21T22:14:58.227 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:14:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 -- 172.21.15.93:0/4195193518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 msgr2=0x7f3bbc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 --2- 172.21.15.93:0/4195193518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc100030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009990 tx=0x7f3bac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 -- 172.21.15.93:0/4195193518 shutdown_connections 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 --2- 172.21.15.93:0/4195193518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bbc100570 0x7f3bbc106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 --2- 172.21.15.93:0/4195193518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 0x7f3bbc0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 --2- 172.21.15.93:0/4195193518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc100030 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 -- 172.21.15.93:0/4195193518 >> 172.21.15.93:0/4195193518 conn(0x7f3bbc0f9fd0 msgr2=0x7f3bbc0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 -- 172.21.15.93:0/4195193518 shutdown_connections 2023-07-21T22:14:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.307+0000 7f3bc2929700 1 -- 172.21.15.93:0/4195193518 wait complete. 2023-07-21T22:14:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 Processor -- start 2023-07-21T22:14:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 -- start start 2023-07-21T22:14:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 0x7f3bbc1946c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bbc100570 0x7f3bbc19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19c240 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19c380 con 0x7f3bbc0fe9f0 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bc2929700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19c510 con 0x7f3bbc100570 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42286/0 (socket says 172.21.15.93:42286) 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 -- 172.21.15.93:0/3756420869 learned_addr learned my addr 172.21.15.93:0/3756420869 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:58.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbbfff700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 0x7f3bbc1946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 -- 172.21.15.93:0/3756420869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bbc100570 msgr2=0x7f3bbc19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bbc100570 0x7f3bbc19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 -- 172.21.15.93:0/3756420869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 msgr2=0x7f3bbc1946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 0x7f3bbc1946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.308+0000 7f3bbb7fe700 1 -- 172.21.15.93:0/3756420869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0095f0 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bbb7fe700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004760 tx=0x7f3bac004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:58.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01d070 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bac004cf0 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc19c790 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac005280 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.309+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc19cbe0 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.310+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc105f40 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.311+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3bac0053e0 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.311+0000 7f3bb97fa700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 0x7f3ba40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.312+0000 7f3bbbfff700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 0x7f3ba40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.312+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3bac0cde60 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.312+0000 7f3bbbfff700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 0x7f3ba40a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0ffa40 tx=0x7f3bb0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:58.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.315+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3bac097690 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 732 B/s rd, 6.3 KiB/s wr, 15 op/s 2023-07-21T22:14:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:active 2023-07-21T22:14:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:14:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:14:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:58 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:14:58.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.545+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bbc02cc70 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.546+0000 7f3bb97fa700 1 -- 172.21.15.93:0/3756420869 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f3bac09acb0 con 0x7f3bbc0ffbe0 2023-07-21T22:14:58.589 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:58.589 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:14:58.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 msgr2=0x7f3ba40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 0x7f3ba40a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0ffa40 tx=0x7f3bb0007400 comp rx=0 tx=0).stop 2023-07-21T22:14:58.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 msgr2=0x7f3bbc194c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004760 tx=0x7f3bac004790 comp rx=0 tx=0).stop 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 shutdown_connections 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3bbc100570 0x7f3bbc19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3ba40a1220 0x7f3ba40a36d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bbc0fe9f0 0x7f3bbc1946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 --2- 172.21.15.93:0/3756420869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3bbc0ffbe0 0x7f3bbc194c00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 >> 172.21.15.93:0/3756420869 conn(0x7f3bbc0f9fd0 msgr2=0x7f3bbc103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:58.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 shutdown_connections 2023-07-21T22:14:58.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.548+0000 7f3bc2929700 1 -- 172.21.15.93:0/3756420869 wait complete. 2023-07-21T22:14:58.593 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:14:58.602 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 732 B/s rd, 6.3 KiB/s wr, 15 op/s 2023-07-21T22:14:58.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:58.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:active 2023-07-21T22:14:58.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:14:58.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:14:58.603 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:58 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:14:58.605 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:14:58.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 732 B/s rd, 6.3 KiB/s wr, 15 op/s 2023-07-21T22:14:58.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] up:active 2023-07-21T22:14:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:14:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:14:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:58 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:14:58.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 -- 172.21.15.93:0/2739124192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4480fe9d0 msgr2=0x7ff4480fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 --2- 172.21.15.93:0/2739124192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4480fe9d0 0x7ff4480fede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff438009990 tx=0x7ff438009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 -- 172.21.15.93:0/2739124192 shutdown_connections 2023-07-21T22:14:58.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 --2- 172.21.15.93:0/2739124192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff448100550 0x7ff448106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 --2- 172.21.15.93:0/2739124192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff4480fe9d0 0x7ff4480fede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 --2- 172.21.15.93:0/2739124192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480ffbc0 0x7ff448100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 -- 172.21.15.93:0/2739124192 >> 172.21.15.93:0/2739124192 conn(0x7ff4480f9fd0 msgr2=0x7ff4480fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 -- 172.21.15.93:0/2739124192 shutdown_connections 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.682+0000 7ff44e707700 1 -- 172.21.15.93:0/2739124192 wait complete. 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 Processor -- start 2023-07-21T22:14:58.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 -- start start 2023-07-21T22:14:58.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480fe9d0 0x7ff4481947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4480ffbc0 0x7ff448194d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819c350 con 0x7ff448100550 2023-07-21T22:14:58.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819c490 con 0x7ff4480fe9d0 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44e707700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819c620 con 0x7ff4480ffbc0 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44cca4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44cca4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42302/0 (socket says 172.21.15.93:42302) 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff44cca4700 1 -- 172.21.15.93:0/631983444 learned_addr learned my addr 172.21.15.93:0/631983444 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.683+0000 7ff447fff700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480fe9d0 0x7ff4481947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 -- 172.21.15.93:0/631983444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4480ffbc0 msgr2=0x7ff448194d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4480ffbc0 0x7ff448194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 -- 172.21.15.93:0/631983444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480fe9d0 msgr2=0x7ff4481947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480fe9d0 0x7ff4481947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 -- 172.21.15.93:0/631983444 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380095f0 con 0x7ff448100550 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44cca4700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b970 tx=0x7ff44000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000c930 con 0x7ff448100550 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff440014440 con 0x7ff448100550 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44819c900 con 0x7ff448100550 2023-07-21T22:14:58.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400075d0 con 0x7ff448100550 2023-07-21T22:14:58.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.684+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44819ce10 con 0x7ff448100550 2023-07-21T22:14:58.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.685+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44810a220 con 0x7ff448100550 2023-07-21T22:14:58.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.686+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff4400145b0 con 0x7ff448100550 2023-07-21T22:14:58.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.686+0000 7ff4457fa700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 0x7ff4340a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:58.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.687+0000 7ff447fff700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 0x7ff4340a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:58.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.687+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff440006e40 con 0x7ff448100550 2023-07-21T22:14:58.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.687+0000 7ff447fff700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 0x7ff4340a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b5c0 tx=0x7ff438005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:58.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.691+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff44004e770 con 0x7ff448100550 2023-07-21T22:14:58.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.882+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff44810a410 con 0x7ff448100550 2023-07-21T22:14:58.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.883+0000 7ff4457fa700 1 -- 172.21.15.93:0/631983444 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7ff4400d0070 con 0x7ff448100550 2023-07-21T22:14:58.927 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:58.927 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:14:58.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 msgr2=0x7ff4340a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 0x7ff4340a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b5c0 tx=0x7ff438005960 comp rx=0 tx=0).stop 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 msgr2=0x7ff44819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b970 tx=0x7ff44000bc80 comp rx=0 tx=0).stop 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 shutdown_connections 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff448100550 0x7ff44819bd80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff4340a1220 0x7ff4340a36d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4480fe9d0 0x7ff4481947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 --2- 172.21.15.93:0/631983444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4480ffbc0 0x7ff448194d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:58.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 >> 172.21.15.93:0/631983444 conn(0x7ff4480f9fd0 msgr2=0x7ff4481030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:58.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 shutdown_connections 2023-07-21T22:14:58.930 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:58.885+0000 7ff44e707700 1 -- 172.21.15.93:0/631983444 wait complete. 2023-07-21T22:14:58.941 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-21T22:14:58.942 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:14:59.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/1535657119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c480ffc10 msgr2=0x7f1c48100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/1535657119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c480ffc10 0x7f1c48100060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009990 tx=0x7f1c3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/1535657119 shutdown_connections 2023-07-21T22:14:59.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/1535657119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c481005a0 0x7f1c48106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/1535657119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/1535657119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c480ffc10 0x7f1c48100060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.019+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/1535657119 >> 172.21.15.93:0/1535657119 conn(0x7f1c480fa020 msgr2=0x7f1c480fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:59.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/1535657119 shutdown_connections 2023-07-21T22:14:59.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/1535657119 wait complete. 2023-07-21T22:14:59.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 Processor -- start 2023-07-21T22:14:59.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- start start 2023-07-21T22:14:59.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c480ffc10 0x7f1c4819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4819d260 con 0x7f1c481005a0 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4819d3d0 con 0x7f1c480ffc10 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.020+0000 7f1c4cdfe700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4819d590 con 0x7f1c480fea20 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42316/0 (socket says 172.21.15.93:42316) 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 -- 172.21.15.93:0/3445888720 learned_addr learned my addr 172.21.15.93:0/3445888720 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c45d9b700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c480ffc10 0x7f1c4819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c4659c700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 -- 172.21.15.93:0/3445888720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 msgr2=0x7f1c480694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 -- 172.21.15.93:0/3445888720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c480ffc10 msgr2=0x7f1c4819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c480ffc10 0x7f1c4819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 -- 172.21.15.93:0/3445888720 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0095f0 con 0x7f1c481005a0 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c4659c700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c46d9d700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ef30 tx=0x7f1c4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40018070 con 0x7f1c481005a0 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4819d810 con 0x7f1c481005a0 2023-07-21T22:14:59.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4819dd20 con 0x7f1c481005a0 2023-07-21T22:14:59.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4000f040 con 0x7f1c481005a0 2023-07-21T22:14:59.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.021+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000cd60 con 0x7f1c481005a0 2023-07-21T22:14:59.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.022+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c28005320 con 0x7f1c481005a0 2023-07-21T22:14:59.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.023+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1c40004ad0 con 0x7f1c481005a0 2023-07-21T22:14:59.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.024+0000 7f1c377fe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 0x7f1c300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.024+0000 7f1c4659c700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 0x7f1c300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.024+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c400cc370 con 0x7f1c481005a0 2023-07-21T22:14:59.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.024+0000 7f1c4659c700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 0x7f1c300a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c38005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:59.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.028+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1c40095ba0 con 0x7f1c481005a0 2023-07-21T22:14:59.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.213+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c28000bf0 con 0x7f1c300a1220 2023-07-21T22:14:59.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.223+0000 7f1c377fe700 1 -- 172.21.15.93:0/3445888720 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336321 (secure 0 0 0) 0x7f1c28000bf0 con 0x7f1c300a1220 2023-07-21T22:14:59.272 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:59.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 msgr2=0x7f1c300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 0x7f1c300a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c38005e40 comp rx=0 tx=0).stop 2023-07-21T22:14:59.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 msgr2=0x7f1c4819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ef30 tx=0x7f1c4000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 shutdown_connections 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c481005a0 0x7f1c4819cac0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c480fea20 0x7f1c480694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1c300a1220 0x7f1c300a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 --2- 172.21.15.93:0/3445888720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c480ffc10 0x7f1c4819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 >> 172.21.15.93:0/3445888720 conn(0x7f1c480fa020 msgr2=0x7f1c480fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:59.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 shutdown_connections 2023-07-21T22:14:59.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.227+0000 7f1c4cdfe700 1 -- 172.21.15.93:0/3445888720 wait complete. 2023-07-21T22:14:59.276 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:14:59.283 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-07-21T22:14:59.207436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614292,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":345,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335756,"kb_used_data":12332,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606068,"statfs":{"total":1151940427776,"available":1151596613632,"internally_reserved":0,"allocated":12627968,"data_stored":6893104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23924,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":164,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249025+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:14:52.249025+0000","last_peered":"2023-07-21T22:14:52.249025+0000","last_clean":"2023-07-21T22:14:52.249025+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:14:52.249025+0000","last_undegraded":"2023-07-21T22:14:52.249025+0000","last_fullsized":"2023-07-21T22:14:52.249025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:33.953568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324185+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:14:52.324185+0000","last_peered":"2023-07-21T22:14:52.324185+0000","last_clean":"2023-07-21T22:14:52.324185+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:14:52.324185+0000","last_undegraded":"2023-07-21T22:14:52.324185+0000","last_fullsized":"2023-07-21T22:14:52.324185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:29.855484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323763+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.323763+0000","last_peered":"2023-07-21T22:14:52.323763+0000","last_clean":"2023-07-21T22:14:52.323763+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:14:52.323763+0000","last_undegraded":"2023-07-21T22:14:52.323763+0000","last_fullsized":"2023-07-21T22:14:52.323763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:27.021961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324294+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:14:52.324294+0000","last_peered":"2023-07-21T22:14:52.324294+0000","last_clean":"2023-07-21T22:14:52.324294+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:14:52.324294+0000","last_undegraded":"2023-07-21T22:14:52.324294+0000","last_fullsized":"2023-07-21T22:14:52.324294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:59:03.712988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550902+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:14:53.550902+0000","last_peered":"2023-07-21T22:14:53.550902+0000","last_clean":"2023-07-21T22:14:53.550902+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:14:53.550902+0000","last_undegraded":"2023-07-21T22:14:53.550902+0000","last_fullsized":"2023-07-21T22:14:53.550902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:00.017709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323849+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:14:52.323849+0000","last_peered":"2023-07-21T22:14:52.323849+0000","last_clean":"2023-07-21T22:14:52.323849+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:14:52.323849+0000","last_undegraded":"2023-07-21T22:14:52.323849+0000","last_fullsized":"2023-07-21T22:14:52.323849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:58.487301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323957+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:14:52.323957+0000","last_peered":"2023-07-21T22:14:52.323957+0000","last_clean":"2023-07-21T22:14:52.323957+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:14:52.323957+0000","last_undegraded":"2023-07-21T22:14:52.323957+0000","last_fullsized":"2023-07-21T22:14:52.323957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:02.172487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248538+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:14:52.248538+0000","last_peered":"2023-07-21T22:14:52.248538+0000","last_clean":"2023-07-21T22:14:52.248538+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:14:52.248538+0000","last_undegraded":"2023-07-21T22:14:52.248538+0000","last_fullsized":"2023-07-21T22:14:52.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:54.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249436+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:14:52.249436+0000","last_peered":"2023-07-21T22:14:52.249436+0000","last_clean":"2023-07-21T22:14:52.249436+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:14:52.249436+0000","last_undegraded":"2023-07-21T22:14:52.249436+0000","last_fullsized":"2023-07-21T22:14:52.249436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:33:03.512159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324232+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:14:52.324232+0000","last_peered":"2023-07-21T22:14:52.324232+0000","last_clean":"2023-07-21T22:14:52.324232+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:14:52.324232+0000","last_undegraded":"2023-07-21T22:14:52.324232+0000","last_fullsized":"2023-07-21T22:14:52.324232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:33.917374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324415+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:14:52.324415+0000","last_peered":"2023-07-21T22:14:52.324415+0000","last_clean":"2023-07-21T22:14:52.324415+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:14:52.324415+0000","last_undegraded":"2023-07-21T22:14:52.324415+0000","last_fullsized":"2023-07-21T22:14:52.324415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:42.648086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324156+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.324156+0000","last_peered":"2023-07-21T22:14:52.324156+0000","last_clean":"2023-07-21T22:14:52.324156+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:14:52.324156+0000","last_undegraded":"2023-07-21T22:14:52.324156+0000","last_fullsized":"2023-07-21T22:14:52.324156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:12.914647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323062+0000","last_change":"2023-07-21T22:14:51.245228+0000","last_active":"2023-07-21T22:14:52.323062+0000","last_peered":"2023-07-21T22:14:52.323062+0000","last_clean":"2023-07-21T22:14:52.323062+0000","last_became_active":"2023-07-21T22:14:51.244919+0000","last_became_peered":"2023-07-21T22:14:51.244919+0000","last_unstale":"2023-07-21T22:14:52.323062+0000","last_undegraded":"2023-07-21T22:14:52.323062+0000","last_fullsized":"2023-07-21T22:14:52.323062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:21:11.471385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323500+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:14:52.323500+0000","last_peered":"2023-07-21T22:14:52.323500+0000","last_clean":"2023-07-21T22:14:52.323500+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:14:52.323500+0000","last_undegraded":"2023-07-21T22:14:52.323500+0000","last_fullsized":"2023-07-21T22:14:52.323500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:05:33.532014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323093+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:14:52.323093+0000","last_peered":"2023-07-21T22:14:52.323093+0000","last_clean":"2023-07-21T22:14:52.323093+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:14:52.323093+0000","last_undegraded":"2023-07-21T22:14:52.323093+0000","last_fullsized":"2023-07-21T22:14:52.323093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:57.567468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248840+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:14:52.248840+0000","last_peered":"2023-07-21T22:14:52.248840+0000","last_clean":"2023-07-21T22:14:52.248840+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:14:52.248840+0000","last_undegraded":"2023-07-21T22:14:52.248840+0000","last_fullsized":"2023-07-21T22:14:52.248840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:28:05.983432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249184+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:14:52.249184+0000","last_peered":"2023-07-21T22:14:52.249184+0000","last_clean":"2023-07-21T22:14:52.249184+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:14:52.249184+0000","last_undegraded":"2023-07-21T22:14:52.249184+0000","last_fullsized":"2023-07-21T22:14:52.249184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:35.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323352+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:14:52.323352+0000","last_peered":"2023-07-21T22:14:52.323352+0000","last_clean":"2023-07-21T22:14:52.323352+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:14:52.323352+0000","last_undegraded":"2023-07-21T22:14:52.323352+0000","last_fullsized":"2023-07-21T22:14:52.323352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:31.713111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323930+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:14:52.323930+0000","last_peered":"2023-07-21T22:14:52.323930+0000","last_clean":"2023-07-21T22:14:52.323930+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:14:52.323930+0000","last_undegraded":"2023-07-21T22:14:52.323930+0000","last_fullsized":"2023-07-21T22:14:52.323930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:45.482633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002755+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:14:57.002755+0000","last_peered":"2023-07-21T22:14:57.002755+0000","last_clean":"2023-07-21T22:14:57.002755+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:14:57.002755+0000","last_undegraded":"2023-07-21T22:14:57.002755+0000","last_fullsized":"2023-07-21T22:14:57.002755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:42.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248992+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:14:52.248992+0000","last_peered":"2023-07-21T22:14:52.248992+0000","last_clean":"2023-07-21T22:14:52.248992+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:14:52.248992+0000","last_undegraded":"2023-07-21T22:14:52.248992+0000","last_fullsized":"2023-07-21T22:14:52.248992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:57.089271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249058+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:14:52.249058+0000","last_peered":"2023-07-21T22:14:52.249058+0000","last_clean":"2023-07-21T22:14:52.249058+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:14:52.249058+0000","last_undegraded":"2023-07-21T22:14:52.249058+0000","last_fullsized":"2023-07-21T22:14:52.249058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:17.931445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324904+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:14:52.324904+0000","last_peered":"2023-07-21T22:14:52.324904+0000","last_clean":"2023-07-21T22:14:52.324904+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:14:52.324904+0000","last_undegraded":"2023-07-21T22:14:52.324904+0000","last_fullsized":"2023-07-21T22:14:52.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:07.388393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285630+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:14:56.285630+0000","last_peered":"2023-07-21T22:14:56.285630+0000","last_clean":"2023-07-21T22:14:56.285630+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:14:56.285630+0000","last_undegraded":"2023-07-21T22:14:56.285630+0000","last_fullsized":"2023-07-21T22:14:56.285630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:45.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248983+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:14:52.248983+0000","last_peered":"2023-07-21T22:14:52.248983+0000","last_clean":"2023-07-21T22:14:52.248983+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:14:52.248983+0000","last_undegraded":"2023-07-21T22:14:52.248983+0000","last_fullsized":"2023-07-21T22:14:52.248983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:33.576139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551119+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:14:53.551119+0000","last_peered":"2023-07-21T22:14:53.551119+0000","last_clean":"2023-07-21T22:14:53.551119+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:14:53.551119+0000","last_undegraded":"2023-07-21T22:14:53.551119+0000","last_fullsized":"2023-07-21T22:14:53.551119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:04:03.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324394+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.324394+0000","last_peered":"2023-07-21T22:14:52.324394+0000","last_clean":"2023-07-21T22:14:52.324394+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:14:52.324394+0000","last_undegraded":"2023-07-21T22:14:52.324394+0000","last_fullsized":"2023-07-21T22:14:52.324394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:10:13.245800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.999086+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:14:56.999086+0000","last_peered":"2023-07-21T22:14:56.999086+0000","last_clean":"2023-07-21T22:14:56.999086+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:14:56.999086+0000","last_undegraded":"2023-07-21T22:14:56.999086+0000","last_fullsized":"2023-07-21T22:14:56.999086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:15:58.146646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551008+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:14:53.551008+0000","last_peered":"2023-07-21T22:14:53.551008+0000","last_clean":"2023-07-21T22:14:53.551008+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:14:53.551008+0000","last_undegraded":"2023-07-21T22:14:53.551008+0000","last_fullsized":"2023-07-21T22:14:53.551008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:04.181744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285879+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:14:56.285879+0000","last_peered":"2023-07-21T22:14:56.285879+0000","last_clean":"2023-07-21T22:14:56.285879+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:14:56.285879+0000","last_undegraded":"2023-07-21T22:14:56.285879+0000","last_fullsized":"2023-07-21T22:14:56.285879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:46:03.474285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324013+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:14:52.324013+0000","last_peered":"2023-07-21T22:14:52.324013+0000","last_clean":"2023-07-21T22:14:52.324013+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:14:52.324013+0000","last_undegraded":"2023-07-21T22:14:52.324013+0000","last_fullsized":"2023-07-21T22:14:52.324013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:15:10.615938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323388+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:14:52.323388+0000","last_peered":"2023-07-21T22:14:52.323388+0000","last_clean":"2023-07-21T22:14:52.323388+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:14:52.323388+0000","last_undegraded":"2023-07-21T22:14:52.323388+0000","last_fullsized":"2023-07-21T22:14:52.323388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:01.057949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248598+0000","last_change":"2023-07-21T22:14:51.248592+0000","last_active":"2023-07-21T22:14:52.248598+0000","last_peered":"2023-07-21T22:14:52.248598+0000","last_clean":"2023-07-21T22:14:52.248598+0000","last_became_active":"2023-07-21T22:14:51.247932+0000","last_became_peered":"2023-07-21T22:14:51.247932+0000","last_unstale":"2023-07-21T22:14:52.248598+0000","last_undegraded":"2023-07-21T22:14:52.248598+0000","last_fullsized":"2023-07-21T22:14:52.248598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:01.815385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001359+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:14:57.001359+0000","last_peered":"2023-07-21T22:14:57.001359+0000","last_clean":"2023-07-21T22:14:57.001359+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:14:57.001359+0000","last_undegraded":"2023-07-21T22:14:57.001359+0000","last_fullsized":"2023-07-21T22:14:57.001359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:40.285840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322887+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:14:52.322887+0000","last_peered":"2023-07-21T22:14:52.322887+0000","last_clean":"2023-07-21T22:14:52.322887+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:14:52.322887+0000","last_undegraded":"2023-07-21T22:14:52.322887+0000","last_fullsized":"2023-07-21T22:14:52.322887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:55:36.899847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.998483+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:14:56.998483+0000","last_peered":"2023-07-21T22:14:56.998483+0000","last_clean":"2023-07-21T22:14:56.998483+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:14:56.998483+0000","last_undegraded":"2023-07-21T22:14:56.998483+0000","last_fullsized":"2023-07-21T22:14:56.998483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:14.250309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249338+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:14:52.249338+0000","last_peered":"2023-07-21T22:14:52.249338+0000","last_clean":"2023-07-21T22:14:52.249338+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:14:52.249338+0000","last_undegraded":"2023-07-21T22:14:52.249338+0000","last_fullsized":"2023-07-21T22:14:52.249338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:06.815314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.271552+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:14:54.271552+0000","last_peered":"2023-07-21T22:14:54.271552+0000","last_clean":"2023-07-21T22:14:54.271552+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:14:54.271552+0000","last_undegraded":"2023-07-21T22:14:54.271552+0000","last_fullsized":"2023-07-21T22:14:54.271552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:38:48.158170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324814+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:14:52.324814+0000","last_peered":"2023-07-21T22:14:52.324814+0000","last_clean":"2023-07-21T22:14:52.324814+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:14:52.324814+0000","last_undegraded":"2023-07-21T22:14:52.324814+0000","last_fullsized":"2023-07-21T22:14:52.324814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:53:48.638340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284307+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:14:56.284307+0000","last_peered":"2023-07-21T22:14:56.284307+0000","last_clean":"2023-07-21T22:14:56.284307+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:14:56.284307+0000","last_undegraded":"2023-07-21T22:14:56.284307+0000","last_fullsized":"2023-07-21T22:14:56.284307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:15.490891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249055+0000","last_change":"2023-07-21T22:14:51.249768+0000","last_active":"2023-07-21T22:14:52.249055+0000","last_peered":"2023-07-21T22:14:52.249055+0000","last_clean":"2023-07-21T22:14:52.249055+0000","last_became_active":"2023-07-21T22:14:51.249503+0000","last_became_peered":"2023-07-21T22:14:51.249503+0000","last_unstale":"2023-07-21T22:14:52.249055+0000","last_undegraded":"2023-07-21T22:14:52.249055+0000","last_fullsized":"2023-07-21T22:14:52.249055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:39:15.294044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'16","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.017000+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:14:57.017000+0000","last_peered":"2023-07-21T22:14:57.017000+0000","last_clean":"2023-07-21T22:14:57.017000+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:14:57.017000+0000","last_undegraded":"2023-07-21T22:14:57.017000+0000","last_fullsized":"2023-07-21T22:14:57.017000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:38.675168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551147+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:14:53.551147+0000","last_peered":"2023-07-21T22:14:53.551147+0000","last_clean":"2023-07-21T22:14:53.551147+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:14:53.551147+0000","last_undegraded":"2023-07-21T22:14:53.551147+0000","last_fullsized":"2023-07-21T22:14:53.551147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:51.431054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006578+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:14:57.006578+0000","last_peered":"2023-07-21T22:14:57.006578+0000","last_clean":"2023-07-21T22:14:57.006578+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:14:57.006578+0000","last_undegraded":"2023-07-21T22:14:57.006578+0000","last_fullsized":"2023-07-21T22:14:57.006578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:55.930555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323112+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:14:52.323112+0000","last_peered":"2023-07-21T22:14:52.323112+0000","last_clean":"2023-07-21T22:14:52.323112+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:14:52.323112+0000","last_undegraded":"2023-07-21T22:14:52.323112+0000","last_fullsized":"2023-07-21T22:14:52.323112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:57:57.488142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284858+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:14:56.284858+0000","last_peered":"2023-07-21T22:14:56.284858+0000","last_clean":"2023-07-21T22:14:56.284858+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:14:56.284858+0000","last_undegraded":"2023-07-21T22:14:56.284858+0000","last_fullsized":"2023-07-21T22:14:56.284858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:28.876044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324774+0000","last_change":"2023-07-21T22:14:51.258940+0000","last_active":"2023-07-21T22:14:52.324774+0000","last_peered":"2023-07-21T22:14:52.324774+0000","last_clean":"2023-07-21T22:14:52.324774+0000","last_became_active":"2023-07-21T22:14:51.258595+0000","last_became_peered":"2023-07-21T22:14:51.258595+0000","last_unstale":"2023-07-21T22:14:52.324774+0000","last_undegraded":"2023-07-21T22:14:52.324774+0000","last_fullsized":"2023-07-21T22:14:52.324774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:39.297921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284308+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:14:56.284308+0000","last_peered":"2023-07-21T22:14:56.284308+0000","last_clean":"2023-07-21T22:14:56.284308+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:14:56.284308+0000","last_undegraded":"2023-07-21T22:14:56.284308+0000","last_fullsized":"2023-07-21T22:14:56.284308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:39.148790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323808+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:14:52.323808+0000","last_peered":"2023-07-21T22:14:52.323808+0000","last_clean":"2023-07-21T22:14:52.323808+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:14:52.323808+0000","last_undegraded":"2023-07-21T22:14:52.323808+0000","last_fullsized":"2023-07-21T22:14:52.323808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:38.761579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'16","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.280888+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:14:55.280888+0000","last_peered":"2023-07-21T22:14:55.280888+0000","last_clean":"2023-07-21T22:14:55.280888+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:14:55.280888+0000","last_undegraded":"2023-07-21T22:14:55.280888+0000","last_fullsized":"2023-07-21T22:14:55.280888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:01:49.310753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323584+0000","last_change":"2023-07-21T22:14:51.252408+0000","last_active":"2023-07-21T22:14:52.323584+0000","last_peered":"2023-07-21T22:14:52.323584+0000","last_clean":"2023-07-21T22:14:52.323584+0000","last_became_active":"2023-07-21T22:14:51.252074+0000","last_became_peered":"2023-07-21T22:14:51.252074+0000","last_unstale":"2023-07-21T22:14:52.323584+0000","last_undegraded":"2023-07-21T22:14:52.323584+0000","last_fullsized":"2023-07-21T22:14:52.323584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.877759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551338+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:14:53.551338+0000","last_peered":"2023-07-21T22:14:53.551338+0000","last_clean":"2023-07-21T22:14:53.551338+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:14:53.551338+0000","last_undegraded":"2023-07-21T22:14:53.551338+0000","last_fullsized":"2023-07-21T22:14:53.551338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:36.286445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249168+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:14:52.249168+0000","last_peered":"2023-07-21T22:14:52.249168+0000","last_clean":"2023-07-21T22:14:52.249168+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:14:52.249168+0000","last_undegraded":"2023-07-21T22:14:52.249168+0000","last_fullsized":"2023-07-21T22:14:52.249168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:12.075583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.101685+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:14:53.101685+0000","last_peered":"2023-07-21T22:14:53.101685+0000","last_clean":"2023-07-21T22:14:53.101685+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:14:53.101685+0000","last_undegraded":"2023-07-21T22:14:53.101685+0000","last_fullsized":"2023-07-21T22:14:53.101685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:03.657128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.530952+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:14:52.530952+0000","last_peered":"2023-07-21T22:14:52.530952+0000","last_clean":"2023-07-21T22:14:52.530952+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:14:52.530952+0000","last_undegraded":"2023-07-21T22:14:52.530952+0000","last_fullsized":"2023-07-21T22:14:52.530952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:11:20.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324770+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:52.324770+0000","last_peered":"2023-07-21T22:14:52.324770+0000","last_clean":"2023-07-21T22:14:52.324770+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:14:52.324770+0000","last_undegraded":"2023-07-21T22:14:52.324770+0000","last_fullsized":"2023-07-21T22:14:52.324770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:37.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324083+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:14:52.324083+0000","last_peered":"2023-07-21T22:14:52.324083+0000","last_clean":"2023-07-21T22:14:52.324083+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:14:52.324083+0000","last_undegraded":"2023-07-21T22:14:52.324083+0000","last_fullsized":"2023-07-21T22:14:52.324083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:57:09.279672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323831+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:14:52.323831+0000","last_peered":"2023-07-21T22:14:52.323831+0000","last_clean":"2023-07-21T22:14:52.323831+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:14:52.323831+0000","last_undegraded":"2023-07-21T22:14:52.323831+0000","last_fullsized":"2023-07-21T22:14:52.323831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:31.048621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323313+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:14:52.323313+0000","last_peered":"2023-07-21T22:14:52.323313+0000","last_clean":"2023-07-21T22:14:52.323313+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:14:52.323313+0000","last_undegraded":"2023-07-21T22:14:52.323313+0000","last_fullsized":"2023-07-21T22:14:52.323313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:39:31.032230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248803+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:14:52.248803+0000","last_peered":"2023-07-21T22:14:52.248803+0000","last_clean":"2023-07-21T22:14:52.248803+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:14:52.248803+0000","last_undegraded":"2023-07-21T22:14:52.248803+0000","last_fullsized":"2023-07-21T22:14:52.248803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:06.145398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284531+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:56.284531+0000","last_peered":"2023-07-21T22:14:56.284531+0000","last_clean":"2023-07-21T22:14:56.284531+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:14:56.284531+0000","last_undegraded":"2023-07-21T22:14:56.284531+0000","last_fullsized":"2023-07-21T22:14:56.284531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:48.946604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323556+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:14:52.323556+0000","last_peered":"2023-07-21T22:14:52.323556+0000","last_clean":"2023-07-21T22:14:52.323556+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:14:52.323556+0000","last_undegraded":"2023-07-21T22:14:52.323556+0000","last_fullsized":"2023-07-21T22:14:52.323556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:04.146166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323206+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:14:52.323206+0000","last_peered":"2023-07-21T22:14:52.323206+0000","last_clean":"2023-07-21T22:14:52.323206+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:14:52.323206+0000","last_undegraded":"2023-07-21T22:14:52.323206+0000","last_fullsized":"2023-07-21T22:14:52.323206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:14:26.470268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323462+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:14:52.323462+0000","last_peered":"2023-07-21T22:14:52.323462+0000","last_clean":"2023-07-21T22:14:52.323462+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:14:52.323462+0000","last_undegraded":"2023-07-21T22:14:52.323462+0000","last_fullsized":"2023-07-21T22:14:52.323462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:41.327520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284749+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:14:56.284749+0000","last_peered":"2023-07-21T22:14:56.284749+0000","last_clean":"2023-07-21T22:14:56.284749+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:14:56.284749+0000","last_undegraded":"2023-07-21T22:14:56.284749+0000","last_fullsized":"2023-07-21T22:14:56.284749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:00.386436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323125+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:14:52.323125+0000","last_peered":"2023-07-21T22:14:52.323125+0000","last_clean":"2023-07-21T22:14:52.323125+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:14:52.323125+0000","last_undegraded":"2023-07-21T22:14:52.323125+0000","last_fullsized":"2023-07-21T22:14:52.323125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:25.036430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285167+0000","last_change":"2023-07-21T22:14:50.231962+0000","last_active":"2023-07-21T22:14:56.285167+0000","last_peered":"2023-07-21T22:14:56.285167+0000","last_clean":"2023-07-21T22:14:56.285167+0000","last_became_active":"2023-07-21T22:14:50.231638+0000","last_became_peered":"2023-07-21T22:14:50.231638+0000","last_unstale":"2023-07-21T22:14:56.285167+0000","last_undegraded":"2023-07-21T22:14:56.285167+0000","last_fullsized":"2023-07-21T22:14:56.285167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:18.134292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249093+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:14:52.249093+0000","last_peered":"2023-07-21T22:14:52.249093+0000","last_clean":"2023-07-21T22:14:52.249093+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:14:52.249093+0000","last_undegraded":"2023-07-21T22:14:52.249093+0000","last_fullsized":"2023-07-21T22:14:52.249093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:15.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270970+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:14:54.270970+0000","last_peered":"2023-07-21T22:14:54.270970+0000","last_clean":"2023-07-21T22:14:54.270970+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:14:54.270970+0000","last_undegraded":"2023-07-21T22:14:54.270970+0000","last_fullsized":"2023-07-21T22:14:54.270970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:43.076462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324274+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:14:52.324274+0000","last_peered":"2023-07-21T22:14:52.324274+0000","last_clean":"2023-07-21T22:14:52.324274+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:14:52.324274+0000","last_undegraded":"2023-07-21T22:14:52.324274+0000","last_fullsized":"2023-07-21T22:14:52.324274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:16.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270269+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:14:54.270269+0000","last_peered":"2023-07-21T22:14:54.270269+0000","last_clean":"2023-07-21T22:14:54.270269+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:14:54.270269+0000","last_undegraded":"2023-07-21T22:14:54.270269+0000","last_fullsized":"2023-07-21T22:14:54.270269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:06:47.274452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550789+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:14:53.550789+0000","last_peered":"2023-07-21T22:14:53.550789+0000","last_clean":"2023-07-21T22:14:53.550789+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:14:53.550789+0000","last_undegraded":"2023-07-21T22:14:53.550789+0000","last_fullsized":"2023-07-21T22:14:53.550789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:02:40.632894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270839+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:14:54.270839+0000","last_peered":"2023-07-21T22:14:54.270839+0000","last_clean":"2023-07-21T22:14:54.270839+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:14:54.270839+0000","last_undegraded":"2023-07-21T22:14:54.270839+0000","last_fullsized":"2023-07-21T22:14:54.270839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:31.455547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248754+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:14:52.248754+0000","last_peered":"2023-07-21T22:14:52.248754+0000","last_clean":"2023-07-21T22:14:52.248754+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:14:52.248754+0000","last_undegraded":"2023-07-21T22:14:52.248754+0000","last_fullsized":"2023-07-21T22:14:52.248754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:25.484174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.867405+0000","last_change":"2023-07-21T22:14:50.229923+0000","last_active":"2023-07-21T22:14:52.867405+0000","last_peered":"2023-07-21T22:14:52.867405+0000","last_clean":"2023-07-21T22:14:52.867405+0000","last_became_active":"2023-07-21T22:14:50.229645+0000","last_became_peered":"2023-07-21T22:14:50.229645+0000","last_unstale":"2023-07-21T22:14:52.867405+0000","last_undegraded":"2023-07-21T22:14:52.867405+0000","last_fullsized":"2023-07-21T22:14:52.867405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:36:56.239739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248731+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:14:52.248731+0000","last_peered":"2023-07-21T22:14:52.248731+0000","last_clean":"2023-07-21T22:14:52.248731+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:14:52.248731+0000","last_undegraded":"2023-07-21T22:14:52.248731+0000","last_fullsized":"2023-07-21T22:14:52.248731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:20.211619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.269986+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:14:54.269986+0000","last_peered":"2023-07-21T22:14:54.269986+0000","last_clean":"2023-07-21T22:14:54.269986+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:14:54.269986+0000","last_undegraded":"2023-07-21T22:14:54.269986+0000","last_fullsized":"2023-07-21T22:14:54.269986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:01.500609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324713+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:14:52.324713+0000","last_peered":"2023-07-21T22:14:52.324713+0000","last_clean":"2023-07-21T22:14:52.324713+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:14:52.324713+0000","last_undegraded":"2023-07-21T22:14:52.324713+0000","last_fullsized":"2023-07-21T22:14:52.324713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:17.553733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001407+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:14:57.001407+0000","last_peered":"2023-07-21T22:14:57.001407+0000","last_clean":"2023-07-21T22:14:57.001407+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:14:57.001407+0000","last_undegraded":"2023-07-21T22:14:57.001407+0000","last_fullsized":"2023-07-21T22:14:57.001407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:15.167223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324532+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:14:52.324532+0000","last_peered":"2023-07-21T22:14:52.324532+0000","last_clean":"2023-07-21T22:14:52.324532+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:14:52.324532+0000","last_undegraded":"2023-07-21T22:14:52.324532+0000","last_fullsized":"2023-07-21T22:14:52.324532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:33.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996861+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:14:56.996861+0000","last_peered":"2023-07-21T22:14:56.996861+0000","last_clean":"2023-07-21T22:14:56.996861+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:14:56.996861+0000","last_undegraded":"2023-07-21T22:14:56.996861+0000","last_fullsized":"2023-07-21T22:14:56.996861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.897090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324289+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:14:52.324289+0000","last_peered":"2023-07-21T22:14:52.324289+0000","last_clean":"2023-07-21T22:14:52.324289+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:14:52.324289+0000","last_undegraded":"2023-07-21T22:14:52.324289+0000","last_fullsized":"2023-07-21T22:14:52.324289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:52.794435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323230+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:14:52.323230+0000","last_peered":"2023-07-21T22:14:52.323230+0000","last_clean":"2023-07-21T22:14:52.323230+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:14:52.323230+0000","last_undegraded":"2023-07-21T22:14:52.323230+0000","last_fullsized":"2023-07-21T22:14:52.323230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:56.943909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324655+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:14:52.324655+0000","last_peered":"2023-07-21T22:14:52.324655+0000","last_clean":"2023-07-21T22:14:52.324655+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:14:52.324655+0000","last_undegraded":"2023-07-21T22:14:52.324655+0000","last_fullsized":"2023-07-21T22:14:52.324655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:05:57.078423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248843+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:14:52.248843+0000","last_peered":"2023-07-21T22:14:52.248843+0000","last_clean":"2023-07-21T22:14:52.248843+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:14:52.248843+0000","last_undegraded":"2023-07-21T22:14:52.248843+0000","last_fullsized":"2023-07-21T22:14:52.248843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:49:10.848487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324305+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:14:52.324305+0000","last_peered":"2023-07-21T22:14:52.324305+0000","last_clean":"2023-07-21T22:14:52.324305+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:14:52.324305+0000","last_undegraded":"2023-07-21T22:14:52.324305+0000","last_fullsized":"2023-07-21T22:14:52.324305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:16:56.773704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323602+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:14:52.323602+0000","last_peered":"2023-07-21T22:14:52.323602+0000","last_clean":"2023-07-21T22:14:52.323602+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:14:52.323602+0000","last_undegraded":"2023-07-21T22:14:52.323602+0000","last_fullsized":"2023-07-21T22:14:52.323602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:58.733025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324433+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:14:52.324433+0000","last_peered":"2023-07-21T22:14:52.324433+0000","last_clean":"2023-07-21T22:14:52.324433+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:14:52.324433+0000","last_undegraded":"2023-07-21T22:14:52.324433+0000","last_fullsized":"2023-07-21T22:14:52.324433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:51.861062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322982+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:14:52.322982+0000","last_peered":"2023-07-21T22:14:52.322982+0000","last_clean":"2023-07-21T22:14:52.322982+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:14:52.322982+0000","last_undegraded":"2023-07-21T22:14:52.322982+0000","last_fullsized":"2023-07-21T22:14:52.322982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:29.650653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248998+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:14:52.248998+0000","last_peered":"2023-07-21T22:14:52.248998+0000","last_clean":"2023-07-21T22:14:52.248998+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:14:52.248998+0000","last_undegraded":"2023-07-21T22:14:52.248998+0000","last_fullsized":"2023-07-21T22:14:52.248998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:30:39.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324003+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:14:52.324003+0000","last_peered":"2023-07-21T22:14:52.324003+0000","last_clean":"2023-07-21T22:14:52.324003+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:14:52.324003+0000","last_undegraded":"2023-07-21T22:14:52.324003+0000","last_fullsized":"2023-07-21T22:14:52.324003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:23.704220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249061+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:14:52.249061+0000","last_peered":"2023-07-21T22:14:52.249061+0000","last_clean":"2023-07-21T22:14:52.249061+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:14:52.249061+0000","last_undegraded":"2023-07-21T22:14:52.249061+0000","last_fullsized":"2023-07-21T22:14:52.249061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:22.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323571+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:14:52.323571+0000","last_peered":"2023-07-21T22:14:52.323571+0000","last_clean":"2023-07-21T22:14:52.323571+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:14:52.323571+0000","last_undegraded":"2023-07-21T22:14:52.323571+0000","last_fullsized":"2023-07-21T22:14:52.323571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:35:26.714623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248660+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:14:52.248660+0000","last_peered":"2023-07-21T22:14:52.248660+0000","last_clean":"2023-07-21T22:14:52.248660+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:14:52.248660+0000","last_undegraded":"2023-07-21T22:14:52.248660+0000","last_fullsized":"2023-07-21T22:14:52.248660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:27:40.710691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249020+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.249020+0000","last_peered":"2023-07-21T22:14:52.249020+0000","last_clean":"2023-07-21T22:14:52.249020+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:14:52.249020+0000","last_undegraded":"2023-07-21T22:14:52.249020+0000","last_fullsized":"2023-07-21T22:14:52.249020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:45.083477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323134+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:14:52.323134+0000","last_peered":"2023-07-21T22:14:52.323134+0000","last_clean":"2023-07-21T22:14:52.323134+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:14:52.323134+0000","last_undegraded":"2023-07-21T22:14:52.323134+0000","last_fullsized":"2023-07-21T22:14:52.323134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:19:31.888652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248874+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:14:52.248874+0000","last_peered":"2023-07-21T22:14:52.248874+0000","last_clean":"2023-07-21T22:14:52.248874+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:14:52.248874+0000","last_undegraded":"2023-07-21T22:14:52.248874+0000","last_fullsized":"2023-07-21T22:14:52.248874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:28.191291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23801,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":163,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":68954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1552384,"data_stored":1033922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.623},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84699999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79700000000000004}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":423221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.628}]},{"osd":1,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.0049999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.032}]},{"osd":9,"last update":"Fri Jul 21 22:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.0920000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.081},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":1.0680000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1013433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86799999999999999,"5min":1.024,"15min":1.024},"last":0.755},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.91500000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.92900000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.378,"15min":0.371},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.373}]},{"osd":1,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.51800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60599999999999998,"15min":0.59499999999999997},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.621},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54500000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63200000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":424378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.38400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.40100000000000002,"15min":0.371},"min":{"1min":0.33300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42699999999999999,"15min":0.39700000000000002},"min":{"1min":0.313,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.441}]},{"osd":5,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86599999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67100000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.443,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.44800000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.436},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.432}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.39300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.45500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.75,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.495},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56100000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54700000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.35899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.496},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.63600000000000001,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.7210000000000001,"5min":4.7210000000000001,"15min":4.7210000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.439,"15min":0.439},"min":{"1min":0.30299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.75,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.745},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.47899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":1.012},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.90100000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":1.0389999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.1200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.318},"min":{"1min":0.29499999999999998,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.32900000000000001},"min":{"1min":0.28499999999999998,"5min":0.19,"15min":0.19},"max":{"1min":0.57799999999999996,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73999999999999999,"5min":1.512,"15min":1.512},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56000000000000005,"15min":0.52600000000000002},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89700000000000002,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.874,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92000000000000004,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95499999999999996,"5min":1.653,"15min":1.653},"last":0.376},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88800000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71799999999999997,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.624},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953514,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.32700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.443},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.51400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94199999999999995,"5min":1.22,"15min":1.22},"last":0.623},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.89600000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.86799999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1515520,"data_stored":1015372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.434,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.373},"min":{"1min":0.26500000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1.47,"15min":1.47},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0169999999999999,"5min":1.53,"15min":1.53},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.78000000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":38113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":20617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:14:59.293 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:14:59.321 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: 2023-07-21T22:14:59.002+0000 7f3e7bb83700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-07-21T22:14:59.321 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: 2023-07-21T22:14:59.002+0000 7f3e7bb83700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-07-21T22:14:59.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 -- 172.21.15.93:0/1554904650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 msgr2=0x7f84b4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 --2- 172.21.15.93:0/1554904650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b4106e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b0a0 tx=0x7f84ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 -- 172.21.15.93:0/1554904650 shutdown_connections 2023-07-21T22:14:59.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 --2- 172.21.15.93:0/1554904650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b4106e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 --2- 172.21.15.93:0/1554904650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 0x7f84b40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 --2- 172.21.15.93:0/1554904650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 -- 172.21.15.93:0/1554904650 >> 172.21.15.93:0/1554904650 conn(0x7f84b40fa000 msgr2=0x7f84b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:59.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 -- 172.21.15.93:0/1554904650 shutdown_connections 2023-07-21T22:14:59.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.400+0000 7f84b8842700 1 -- 172.21.15.93:0/1554904650 wait complete. 2023-07-21T22:14:59.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 Processor -- start 2023-07-21T22:14:59.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 -- start start 2023-07-21T22:14:59.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 0x7f84b4195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419d410 con 0x7f84b4100560 2023-07-21T22:14:59.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419d550 con 0x7f84b40fe9e0 2023-07-21T22:14:59.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b8842700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419d6e0 con 0x7f84b40ffbd0 2023-07-21T22:14:59.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42330/0 (socket says 172.21.15.93:42330) 2023-07-21T22:14:59.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.401+0000 7f84b2d9d700 1 -- 172.21.15.93:0/1003585387 learned_addr learned my addr 172.21.15.93:0/1003585387 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:14:59.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b259c700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 0x7f84b4195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b1d9b700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 -- 172.21.15.93:0/1003585387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 msgr2=0x7f84b4195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 -- 172.21.15.93:0/1003585387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 msgr2=0x7f84b4195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 0x7f84b4195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 -- 172.21.15.93:0/1003585387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac009d00 con 0x7f84b4100560 2023-07-21T22:14:59.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b1d9b700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:59.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b2d9d700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b830 tx=0x7f84ac004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:59.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00e040 con 0x7f84b4100560 2023-07-21T22:14:59.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac0094d0 con 0x7f84b4100560 2023-07-21T22:14:59.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac01bd20 con 0x7f84b4100560 2023-07-21T22:14:59.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b419d960 con 0x7f84b4100560 2023-07-21T22:14:59.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.402+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b419dde0 con 0x7f84b4100560 2023-07-21T22:14:59.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.403+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4105e30 con 0x7f84b4100560 2023-07-21T22:14:59.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.404+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f84ac01b450 con 0x7f84b4100560 2023-07-21T22:14:59.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.404+0000 7f84a37fe700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 0x7f849c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:59.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.405+0000 7f84b259c700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 0x7f849c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:59.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.405+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84ac0ce7e0 con 0x7f84b4100560 2023-07-21T22:14:59.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.405+0000 7f84b259c700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 0x7f849c0a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ffa30 tx=0x7f84a800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:59.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.409+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f84ac098010 con 0x7f84b4100560 2023-07-21T22:14:59.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-mon[98288]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-mon[98288]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3756420869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:14:59 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/631983444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:59 smithi134 ceph-mon[97873]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:59 smithi134 ceph-mon[97873]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:59 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3756420869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:14:59 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/631983444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.592+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84b4060950 con 0x7f849c0a12a0 2023-07-21T22:14:59.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.602+0000 7f84a37fe700 1 -- 172.21.15.93:0/1003585387 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336321 (secure 0 0 0) 0x7f84b4060950 con 0x7f849c0a12a0 2023-07-21T22:14:59.648 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:14:59.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 msgr2=0x7f849c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 0x7f849c0a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84b40ffa30 tx=0x7f84a800a380 comp rx=0 tx=0).stop 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 msgr2=0x7f84b419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b830 tx=0x7f84ac004d00 comp rx=0 tx=0).stop 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 shutdown_connections 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84b4100560 0x7f84b419ce40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f849c0a12a0 0x7f849c0a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84b40fe9e0 0x7f84b4195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 --2- 172.21.15.93:0/1003585387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84b40ffbd0 0x7f84b4195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 >> 172.21.15.93:0/1003585387 conn(0x7f84b40fa000 msgr2=0x7f84b4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 shutdown_connections 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:14:59.605+0000 7f84b8842700 1 -- 172.21.15.93:0/1003585387 wait complete. 2023-07-21T22:14:59.651 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:14:59.659 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:59 smithi093 ceph-mon[96579]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.659 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:59 smithi093 ceph-mon[96579]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:59.660 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:59 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3756420869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.660 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:14:59 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/631983444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:14:59.661 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-07-21T22:14:59.207436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614292,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":345,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335756,"kb_used_data":12332,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606068,"statfs":{"total":1151940427776,"available":1151596613632,"internally_reserved":0,"allocated":12627968,"data_stored":6893104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23924,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":164,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249025+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:14:52.249025+0000","last_peered":"2023-07-21T22:14:52.249025+0000","last_clean":"2023-07-21T22:14:52.249025+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:14:52.249025+0000","last_undegraded":"2023-07-21T22:14:52.249025+0000","last_fullsized":"2023-07-21T22:14:52.249025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:33.953568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324185+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:14:52.324185+0000","last_peered":"2023-07-21T22:14:52.324185+0000","last_clean":"2023-07-21T22:14:52.324185+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:14:52.324185+0000","last_undegraded":"2023-07-21T22:14:52.324185+0000","last_fullsized":"2023-07-21T22:14:52.324185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:29.855484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323763+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.323763+0000","last_peered":"2023-07-21T22:14:52.323763+0000","last_clean":"2023-07-21T22:14:52.323763+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:14:52.323763+0000","last_undegraded":"2023-07-21T22:14:52.323763+0000","last_fullsized":"2023-07-21T22:14:52.323763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:27.021961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324294+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:14:52.324294+0000","last_peered":"2023-07-21T22:14:52.324294+0000","last_clean":"2023-07-21T22:14:52.324294+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:14:52.324294+0000","last_undegraded":"2023-07-21T22:14:52.324294+0000","last_fullsized":"2023-07-21T22:14:52.324294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:59:03.712988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550902+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:14:53.550902+0000","last_peered":"2023-07-21T22:14:53.550902+0000","last_clean":"2023-07-21T22:14:53.550902+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:14:53.550902+0000","last_undegraded":"2023-07-21T22:14:53.550902+0000","last_fullsized":"2023-07-21T22:14:53.550902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:00.017709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323849+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:14:52.323849+0000","last_peered":"2023-07-21T22:14:52.323849+0000","last_clean":"2023-07-21T22:14:52.323849+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:14:52.323849+0000","last_undegraded":"2023-07-21T22:14:52.323849+0000","last_fullsized":"2023-07-21T22:14:52.323849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:58.487301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323957+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:14:52.323957+0000","last_peered":"2023-07-21T22:14:52.323957+0000","last_clean":"2023-07-21T22:14:52.323957+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:14:52.323957+0000","last_undegraded":"2023-07-21T22:14:52.323957+0000","last_fullsized":"2023-07-21T22:14:52.323957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:02.172487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248538+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:14:52.248538+0000","last_peered":"2023-07-21T22:14:52.248538+0000","last_clean":"2023-07-21T22:14:52.248538+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:14:52.248538+0000","last_undegraded":"2023-07-21T22:14:52.248538+0000","last_fullsized":"2023-07-21T22:14:52.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:54.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249436+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:14:52.249436+0000","last_peered":"2023-07-21T22:14:52.249436+0000","last_clean":"2023-07-21T22:14:52.249436+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:14:52.249436+0000","last_undegraded":"2023-07-21T22:14:52.249436+0000","last_fullsized":"2023-07-21T22:14:52.249436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:33:03.512159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324232+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:14:52.324232+0000","last_peered":"2023-07-21T22:14:52.324232+0000","last_clean":"2023-07-21T22:14:52.324232+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:14:52.324232+0000","last_undegraded":"2023-07-21T22:14:52.324232+0000","last_fullsized":"2023-07-21T22:14:52.324232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:33.917374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324415+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:14:52.324415+0000","last_peered":"2023-07-21T22:14:52.324415+0000","last_clean":"2023-07-21T22:14:52.324415+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:14:52.324415+0000","last_undegraded":"2023-07-21T22:14:52.324415+0000","last_fullsized":"2023-07-21T22:14:52.324415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:42.648086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324156+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.324156+0000","last_peered":"2023-07-21T22:14:52.324156+0000","last_clean":"2023-07-21T22:14:52.324156+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:14:52.324156+0000","last_undegraded":"2023-07-21T22:14:52.324156+0000","last_fullsized":"2023-07-21T22:14:52.324156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:12.914647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323062+0000","last_change":"2023-07-21T22:14:51.245228+0000","last_active":"2023-07-21T22:14:52.323062+0000","last_peered":"2023-07-21T22:14:52.323062+0000","last_clean":"2023-07-21T22:14:52.323062+0000","last_became_active":"2023-07-21T22:14:51.244919+0000","last_became_peered":"2023-07-21T22:14:51.244919+0000","last_unstale":"2023-07-21T22:14:52.323062+0000","last_undegraded":"2023-07-21T22:14:52.323062+0000","last_fullsized":"2023-07-21T22:14:52.323062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:21:11.471385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323500+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:14:52.323500+0000","last_peered":"2023-07-21T22:14:52.323500+0000","last_clean":"2023-07-21T22:14:52.323500+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:14:52.323500+0000","last_undegraded":"2023-07-21T22:14:52.323500+0000","last_fullsized":"2023-07-21T22:14:52.323500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:05:33.532014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323093+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:14:52.323093+0000","last_peered":"2023-07-21T22:14:52.323093+0000","last_clean":"2023-07-21T22:14:52.323093+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:14:52.323093+0000","last_undegraded":"2023-07-21T22:14:52.323093+0000","last_fullsized":"2023-07-21T22:14:52.323093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:57.567468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248840+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:14:52.248840+0000","last_peered":"2023-07-21T22:14:52.248840+0000","last_clean":"2023-07-21T22:14:52.248840+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:14:52.248840+0000","last_undegraded":"2023-07-21T22:14:52.248840+0000","last_fullsized":"2023-07-21T22:14:52.248840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:28:05.983432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249184+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:14:52.249184+0000","last_peered":"2023-07-21T22:14:52.249184+0000","last_clean":"2023-07-21T22:14:52.249184+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:14:52.249184+0000","last_undegraded":"2023-07-21T22:14:52.249184+0000","last_fullsized":"2023-07-21T22:14:52.249184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:35.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323352+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:14:52.323352+0000","last_peered":"2023-07-21T22:14:52.323352+0000","last_clean":"2023-07-21T22:14:52.323352+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:14:52.323352+0000","last_undegraded":"2023-07-21T22:14:52.323352+0000","last_fullsized":"2023-07-21T22:14:52.323352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:31.713111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323930+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:14:52.323930+0000","last_peered":"2023-07-21T22:14:52.323930+0000","last_clean":"2023-07-21T22:14:52.323930+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:14:52.323930+0000","last_undegraded":"2023-07-21T22:14:52.323930+0000","last_fullsized":"2023-07-21T22:14:52.323930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:45.482633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002755+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:14:57.002755+0000","last_peered":"2023-07-21T22:14:57.002755+0000","last_clean":"2023-07-21T22:14:57.002755+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:14:57.002755+0000","last_undegraded":"2023-07-21T22:14:57.002755+0000","last_fullsized":"2023-07-21T22:14:57.002755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:42.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248992+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:14:52.248992+0000","last_peered":"2023-07-21T22:14:52.248992+0000","last_clean":"2023-07-21T22:14:52.248992+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:14:52.248992+0000","last_undegraded":"2023-07-21T22:14:52.248992+0000","last_fullsized":"2023-07-21T22:14:52.248992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:57.089271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249058+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:14:52.249058+0000","last_peered":"2023-07-21T22:14:52.249058+0000","last_clean":"2023-07-21T22:14:52.249058+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:14:52.249058+0000","last_undegraded":"2023-07-21T22:14:52.249058+0000","last_fullsized":"2023-07-21T22:14:52.249058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:17.931445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324904+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:14:52.324904+0000","last_peered":"2023-07-21T22:14:52.324904+0000","last_clean":"2023-07-21T22:14:52.324904+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:14:52.324904+0000","last_undegraded":"2023-07-21T22:14:52.324904+0000","last_fullsized":"2023-07-21T22:14:52.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:07.388393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285630+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:14:56.285630+0000","last_peered":"2023-07-21T22:14:56.285630+0000","last_clean":"2023-07-21T22:14:56.285630+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:14:56.285630+0000","last_undegraded":"2023-07-21T22:14:56.285630+0000","last_fullsized":"2023-07-21T22:14:56.285630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:45.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248983+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:14:52.248983+0000","last_peered":"2023-07-21T22:14:52.248983+0000","last_clean":"2023-07-21T22:14:52.248983+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:14:52.248983+0000","last_undegraded":"2023-07-21T22:14:52.248983+0000","last_fullsized":"2023-07-21T22:14:52.248983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:33.576139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551119+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:14:53.551119+0000","last_peered":"2023-07-21T22:14:53.551119+0000","last_clean":"2023-07-21T22:14:53.551119+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:14:53.551119+0000","last_undegraded":"2023-07-21T22:14:53.551119+0000","last_fullsized":"2023-07-21T22:14:53.551119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:04:03.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324394+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.324394+0000","last_peered":"2023-07-21T22:14:52.324394+0000","last_clean":"2023-07-21T22:14:52.324394+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:14:52.324394+0000","last_undegraded":"2023-07-21T22:14:52.324394+0000","last_fullsized":"2023-07-21T22:14:52.324394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:10:13.245800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.999086+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:14:56.999086+0000","last_peered":"2023-07-21T22:14:56.999086+0000","last_clean":"2023-07-21T22:14:56.999086+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:14:56.999086+0000","last_undegraded":"2023-07-21T22:14:56.999086+0000","last_fullsized":"2023-07-21T22:14:56.999086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:15:58.146646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551008+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:14:53.551008+0000","last_peered":"2023-07-21T22:14:53.551008+0000","last_clean":"2023-07-21T22:14:53.551008+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:14:53.551008+0000","last_undegraded":"2023-07-21T22:14:53.551008+0000","last_fullsized":"2023-07-21T22:14:53.551008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:04.181744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285879+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:14:56.285879+0000","last_peered":"2023-07-21T22:14:56.285879+0000","last_clean":"2023-07-21T22:14:56.285879+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:14:56.285879+0000","last_undegraded":"2023-07-21T22:14:56.285879+0000","last_fullsized":"2023-07-21T22:14:56.285879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:46:03.474285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324013+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:14:52.324013+0000","last_peered":"2023-07-21T22:14:52.324013+0000","last_clean":"2023-07-21T22:14:52.324013+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:14:52.324013+0000","last_undegraded":"2023-07-21T22:14:52.324013+0000","last_fullsized":"2023-07-21T22:14:52.324013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:15:10.615938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323388+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:14:52.323388+0000","last_peered":"2023-07-21T22:14:52.323388+0000","last_clean":"2023-07-21T22:14:52.323388+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:14:52.323388+0000","last_undegraded":"2023-07-21T22:14:52.323388+0000","last_fullsized":"2023-07-21T22:14:52.323388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:01.057949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248598+0000","last_change":"2023-07-21T22:14:51.248592+0000","last_active":"2023-07-21T22:14:52.248598+0000","last_peered":"2023-07-21T22:14:52.248598+0000","last_clean":"2023-07-21T22:14:52.248598+0000","last_became_active":"2023-07-21T22:14:51.247932+0000","last_became_peered":"2023-07-21T22:14:51.247932+0000","last_unstale":"2023-07-21T22:14:52.248598+0000","last_undegraded":"2023-07-21T22:14:52.248598+0000","last_fullsized":"2023-07-21T22:14:52.248598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:01.815385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001359+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:14:57.001359+0000","last_peered":"2023-07-21T22:14:57.001359+0000","last_clean":"2023-07-21T22:14:57.001359+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:14:57.001359+0000","last_undegraded":"2023-07-21T22:14:57.001359+0000","last_fullsized":"2023-07-21T22:14:57.001359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:40.285840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322887+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:14:52.322887+0000","last_peered":"2023-07-21T22:14:52.322887+0000","last_clean":"2023-07-21T22:14:52.322887+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:14:52.322887+0000","last_undegraded":"2023-07-21T22:14:52.322887+0000","last_fullsized":"2023-07-21T22:14:52.322887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:55:36.899847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.998483+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:14:56.998483+0000","last_peered":"2023-07-21T22:14:56.998483+0000","last_clean":"2023-07-21T22:14:56.998483+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:14:56.998483+0000","last_undegraded":"2023-07-21T22:14:56.998483+0000","last_fullsized":"2023-07-21T22:14:56.998483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:14.250309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249338+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:14:52.249338+0000","last_peered":"2023-07-21T22:14:52.249338+0000","last_clean":"2023-07-21T22:14:52.249338+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:14:52.249338+0000","last_undegraded":"2023-07-21T22:14:52.249338+0000","last_fullsized":"2023-07-21T22:14:52.249338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:06.815314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.271552+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:14:54.271552+0000","last_peered":"2023-07-21T22:14:54.271552+0000","last_clean":"2023-07-21T22:14:54.271552+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:14:54.271552+0000","last_undegraded":"2023-07-21T22:14:54.271552+0000","last_fullsized":"2023-07-21T22:14:54.271552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:38:48.158170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324814+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:14:52.324814+0000","last_peered":"2023-07-21T22:14:52.324814+0000","last_clean":"2023-07-21T22:14:52.324814+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:14:52.324814+0000","last_undegraded":"2023-07-21T22:14:52.324814+0000","last_fullsized":"2023-07-21T22:14:52.324814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:53:48.638340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284307+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:14:56.284307+0000","last_peered":"2023-07-21T22:14:56.284307+0000","last_clean":"2023-07-21T22:14:56.284307+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:14:56.284307+0000","last_undegraded":"2023-07-21T22:14:56.284307+0000","last_fullsized":"2023-07-21T22:14:56.284307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:15.490891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249055+0000","last_change":"2023-07-21T22:14:51.249768+0000","last_active":"2023-07-21T22:14:52.249055+0000","last_peered":"2023-07-21T22:14:52.249055+0000","last_clean":"2023-07-21T22:14:52.249055+0000","last_became_active":"2023-07-21T22:14:51.249503+0000","last_became_peered":"2023-07-21T22:14:51.249503+0000","last_unstale":"2023-07-21T22:14:52.249055+0000","last_undegraded":"2023-07-21T22:14:52.249055+0000","last_fullsized":"2023-07-21T22:14:52.249055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:39:15.294044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'16","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.017000+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:14:57.017000+0000","last_peered":"2023-07-21T22:14:57.017000+0000","last_clean":"2023-07-21T22:14:57.017000+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:14:57.017000+0000","last_undegraded":"2023-07-21T22:14:57.017000+0000","last_fullsized":"2023-07-21T22:14:57.017000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:38.675168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":18,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551147+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:14:53.551147+0000","last_peered":"2023-07-21T22:14:53.551147+0000","last_clean":"2023-07-21T22:14:53.551147+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:14:53.551147+0000","last_undegraded":"2023-07-21T22:14:53.551147+0000","last_fullsized":"2023-07-21T22:14:53.551147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:51.431054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006578+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:14:57.006578+0000","last_peered":"2023-07-21T22:14:57.006578+0000","last_clean":"2023-07-21T22:14:57.006578+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:14:57.006578+0000","last_undegraded":"2023-07-21T22:14:57.006578+0000","last_fullsized":"2023-07-21T22:14:57.006578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:55.930555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323112+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:14:52.323112+0000","last_peered":"2023-07-21T22:14:52.323112+0000","last_clean":"2023-07-21T22:14:52.323112+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:14:52.323112+0000","last_undegraded":"2023-07-21T22:14:52.323112+0000","last_fullsized":"2023-07-21T22:14:52.323112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:57:57.488142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284858+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:14:56.284858+0000","last_peered":"2023-07-21T22:14:56.284858+0000","last_clean":"2023-07-21T22:14:56.284858+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:14:56.284858+0000","last_undegraded":"2023-07-21T22:14:56.284858+0000","last_fullsized":"2023-07-21T22:14:56.284858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:28.876044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324774+0000","last_change":"2023-07-21T22:14:51.258940+0000","last_active":"2023-07-21T22:14:52.324774+0000","last_peered":"2023-07-21T22:14:52.324774+0000","last_clean":"2023-07-21T22:14:52.324774+0000","last_became_active":"2023-07-21T22:14:51.258595+0000","last_became_peered":"2023-07-21T22:14:51.258595+0000","last_unstale":"2023-07-21T22:14:52.324774+0000","last_undegraded":"2023-07-21T22:14:52.324774+0000","last_fullsized":"2023-07-21T22:14:52.324774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:39.297921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284308+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:14:56.284308+0000","last_peered":"2023-07-21T22:14:56.284308+0000","last_clean":"2023-07-21T22:14:56.284308+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:14:56.284308+0000","last_undegraded":"2023-07-21T22:14:56.284308+0000","last_fullsized":"2023-07-21T22:14:56.284308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:39.148790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323808+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:14:52.323808+0000","last_peered":"2023-07-21T22:14:52.323808+0000","last_clean":"2023-07-21T22:14:52.323808+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:14:52.323808+0000","last_undegraded":"2023-07-21T22:14:52.323808+0000","last_fullsized":"2023-07-21T22:14:52.323808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:38.761579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'16","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.280888+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:14:55.280888+0000","last_peered":"2023-07-21T22:14:55.280888+0000","last_clean":"2023-07-21T22:14:55.280888+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:14:55.280888+0000","last_undegraded":"2023-07-21T22:14:55.280888+0000","last_fullsized":"2023-07-21T22:14:55.280888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:01:49.310753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323584+0000","last_change":"2023-07-21T22:14:51.252408+0000","last_active":"2023-07-21T22:14:52.323584+0000","last_peered":"2023-07-21T22:14:52.323584+0000","last_clean":"2023-07-21T22:14:52.323584+0000","last_became_active":"2023-07-21T22:14:51.252074+0000","last_became_peered":"2023-07-21T22:14:51.252074+0000","last_unstale":"2023-07-21T22:14:52.323584+0000","last_undegraded":"2023-07-21T22:14:52.323584+0000","last_fullsized":"2023-07-21T22:14:52.323584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.877759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551338+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:14:53.551338+0000","last_peered":"2023-07-21T22:14:53.551338+0000","last_clean":"2023-07-21T22:14:53.551338+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:14:53.551338+0000","last_undegraded":"2023-07-21T22:14:53.551338+0000","last_fullsized":"2023-07-21T22:14:53.551338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:36.286445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249168+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:14:52.249168+0000","last_peered":"2023-07-21T22:14:52.249168+0000","last_clean":"2023-07-21T22:14:52.249168+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:14:52.249168+0000","last_undegraded":"2023-07-21T22:14:52.249168+0000","last_fullsized":"2023-07-21T22:14:52.249168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:12.075583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.101685+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:14:53.101685+0000","last_peered":"2023-07-21T22:14:53.101685+0000","last_clean":"2023-07-21T22:14:53.101685+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:14:53.101685+0000","last_undegraded":"2023-07-21T22:14:53.101685+0000","last_fullsized":"2023-07-21T22:14:53.101685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:03.657128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.530952+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:14:52.530952+0000","last_peered":"2023-07-21T22:14:52.530952+0000","last_clean":"2023-07-21T22:14:52.530952+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:14:52.530952+0000","last_undegraded":"2023-07-21T22:14:52.530952+0000","last_fullsized":"2023-07-21T22:14:52.530952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:11:20.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324770+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:52.324770+0000","last_peered":"2023-07-21T22:14:52.324770+0000","last_clean":"2023-07-21T22:14:52.324770+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:14:52.324770+0000","last_undegraded":"2023-07-21T22:14:52.324770+0000","last_fullsized":"2023-07-21T22:14:52.324770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:37.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324083+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:14:52.324083+0000","last_peered":"2023-07-21T22:14:52.324083+0000","last_clean":"2023-07-21T22:14:52.324083+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:14:52.324083+0000","last_undegraded":"2023-07-21T22:14:52.324083+0000","last_fullsized":"2023-07-21T22:14:52.324083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:57:09.279672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323831+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:14:52.323831+0000","last_peered":"2023-07-21T22:14:52.323831+0000","last_clean":"2023-07-21T22:14:52.323831+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:14:52.323831+0000","last_undegraded":"2023-07-21T22:14:52.323831+0000","last_fullsized":"2023-07-21T22:14:52.323831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:31.048621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323313+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:14:52.323313+0000","last_peered":"2023-07-21T22:14:52.323313+0000","last_clean":"2023-07-21T22:14:52.323313+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:14:52.323313+0000","last_undegraded":"2023-07-21T22:14:52.323313+0000","last_fullsized":"2023-07-21T22:14:52.323313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:39:31.032230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248803+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:14:52.248803+0000","last_peered":"2023-07-21T22:14:52.248803+0000","last_clean":"2023-07-21T22:14:52.248803+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:14:52.248803+0000","last_undegraded":"2023-07-21T22:14:52.248803+0000","last_fullsized":"2023-07-21T22:14:52.248803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:06.145398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284531+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:56.284531+0000","last_peered":"2023-07-21T22:14:56.284531+0000","last_clean":"2023-07-21T22:14:56.284531+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:14:56.284531+0000","last_undegraded":"2023-07-21T22:14:56.284531+0000","last_fullsized":"2023-07-21T22:14:56.284531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:48.946604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323556+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:14:52.323556+0000","last_peered":"2023-07-21T22:14:52.323556+0000","last_clean":"2023-07-21T22:14:52.323556+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:14:52.323556+0000","last_undegraded":"2023-07-21T22:14:52.323556+0000","last_fullsized":"2023-07-21T22:14:52.323556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:04.146166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323206+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:14:52.323206+0000","last_peered":"2023-07-21T22:14:52.323206+0000","last_clean":"2023-07-21T22:14:52.323206+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:14:52.323206+0000","last_undegraded":"2023-07-21T22:14:52.323206+0000","last_fullsized":"2023-07-21T22:14:52.323206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:14:26.470268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323462+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:14:52.323462+0000","last_peered":"2023-07-21T22:14:52.323462+0000","last_clean":"2023-07-21T22:14:52.323462+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:14:52.323462+0000","last_undegraded":"2023-07-21T22:14:52.323462+0000","last_fullsized":"2023-07-21T22:14:52.323462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:41.327520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284749+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:14:56.284749+0000","last_peered":"2023-07-21T22:14:56.284749+0000","last_clean":"2023-07-21T22:14:56.284749+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:14:56.284749+0000","last_undegraded":"2023-07-21T22:14:56.284749+0000","last_fullsized":"2023-07-21T22:14:56.284749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:00.386436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323125+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:14:52.323125+0000","last_peered":"2023-07-21T22:14:52.323125+0000","last_clean":"2023-07-21T22:14:52.323125+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:14:52.323125+0000","last_undegraded":"2023-07-21T22:14:52.323125+0000","last_fullsized":"2023-07-21T22:14:52.323125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:25.036430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285167+0000","last_change":"2023-07-21T22:14:50.231962+0000","last_active":"2023-07-21T22:14:56.285167+0000","last_peered":"2023-07-21T22:14:56.285167+0000","last_clean":"2023-07-21T22:14:56.285167+0000","last_became_active":"2023-07-21T22:14:50.231638+0000","last_became_peered":"2023-07-21T22:14:50.231638+0000","last_unstale":"2023-07-21T22:14:56.285167+0000","last_undegraded":"2023-07-21T22:14:56.285167+0000","last_fullsized":"2023-07-21T22:14:56.285167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:18.134292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249093+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:14:52.249093+0000","last_peered":"2023-07-21T22:14:52.249093+0000","last_clean":"2023-07-21T22:14:52.249093+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:14:52.249093+0000","last_undegraded":"2023-07-21T22:14:52.249093+0000","last_fullsized":"2023-07-21T22:14:52.249093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:15.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270970+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:14:54.270970+0000","last_peered":"2023-07-21T22:14:54.270970+0000","last_clean":"2023-07-21T22:14:54.270970+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:14:54.270970+0000","last_undegraded":"2023-07-21T22:14:54.270970+0000","last_fullsized":"2023-07-21T22:14:54.270970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:43.076462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324274+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:14:52.324274+0000","last_peered":"2023-07-21T22:14:52.324274+0000","last_clean":"2023-07-21T22:14:52.324274+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:14:52.324274+0000","last_undegraded":"2023-07-21T22:14:52.324274+0000","last_fullsized":"2023-07-21T22:14:52.324274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:16.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270269+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:14:54.270269+0000","last_peered":"2023-07-21T22:14:54.270269+0000","last_clean":"2023-07-21T22:14:54.270269+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:14:54.270269+0000","last_undegraded":"2023-07-21T22:14:54.270269+0000","last_fullsized":"2023-07-21T22:14:54.270269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:06:47.274452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550789+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:14:53.550789+0000","last_peered":"2023-07-21T22:14:53.550789+0000","last_clean":"2023-07-21T22:14:53.550789+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:14:53.550789+0000","last_undegraded":"2023-07-21T22:14:53.550789+0000","last_fullsized":"2023-07-21T22:14:53.550789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:02:40.632894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270839+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:14:54.270839+0000","last_peered":"2023-07-21T22:14:54.270839+0000","last_clean":"2023-07-21T22:14:54.270839+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:14:54.270839+0000","last_undegraded":"2023-07-21T22:14:54.270839+0000","last_fullsized":"2023-07-21T22:14:54.270839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:31.455547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248754+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:14:52.248754+0000","last_peered":"2023-07-21T22:14:52.248754+0000","last_clean":"2023-07-21T22:14:52.248754+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:14:52.248754+0000","last_undegraded":"2023-07-21T22:14:52.248754+0000","last_fullsized":"2023-07-21T22:14:52.248754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:25.484174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.867405+0000","last_change":"2023-07-21T22:14:50.229923+0000","last_active":"2023-07-21T22:14:52.867405+0000","last_peered":"2023-07-21T22:14:52.867405+0000","last_clean":"2023-07-21T22:14:52.867405+0000","last_became_active":"2023-07-21T22:14:50.229645+0000","last_became_peered":"2023-07-21T22:14:50.229645+0000","last_unstale":"2023-07-21T22:14:52.867405+0000","last_undegraded":"2023-07-21T22:14:52.867405+0000","last_fullsized":"2023-07-21T22:14:52.867405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:36:56.239739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248731+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:14:52.248731+0000","last_peered":"2023-07-21T22:14:52.248731+0000","last_clean":"2023-07-21T22:14:52.248731+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:14:52.248731+0000","last_undegraded":"2023-07-21T22:14:52.248731+0000","last_fullsized":"2023-07-21T22:14:52.248731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:20.211619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.269986+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:14:54.269986+0000","last_peered":"2023-07-21T22:14:54.269986+0000","last_clean":"2023-07-21T22:14:54.269986+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:14:54.269986+0000","last_undegraded":"2023-07-21T22:14:54.269986+0000","last_fullsized":"2023-07-21T22:14:54.269986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:01.500609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324713+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:14:52.324713+0000","last_peered":"2023-07-21T22:14:52.324713+0000","last_clean":"2023-07-21T22:14:52.324713+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:14:52.324713+0000","last_undegraded":"2023-07-21T22:14:52.324713+0000","last_fullsized":"2023-07-21T22:14:52.324713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:17.553733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001407+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:14:57.001407+0000","last_peered":"2023-07-21T22:14:57.001407+0000","last_clean":"2023-07-21T22:14:57.001407+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:14:57.001407+0000","last_undegraded":"2023-07-21T22:14:57.001407+0000","last_fullsized":"2023-07-21T22:14:57.001407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:15.167223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324532+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:14:52.324532+0000","last_peered":"2023-07-21T22:14:52.324532+0000","last_clean":"2023-07-21T22:14:52.324532+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:14:52.324532+0000","last_undegraded":"2023-07-21T22:14:52.324532+0000","last_fullsized":"2023-07-21T22:14:52.324532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:33.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996861+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:14:56.996861+0000","last_peered":"2023-07-21T22:14:56.996861+0000","last_clean":"2023-07-21T22:14:56.996861+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:14:56.996861+0000","last_undegraded":"2023-07-21T22:14:56.996861+0000","last_fullsized":"2023-07-21T22:14:56.996861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.897090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324289+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:14:52.324289+0000","last_peered":"2023-07-21T22:14:52.324289+0000","last_clean":"2023-07-21T22:14:52.324289+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:14:52.324289+0000","last_undegraded":"2023-07-21T22:14:52.324289+0000","last_fullsized":"2023-07-21T22:14:52.324289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:52.794435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323230+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:14:52.323230+0000","last_peered":"2023-07-21T22:14:52.323230+0000","last_clean":"2023-07-21T22:14:52.323230+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:14:52.323230+0000","last_undegraded":"2023-07-21T22:14:52.323230+0000","last_fullsized":"2023-07-21T22:14:52.323230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:56.943909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324655+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:14:52.324655+0000","last_peered":"2023-07-21T22:14:52.324655+0000","last_clean":"2023-07-21T22:14:52.324655+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:14:52.324655+0000","last_undegraded":"2023-07-21T22:14:52.324655+0000","last_fullsized":"2023-07-21T22:14:52.324655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:05:57.078423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248843+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:14:52.248843+0000","last_peered":"2023-07-21T22:14:52.248843+0000","last_clean":"2023-07-21T22:14:52.248843+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:14:52.248843+0000","last_undegraded":"2023-07-21T22:14:52.248843+0000","last_fullsized":"2023-07-21T22:14:52.248843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:49:10.848487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324305+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:14:52.324305+0000","last_peered":"2023-07-21T22:14:52.324305+0000","last_clean":"2023-07-21T22:14:52.324305+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:14:52.324305+0000","last_undegraded":"2023-07-21T22:14:52.324305+0000","last_fullsized":"2023-07-21T22:14:52.324305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:16:56.773704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323602+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:14:52.323602+0000","last_peered":"2023-07-21T22:14:52.323602+0000","last_clean":"2023-07-21T22:14:52.323602+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:14:52.323602+0000","last_undegraded":"2023-07-21T22:14:52.323602+0000","last_fullsized":"2023-07-21T22:14:52.323602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:58.733025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324433+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:14:52.324433+0000","last_peered":"2023-07-21T22:14:52.324433+0000","last_clean":"2023-07-21T22:14:52.324433+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:14:52.324433+0000","last_undegraded":"2023-07-21T22:14:52.324433+0000","last_fullsized":"2023-07-21T22:14:52.324433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:51.861062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322982+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:14:52.322982+0000","last_peered":"2023-07-21T22:14:52.322982+0000","last_clean":"2023-07-21T22:14:52.322982+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:14:52.322982+0000","last_undegraded":"2023-07-21T22:14:52.322982+0000","last_fullsized":"2023-07-21T22:14:52.322982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:29.650653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248998+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:14:52.248998+0000","last_peered":"2023-07-21T22:14:52.248998+0000","last_clean":"2023-07-21T22:14:52.248998+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:14:52.248998+0000","last_undegraded":"2023-07-21T22:14:52.248998+0000","last_fullsized":"2023-07-21T22:14:52.248998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:30:39.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324003+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:14:52.324003+0000","last_peered":"2023-07-21T22:14:52.324003+0000","last_clean":"2023-07-21T22:14:52.324003+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:14:52.324003+0000","last_undegraded":"2023-07-21T22:14:52.324003+0000","last_fullsized":"2023-07-21T22:14:52.324003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:23.704220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249061+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:14:52.249061+0000","last_peered":"2023-07-21T22:14:52.249061+0000","last_clean":"2023-07-21T22:14:52.249061+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:14:52.249061+0000","last_undegraded":"2023-07-21T22:14:52.249061+0000","last_fullsized":"2023-07-21T22:14:52.249061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:22.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323571+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:14:52.323571+0000","last_peered":"2023-07-21T22:14:52.323571+0000","last_clean":"2023-07-21T22:14:52.323571+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:14:52.323571+0000","last_undegraded":"2023-07-21T22:14:52.323571+0000","last_fullsized":"2023-07-21T22:14:52.323571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:35:26.714623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248660+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:14:52.248660+0000","last_peered":"2023-07-21T22:14:52.248660+0000","last_clean":"2023-07-21T22:14:52.248660+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:14:52.248660+0000","last_undegraded":"2023-07-21T22:14:52.248660+0000","last_fullsized":"2023-07-21T22:14:52.248660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:27:40.710691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249020+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.249020+0000","last_peered":"2023-07-21T22:14:52.249020+0000","last_clean":"2023-07-21T22:14:52.249020+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:14:52.249020+0000","last_undegraded":"2023-07-21T22:14:52.249020+0000","last_fullsized":"2023-07-21T22:14:52.249020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:45.083477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323134+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:14:52.323134+0000","last_peered":"2023-07-21T22:14:52.323134+0000","last_clean":"2023-07-21T22:14:52.323134+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:14:52.323134+0000","last_undegraded":"2023-07-21T22:14:52.323134+0000","last_fullsized":"2023-07-21T22:14:52.323134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:19:31.888652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248874+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:14:52.248874+0000","last_peered":"2023-07-21T22:14:52.248874+0000","last_clean":"2023-07-21T22:14:52.248874+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:14:52.248874+0000","last_undegraded":"2023-07-21T22:14:52.248874+0000","last_fullsized":"2023-07-21T22:14:52.248874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:28.191291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23801,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":163,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":68954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1552384,"data_stored":1033922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.82399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.623},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77800000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.84699999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79700000000000004}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":423221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.628}]},{"osd":1,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.0049999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.032}]},{"osd":9,"last update":"Fri Jul 21 22:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.0920000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.081},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":1.0680000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1013433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86799999999999999,"5min":1.024,"15min":1.024},"last":0.755},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.91500000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.92900000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.879},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.378,"15min":0.371},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38500000000000001,"15min":0.378},"min":{"1min":0.308,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.373}]},{"osd":1,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59899999999999998,"15min":0.58499999999999996},"min":{"1min":0.498,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.34899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.34100000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.51800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60599999999999998,"15min":0.59499999999999997},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.621},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54500000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63200000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":424378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.38400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.41299999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.40100000000000002,"15min":0.371},"min":{"1min":0.33300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42699999999999999,"15min":0.39700000000000002},"min":{"1min":0.313,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.441}]},{"osd":5,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.40000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.56499999999999995,"15min":0.55100000000000005},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60399999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64200000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.86599999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67100000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.443,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.44800000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76600000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.436},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.432}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.39300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.45500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.55100000000000005,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.75,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.495},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56100000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54700000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.66000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":443619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.35899999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.68899999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.31900000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.498,"15min":0.496},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78500000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.63600000000000001,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.7210000000000001,"5min":4.7210000000000001,"15min":4.7210000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.439,"15min":0.439},"min":{"1min":0.30299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.75,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.745},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63300000000000001,"15min":0.63700000000000001},"min":{"1min":0.47899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":1.012},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.90100000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":1.0389999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.1200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.318},"min":{"1min":0.29499999999999998,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.52200000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.35899999999999999,"15min":0.32900000000000001},"min":{"1min":0.28499999999999998,"5min":0.19,"15min":0.19},"max":{"1min":0.57799999999999996,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73999999999999999,"5min":1.512,"15min":1.512},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.56000000000000005,"15min":0.52600000000000002},"min":{"1min":0.47199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89700000000000002,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.377,"15min":0.377},"max":{"1min":0.874,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92000000000000004,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56299999999999994},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95499999999999996,"5min":1.653,"15min":1.653},"last":0.376},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88800000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71799999999999997,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.59099999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:14:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.624},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953514,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.32700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.443},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.51400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94199999999999995,"5min":1.22,"15min":1.22},"last":0.623},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.46200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.89600000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.86799999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1515520,"data_stored":1015372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81000000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.434,"5min":0.313,"15min":0.313},"max":{"1min":0.66700000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.24299999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.373},"min":{"1min":0.26500000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.622}]},{"osd":4,"last update":"Fri Jul 21 22:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.42599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89500000000000002,"5min":1.47,"15min":1.47},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0169999999999999,"5min":1.53,"15min":1.53},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.78000000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":38113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":20617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:14:59.669 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-21T22:14:59.670 INFO:tasks.cephadm:Setting up client nodes... 2023-07-21T22:14:59.670 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-21T22:15:00.247 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:00.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:00 smithi194 ceph-mon[98288]: pgmap v256: 97 pgs: 97 active+clean; 600 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2023-07-21T22:15:00.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:00 smithi194 ceph-mon[98288]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:00.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:00 smithi194 ceph-mon[98288]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:00.598 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:00 smithi093 ceph-mon[96579]: pgmap v256: 97 pgs: 97 active+clean; 600 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2023-07-21T22:15:00.598 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:00 smithi093 ceph-mon[96579]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:00.598 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:00 smithi093 ceph-mon[96579]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:00.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:00 smithi134 ceph-mon[97873]: pgmap v256: 97 pgs: 97 active+clean; 600 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8.6 KiB/s wr, 22 op/s 2023-07-21T22:15:00.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:00 smithi134 ceph-mon[97873]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:00.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:00 smithi134 ceph-mon[97873]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:01.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 -- 172.21.15.93:0/2023611333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d4100080 msgr2=0x7f24d4100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:01.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 --2- 172.21.15.93:0/2023611333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d4100080 0x7f24d4100490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009990 tx=0x7f24c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 -- 172.21.15.93:0/2023611333 shutdown_connections 2023-07-21T22:15:01.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 --2- 172.21.15.93:0/2023611333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d40ff690 0x7f24d4106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 --2- 172.21.15.93:0/2023611333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d4100080 0x7f24d4100490 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 --2- 172.21.15.93:0/2023611333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40fece0 0x7f24d40ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 -- 172.21.15.93:0/2023611333 >> 172.21.15.93:0/2023611333 conn(0x7f24d40f9880 msgr2=0x7f24d40fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:01.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 -- 172.21.15.93:0/2023611333 shutdown_connections 2023-07-21T22:15:01.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.291+0000 7f24d9e79700 1 -- 172.21.15.93:0/2023611333 wait complete. 2023-07-21T22:15:01.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 Processor -- start 2023-07-21T22:15:01.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 -- start start 2023-07-21T22:15:01.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:01.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff690 0x7f24d419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:01.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 0x7f24d419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:01.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d419f5b0 con 0x7f24d40fece0 2023-07-21T22:15:01.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d419f720 con 0x7f24d4100080 2023-07-21T22:15:01.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d9e79700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d419f8e0 con 0x7f24d40ff690 2023-07-21T22:15:01.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:01.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42338/0 (socket says 172.21.15.93:42338) 2023-07-21T22:15:01.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d37fe700 1 -- 172.21.15.93:0/4659419 learned_addr learned my addr 172.21.15.93:0/4659419 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:01.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.292+0000 7f24d3fff700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 0x7f24d419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:01.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d2ffd700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff690 0x7f24d419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:01.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 -- 172.21.15.93:0/4659419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff690 msgr2=0x7f24d419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:01.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff690 0x7f24d419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 -- 172.21.15.93:0/4659419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 msgr2=0x7f24d419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:01.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 0x7f24d419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 -- 172.21.15.93:0/4659419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80095f0 con 0x7f24d40fece0 2023-07-21T22:15:01.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d3fff700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:01.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d37fe700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24c800b5c0 tx=0x7f24c8003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:01.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c801d070 con 0x7f24d40fece0 2023-07-21T22:15:01.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d419fb60 con 0x7f24d40fece0 2023-07-21T22:15:01.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c800f460 con 0x7f24d40fece0 2023-07-21T22:15:01.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8017650 con 0x7f24d40fece0 2023-07-21T22:15:01.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.293+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d419ffd0 con 0x7f24d40fece0 2023-07-21T22:15:01.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.294+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d4105990 con 0x7f24d40fece0 2023-07-21T22:15:01.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.295+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f24c800f5d0 con 0x7f24d40fece0 2023-07-21T22:15:01.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.295+0000 7f24d0ff9700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 0x7f24bc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:01.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.296+0000 7f24d2ffd700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 0x7f24bc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:01.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.296+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24c80cdd60 con 0x7f24d40fece0 2023-07-21T22:15:01.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.296+0000 7f24d2ffd700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 0x7f24bc0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24c4005ea0 tx=0x7f24c4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:01.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.300+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f24c8097490 con 0x7f24d40fece0 2023-07-21T22:15:01.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.561+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f24d402cb40 con 0x7f24d40fece0 2023-07-21T22:15:01.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.566+0000 7f24d0ff9700 1 -- 172.21.15.93:0/4659419 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f24c809aab0 con 0x7f24d40fece0 2023-07-21T22:15:01.609 INFO:teuthology.orchestra.run.smithi093.stdout:[client.0] 2023-07-21T22:15:01.610 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQBlA7tk7dKRIRAAFbgcWiN5WjETqsfihwa55g== 2023-07-21T22:15:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 msgr2=0x7f24bc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 0x7f24bc0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24c4005ea0 tx=0x7f24c4005e30 comp rx=0 tx=0).stop 2023-07-21T22:15:01.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 msgr2=0x7f24d40fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:01.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24c800b5c0 tx=0x7f24c8003cf0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 shutdown_connections 2023-07-21T22:15:01.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24bc0a1260 0x7f24bc0a3710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24d4100080 0x7f24d419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24d40fece0 0x7f24d40fe730 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 --2- 172.21.15.93:0/4659419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff690 0x7f24d419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:01.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 >> 172.21.15.93:0/4659419 conn(0x7f24d40f9880 msgr2=0x7f24d40fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:01.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 shutdown_connections 2023-07-21T22:15:01.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:01.568+0000 7f24d9e79700 1 -- 172.21.15.93:0/4659419 wait complete. 2023-07-21T22:15:02.001 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:15:02.001 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-21T22:15:02.002 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-21T22:15:02.044 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-21T22:15:02.045 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-21T22:15:02.045 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-21T22:15:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:02 smithi194 ceph-mon[98288]: pgmap v257: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2023-07-21T22:15:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:02 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:15:02.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:02 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:15:02.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:02 smithi134 ceph-mon[97873]: pgmap v257: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2023-07-21T22:15:02.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:02 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:15:02.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:02 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:15:02.634 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:02.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:02 smithi093 ceph-mon[96579]: pgmap v257: 97 pgs: 97 active+clean; 601 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 7.9 KiB/s wr, 22 op/s 2023-07-21T22:15:02.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:02 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:15:02.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:02 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4659419' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:15:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 -- 172.21.15.93:0/700754439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 msgr2=0x7f33dc10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 --2- 172.21.15.93:0/700754439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc10b8f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b0a0 tx=0x7f33d400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 -- 172.21.15.93:0/700754439 shutdown_connections 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 --2- 172.21.15.93:0/700754439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc10b8f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 --2- 172.21.15.93:0/700754439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc104ab0 0x7f33dc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 --2- 172.21.15.93:0/700754439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc0feab0 0x7f33dc0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 -- 172.21.15.93:0/700754439 >> 172.21.15.93:0/700754439 conn(0x7f33dc075b70 msgr2=0x7f33dc075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 -- 172.21.15.93:0/700754439 shutdown_connections 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.764+0000 7f33e2b30700 1 -- 172.21.15.93:0/700754439 wait complete. 2023-07-21T22:15:03.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 Processor -- start 2023-07-21T22:15:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 -- start start 2023-07-21T22:15:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc0feab0 0x7f33dc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:03.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc104ab0 0x7f33dc1949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc195010 con 0x7f33dc0ff460 2023-07-21T22:15:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc195180 con 0x7f33dc0feab0 2023-07-21T22:15:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e2b30700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc195340 con 0x7f33dc104ab0 2023-07-21T22:15:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:03.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42360/0 (socket says 172.21.15.93:42360) 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 -- 172.21.15.93:0/2371125906 learned_addr learned my addr 172.21.15.93:0/2371125906 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e10cd700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc104ab0 0x7f33dc1949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33e08cc700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc0feab0 0x7f33dc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 -- 172.21.15.93:0/2371125906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc104ab0 msgr2=0x7f33dc1949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc104ab0 0x7f33dc1949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 -- 172.21.15.93:0/2371125906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc0feab0 msgr2=0x7f33dc19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:03.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc0feab0 0x7f33dc19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 -- 172.21.15.93:0/2371125906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009d00 con 0x7f33dc0ff460 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.765+0000 7f33dbfff700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c9b0 tx=0x7f33c800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.766+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8007b00 con 0x7f33dc0ff460 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.766+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33c800ce50 con 0x7f33dc0ff460 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.766+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc195620 con 0x7f33dc0ff460 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.766+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8018660 con 0x7f33dc0ff460 2023-07-21T22:15:03.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.766+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc0681a0 con 0x7f33dc0ff460 2023-07-21T22:15:03.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.767+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc04e210 con 0x7f33dc0ff460 2023-07-21T22:15:03.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.768+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f33c800f450 con 0x7f33dc0ff460 2023-07-21T22:15:03.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.768+0000 7f33d9ffb700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 0x7f33cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:03.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.768+0000 7f33e08cc700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 0x7f33cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:03.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.769+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33c80cc830 con 0x7f33dc0ff460 2023-07-21T22:15:03.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.769+0000 7f33e08cc700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 0x7f33cc0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0ffbf0 tx=0x7f33d000f820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:03.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:03.773+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f33c8095fd0 con 0x7f33dc0ff460 2023-07-21T22:15:04.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.013+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f33dc068460 con 0x7f33dc0ff460 2023-07-21T22:15:04.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.017+0000 7f33d9ffb700 1 -- 172.21.15.93:0/2371125906 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260125 (secure 0 0 0) 0x7f33c8007e60 con 0x7f33dc0ff460 2023-07-21T22:15:04.064 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:04.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 msgr2=0x7f33cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:04.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 0x7f33cc0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0ffbf0 tx=0x7f33d000f820 comp rx=0 tx=0).stop 2023-07-21T22:15:04.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 msgr2=0x7f33dc19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:04.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c9b0 tx=0x7f33c800ccc0 comp rx=0 tx=0).stop 2023-07-21T22:15:04.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 shutdown_connections 2023-07-21T22:15:04.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33dc104ab0 0x7f33dc1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:04.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f33cc0a11e0 0x7f33cc0a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:04.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33dc0feab0 0x7f33dc19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:04.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 --2- 172.21.15.93:0/2371125906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33dc0ff460 0x7f33dc19ae60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:04.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 >> 172.21.15.93:0/2371125906 conn(0x7f33dc075b70 msgr2=0x7f33dc102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:04.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 shutdown_connections 2023-07-21T22:15:04.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:04.020+0000 7f33e2b30700 1 -- 172.21.15.93:0/2371125906 wait complete. 2023-07-21T22:15:04.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:04 smithi093 ceph-mon[96579]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2023-07-21T22:15:04.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:04 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2371125906' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:15:04.477 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":2721150511},{"type":"v1","addr":"172.21.15.93:6801","nonce":2721150511}]},"active_addr":"172.21.15.93:6801/2721150511","active_change":"2023-07-21T22:08:51.131214+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24104,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24115,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.93:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":3365813988}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":2153177670}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":503504133}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":2266779369}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":1882208449}]}]} 2023-07-21T22:15:04.484 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-21T22:15:04.485 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-21T22:15:04.485 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:15:04.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:04 smithi194 ceph-mon[98288]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2023-07-21T22:15:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:04 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2371125906' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:15:04.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:04 smithi134 ceph-mon[97873]: pgmap v258: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2023-07-21T22:15:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:04 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2371125906' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:15:05.035 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:06.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 -- 172.21.15.93:0/2765629415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff450 msgr2=0x7f2d8010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 --2- 172.21.15.93:0/2765629415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff450 0x7f2d8010b8e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800b0a0 tx=0x7f2d7800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 -- 172.21.15.93:0/2765629415 shutdown_connections 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 --2- 172.21.15.93:0/2765629415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff450 0x7f2d8010b8e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 --2- 172.21.15.93:0/2765629415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d80104aa0 0x7f2d80104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 --2- 172.21.15.93:0/2765629415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 0x7f2d800fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 -- 172.21.15.93:0/2765629415 >> 172.21.15.93:0/2765629415 conn(0x7f2d800fa5a0 msgr2=0x7f2d800fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:06.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 -- 172.21.15.93:0/2765629415 shutdown_connections 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.171+0000 7f2d853f5700 1 -- 172.21.15.93:0/2765629415 wait complete. 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 Processor -- start 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 -- start start 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 0x7f2d80194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 0x7f2d801952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80199d20 con 0x7f2d80104aa0 2023-07-21T22:15:06.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80199e90 con 0x7f2d800ff450 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d853f5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019a050 con 0x7f2d800feaa0 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42376/0 (socket says 172.21.15.93:42376) 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 -- 172.21.15.93:0/1844809635 learned_addr learned my addr 172.21.15.93:0/1844809635 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d77fff700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 0x7f2d801952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7effd700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 0x7f2d80194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:06.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 -- 172.21.15.93:0/1844809635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 msgr2=0x7f2d80194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 0x7f2d80194d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 -- 172.21.15.93:0/1844809635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 msgr2=0x7f2d801952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 0x7f2d801952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d7f7fe700 1 -- 172.21.15.93:0/1844809635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009d00 con 0x7f2d80104aa0 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.172+0000 7f2d77fff700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 0x7f2d801952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d7f7fe700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d780062a0 tx=0x7f2d78004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800e040 con 0x7f2d80104aa0 2023-07-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d78003c00 con 0x7f2d80104aa0 2023-07-21T22:15:06.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8019a2d0 con 0x7f2d80104aa0 2023-07-21T22:15:06.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7801a960 con 0x7f2d80104aa0 2023-07-21T22:15:06.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.173+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80068ba0 con 0x7f2d80104aa0 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.174+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d800681a0 con 0x7f2d80104aa0 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.175+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2d78003d70 con 0x7f2d80104aa0 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.175+0000 7f2d7cff9700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 0x7f2d6c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.175+0000 7f2d7effd700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 0x7f2d6c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.175+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d780cfe70 con 0x7f2d80104aa0 2023-07-21T22:15:06.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.176+0000 7f2d7effd700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 0x7f2d6c0a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:06.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.180+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2d780996a0 con 0x7f2d80104aa0 2023-07-21T22:15:06.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.371+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d8004e210 con 0x7f2d80104aa0 2023-07-21T22:15:06.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.372+0000 7f2d7cff9700 1 -- 172.21.15.93:0/1844809635 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f2d7809ccc0 con 0x7f2d80104aa0 2023-07-21T22:15:06.416 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:06.416 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:15:06.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 msgr2=0x7f2d6c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 0x7f2d6c0a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70008040 comp rx=0 tx=0).stop 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 msgr2=0x7f2d80199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d780062a0 tx=0x7f2d78004a50 comp rx=0 tx=0).stop 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 shutdown_connections 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d80104aa0 0x7f2d80199710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800feaa0 0x7f2d80194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2d6c0a11e0 0x7f2d6c0a3690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 --2- 172.21.15.93:0/1844809635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800ff450 0x7f2d801952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:06.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 >> 172.21.15.93:0/1844809635 conn(0x7f2d800fa5a0 msgr2=0x7f2d800fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:06.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 shutdown_connections 2023-07-21T22:15:06.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:06.374+0000 7f2d853f5700 1 -- 172.21.15.93:0/1844809635 wait complete. 2023-07-21T22:15:06.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:06 smithi194 ceph-mon[98288]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-21T22:15:06.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:06 smithi134 ceph-mon[97873]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-21T22:15:06.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:06 smithi093 ceph-mon[96579]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-21T22:15:06.787 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-21T22:15:06.788 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:15:07.340 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:07.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:07 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1844809635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:07.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:07 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1844809635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:07.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:07 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1844809635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:08.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:08 smithi093 ceph-mon[96579]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2023-07-21T22:15:08.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.504+0000 7f5520288700 1 -- 172.21.15.93:0/1253464107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5518104ab0 msgr2=0x7f5518104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.504+0000 7f5520288700 1 --2- 172.21.15.93:0/1253464107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5518104ab0 0x7f5518104e80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5514009990 tx=0x7f5514009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- 172.21.15.93:0/1253464107 shutdown_connections 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- 172.21.15.93:0/1253464107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55180ff460 0x7f551810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- 172.21.15.93:0/1253464107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5518104ab0 0x7f5518104e80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- 172.21.15.93:0/1253464107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55180feab0 0x7f55180fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- 172.21.15.93:0/1253464107 >> 172.21.15.93:0/1253464107 conn(0x7f5518075b70 msgr2=0x7f5518075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- 172.21.15.93:0/1253464107 shutdown_connections 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- 172.21.15.93:0/1253464107 wait complete. 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 Processor -- start 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- start start 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55180ff460 0x7f5518100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5518104ab0 0x7f5518101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:08.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.505+0000 7f5520288700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55181a3fc0 con 0x7f55180feab0 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f5520288700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55181a4130 con 0x7f55180ff460 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f5520288700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55181a42f0 con 0x7f5518104ab0 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42384/0 (socket says 172.21.15.93:42384) 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 -- 172.21.15.93:0/3270910214 learned_addr learned my addr 172.21.15.93:0/3270910214 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e825700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5518104ab0 0x7f5518101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:08.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 -- 172.21.15.93:0/3270910214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5518104ab0 msgr2=0x7f5518101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5518104ab0 0x7f5518101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 -- 172.21.15.93:0/3270910214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55180ff460 msgr2=0x7f5518100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55180ff460 0x7f5518100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 -- 172.21.15.93:0/3270910214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140095f0 con 0x7f55180feab0 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f551e024700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55140098e0 tx=0x7f551400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551401d070 con 0x7f55180feab0 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5514005510 con 0x7f55180feab0 2023-07-21T22:15:08.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55181a4570 con 0x7f55180feab0 2023-07-21T22:15:08.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514004c90 con 0x7f55180feab0 2023-07-21T22:15:08.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.506+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55181a4980 con 0x7f55180feab0 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.507+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551804e210 con 0x7f55180feab0 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.508+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f551400fc10 con 0x7f55180feab0 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.509+0000 7f550f7fe700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 0x7f55040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.509+0000 7f551d823700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 0x7f55040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.509+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55140cda30 con 0x7f55180feab0 2023-07-21T22:15:08.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.509+0000 7f551d823700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 0x7f55040a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5508005950 tx=0x7f55080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:08.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.512+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5514097160 con 0x7f55180feab0 2023-07-21T22:15:08.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:08 smithi134 ceph-mon[97873]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2023-07-21T22:15:08.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.699+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5518101bf0 con 0x7f55180feab0 2023-07-21T22:15:08.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.701+0000 7f550f7fe700 1 -- 172.21.15.93:0/3270910214 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f5514030090 con 0x7f55180feab0 2023-07-21T22:15:08.745 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:08.745 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:15:08.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 msgr2=0x7f55040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:08.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 0x7f55040a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5508005950 tx=0x7f55080058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 msgr2=0x7f55181006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55140098e0 tx=0x7f551400f720 comp rx=0 tx=0).stop 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 shutdown_connections 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5518104ab0 0x7f5518101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55180feab0 0x7f55181006c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f55040a11e0 0x7f55040a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 --2- 172.21.15.93:0/3270910214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55180ff460 0x7f5518100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:08.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 >> 172.21.15.93:0/3270910214 conn(0x7f5518075b70 msgr2=0x7f55180fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:08.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 shutdown_connections 2023-07-21T22:15:08.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:08.703+0000 7f5520288700 1 -- 172.21.15.93:0/3270910214 wait complete. 2023-07-21T22:15:08.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:08 smithi194 ceph-mon[98288]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2023-07-21T22:15:09.083 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-21T22:15:09.084 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-21T22:15:09.084 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-21T22:15:09.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-21T22:15:09.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-21T22:15:09.086 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-21T22:15:09.086 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-21T22:15:09.087 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-21T22:15:09.087 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-21T22:15:09.088 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-21T22:15:09.088 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-21T22:15:09.089 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-21T22:15:09.363 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3270910214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:09.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:09 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3270910214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:09.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:09 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3270910214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:09.904 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.083 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.084 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.085 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.182 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.459 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.460 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.462 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:10.551 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:10 smithi093 ceph-mon[96579]: pgmap v261: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2023-07-21T22:15:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:10 smithi194 ceph-mon[98288]: pgmap v261: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2023-07-21T22:15:10.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:10 smithi134 ceph-mon[97873]: pgmap v261: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2023-07-21T22:15:11.006 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:12.657 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:12.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:12 smithi093 ceph-mon[96579]: pgmap v262: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2023-07-21T22:15:12.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:12 smithi194 ceph-mon[98288]: pgmap v262: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2023-07-21T22:15:12.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:12 smithi134 ceph-mon[97873]: pgmap v262: 97 pgs: 97 active+clean; 631 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2023-07-21T22:15:13.141 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:13.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/972956495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d40ff4b0 msgr2=0x7fc6d410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/972956495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d40ff4b0 0x7fc6d410b940 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00b0a0 tx=0x7fc6cc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/972956495 shutdown_connections 2023-07-21T22:15:13.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/972956495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d40ff4b0 0x7fc6d410b940 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/972956495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4104b00 0x7fc6d4104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/972956495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6d40feb00 0x7fc6d40fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/972956495 >> 172.21.15.93:0/972956495 conn(0x7fc6d40759b0 msgr2=0x7fc6d4075db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/972956495 shutdown_connections 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.470+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/972956495 wait complete. 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 Processor -- start 2023-07-21T22:15:13.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 -- start start 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6d40feb00 0x7fc6d4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d40ff4b0 0x7fc6d41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 0x7fc6d419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d419ce40 con 0x7fc6d4104b00 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d419cfb0 con 0x7fc6d40feb00 2023-07-21T22:15:13.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d8f7a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d419d170 con 0x7fc6d40ff4b0 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 0x7fc6d419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 0x7fc6d419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42402/0 (socket says 172.21.15.93:42402) 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d2d9d700 1 -- 172.21.15.93:0/757188360 learned_addr learned my addr 172.21.15.93:0/757188360 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d259c700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6d40feb00 0x7fc6d4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.471+0000 7fc6d1d9b700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d40ff4b0 0x7fc6d41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 -- 172.21.15.93:0/757188360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d40ff4b0 msgr2=0x7fc6d41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d40ff4b0 0x7fc6d41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 -- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6d40feb00 msgr2=0x7fc6d4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6d40feb00 0x7fc6d4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009d00 con 0x7fc6d4104b00 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d2d9d700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 0x7fc6d419c830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0062a0 tx=0x7fc6cc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00e040 con 0x7fc6d4104b00 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6cc004020 con 0x7fc6d4104b00 2023-07-21T22:15:13.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d419d3f0 con 0x7fc6d4104b00 2023-07-21T22:15:13.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc004ea0 con 0x7fc6d4104b00 2023-07-21T22:15:13.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.472+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d41a4cb0 con 0x7fc6d4104b00 2023-07-21T22:15:13.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.473+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6d404e210 con 0x7fc6d4104b00 2023-07-21T22:15:13.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.475+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc6cc003a00 con 0x7fc6d4104b00 2023-07-21T22:15:13.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.475+0000 7fc6c37fe700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc6bc0a11e0 0x7fc6bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.475+0000 7fc6d259c700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc6bc0a11e0 0x7fc6bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.475+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6cc0ce200 con 0x7fc6d4104b00 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6c37fe700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] conn(0x7fc6bc0aa9b0 0x7fc6bc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6bc0ad470 con 0x7fc6bc0aa9b0 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6cc0ce6a0 con 0x7fc6d4104b00 2023-07-21T22:15:13.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6d259c700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc6bc0a11e0 0x7fc6bc0a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005950 tx=0x7fc6c400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:13.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6d1d9b700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] conn(0x7fc6bc0aa9b0 0x7fc6bc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:13.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.476+0000 7fc6d1d9b700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] conn(0x7fc6bc0aa9b0 0x7fc6bc0acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:13.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.477+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== osd.10 v2:172.21.15.134:6824/51617631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc6bc0ad470 con 0x7fc6bc0aa9b0 2023-07-21T22:15:13.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.494+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 --> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc6d410ed00 con 0x7fc6bc0aa9b0 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.495+0000 7fc6c37fe700 1 -- 172.21.15.93:0/757188360 <== osd.10 v2:172.21.15.134:6824/51617631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc6c800a8a0 con 0x7fc6bc0aa9b0 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.495+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] conn(0x7fc6bc0aa9b0 msgr2=0x7fc6bc0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.495+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] conn(0x7fc6bc0aa9b0 0x7fc6bc0acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc6bc0a11e0 msgr2=0x7fc6bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc6bc0a11e0 0x7fc6bc0a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005950 tx=0x7fc6c400a300 comp rx=0 tx=0).stop 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 msgr2=0x7fc6d419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:13.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 --2- 172.21.15.93:0/757188360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6d4104b00 0x7fc6d419c830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0062a0 tx=0x7fc6cc00cf70 comp rx=0 tx=0).stop 2023-07-21T22:15:13.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d2d9d700 1 -- 172.21.15.93:0/757188360 reap_dead start 2023-07-21T22:15:13.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 shutdown_connections 2023-07-21T22:15:13.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 >> 172.21.15.93:0/757188360 conn(0x7fc6d40759b0 msgr2=0x7fc6d4109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:13.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 shutdown_connections 2023-07-21T22:15:13.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:13.496+0000 7fc6d8f7a700 1 -- 172.21.15.93:0/757188360 wait complete. 2023-07-21T22:15:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 -- 172.21.15.93:0/4149943847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15100feab0 msgr2=0x7f15100fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 --2- 172.21.15.93:0/4149943847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15100feab0 0x7f15100fef20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1500009990 tx=0x7f1500009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 -- 172.21.15.93:0/4149943847 shutdown_connections 2023-07-21T22:15:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 --2- 172.21.15.93:0/4149943847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15100ff460 0x7f151010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 --2- 172.21.15.93:0/4149943847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1510104ab0 0x7f1510104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 --2- 172.21.15.93:0/4149943847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15100feab0 0x7f15100fef20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 -- 172.21.15.93:0/4149943847 >> 172.21.15.93:0/4149943847 conn(0x7f1510075b70 msgr2=0x7f1510075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 -- 172.21.15.93:0/4149943847 shutdown_connections 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.210+0000 7f151726d700 1 -- 172.21.15.93:0/4149943847 wait complete. 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 Processor -- start 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 -- start start 2023-07-21T22:15:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15100feab0 0x7f1510196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15100ff460 0x7f1510196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 0x7f15101a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1510197890 con 0x7f1510104ab0 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101a1fd0 con 0x7f15100feab0 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151726d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101a2190 con 0x7f15100ff460 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 0x7f15101a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 0x7f15101a1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42418/0 (socket says 172.21.15.93:42418) 2023-07-21T22:15:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 -- 172.21.15.93:0/675369479 learned_addr learned my addr 172.21.15.93:0/675369479 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f1515009700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15100feab0 0x7f1510196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f1514808700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15100ff460 0x7f1510196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 -- 172.21.15.93:0/675369479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15100ff460 msgr2=0x7f1510196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15100ff460 0x7f1510196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 -- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15100feab0 msgr2=0x7f1510196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15100feab0 0x7f1510196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.211+0000 7f151580a700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000095f0 con 0x7f1510104ab0 2023-07-21T22:15:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f151580a700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 0x7f15101a1a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1508007f10 tx=0x7f150800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508017070 con 0x7f1510104ab0 2023-07-21T22:15:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15101a2470 con 0x7f1510104ab0 2023-07-21T22:15:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f150800f040 con 0x7f1510104ab0 2023-07-21T22:15:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508005740 con 0x7f1510104ab0 2023-07-21T22:15:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.212+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15101a2920 con 0x7f1510104ab0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.213+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f151004e210 con 0x7f1510104ab0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.214+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f150800dc60 con 0x7f1510104ab0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.214+0000 7f15067fc700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f14fc0a11e0 0x7f14fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.214+0000 7f1515009700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f14fc0a11e0 0x7f14fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.214+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f15080cc810 con 0x7f1510104ab0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f15067fc700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] conn(0x7f14fc0aa9b0 0x7f14fc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14fc0ad470 con 0x7f14fc0aa9b0 2023-07-21T22:15:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15080ccb80 con 0x7f1510104ab0 2023-07-21T22:15:14.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f1515009700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f14fc0a11e0 0x7f14fc0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15100ffbf0 tx=0x7f150c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f1514808700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] conn(0x7f14fc0aa9b0 0x7f14fc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.215+0000 7f1514808700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] conn(0x7f14fc0aa9b0 0x7f14fc0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.217+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== osd.4 v2:172.21.15.134:6808/2960934129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f14fc0ad470 con 0x7f14fc0aa9b0 2023-07-21T22:15:14.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.233+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 --> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f151010ecb0 con 0x7f14fc0aa9b0 2023-07-21T22:15:14.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f15067fc700 1 -- 172.21.15.93:0/675369479 <== osd.4 v2:172.21.15.134:6808/2960934129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f150001e040 con 0x7f14fc0aa9b0 2023-07-21T22:15:14.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] conn(0x7f14fc0aa9b0 msgr2=0x7f14fc0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] conn(0x7f14fc0aa9b0 0x7f14fc0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f14fc0a11e0 msgr2=0x7f14fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f14fc0a11e0 0x7f14fc0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15100ffbf0 tx=0x7f150c00b410 comp rx=0 tx=0).stop 2023-07-21T22:15:14.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 msgr2=0x7f15101a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 --2- 172.21.15.93:0/675369479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1510104ab0 0x7f15101a1a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1508007f10 tx=0x7f150800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151580a700 1 -- 172.21.15.93:0/675369479 reap_dead start 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 shutdown_connections 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 >> 172.21.15.93:0/675369479 conn(0x7f1510075b70 msgr2=0x7f15100fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 shutdown_connections 2023-07-21T22:15:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.234+0000 7f151726d700 1 -- 172.21.15.93:0/675369479 wait complete. 2023-07-21T22:15:14.579 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:14 smithi093 ceph-mon[96579]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-21T22:15:14.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 -- 172.21.15.93:0/3533525114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3134101790 msgr2=0x7f3134109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 --2- 172.21.15.93:0/3533525114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3134101790 0x7f3134109c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3120009990 tx=0x7f3120009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 -- 172.21.15.93:0/3533525114 shutdown_connections 2023-07-21T22:15:14.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 --2- 172.21.15.93:0/3533525114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f313410a270 0x7f313410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 --2- 172.21.15.93:0/3533525114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134100df0 0x7f31341011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 --2- 172.21.15.93:0/3533525114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3134101790 0x7f3134109c00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 -- 172.21.15.93:0/3533525114 >> 172.21.15.93:0/3533525114 conn(0x7f3134075b60 msgr2=0x7f3134075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:14.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 -- 172.21.15.93:0/3533525114 shutdown_connections 2023-07-21T22:15:14.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.773+0000 7f313b1d3700 1 -- 172.21.15.93:0/3533525114 wait complete. 2023-07-21T22:15:14.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 Processor -- start 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 -- start start 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134100df0 0x7f3134195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134101790 0x7f3134196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 0x7f313419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419acd0 con 0x7f313410a270 2023-07-21T22:15:14.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419ae40 con 0x7f3134100df0 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.774+0000 7f313b1d3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419b000 con 0x7f3134101790 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 0x7f313419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 0x7f313419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42434/0 (socket says 172.21.15.93:42434) 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 -- 172.21.15.93:0/340432774 learned_addr learned my addr 172.21.15.93:0/340432774 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3138f6f700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134100df0 0x7f3134195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3133fff700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134101790 0x7f3134196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 -- 172.21.15.93:0/340432774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134101790 msgr2=0x7f3134196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134101790 0x7f3134196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 -- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134100df0 msgr2=0x7f3134195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134100df0 0x7f3134195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31200095f0 con 0x7f313410a270 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3133fff700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134101790 0x7f3134196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3139770700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 0x7f313419a6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f312c007ee0 tx=0x7f312c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c017070 con 0x7f313410a270 2023-07-21T22:15:14.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312c00f040 con 0x7f313410a270 2023-07-21T22:15:14.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313419b2e0 con 0x7f313410a270 2023-07-21T22:15:14.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.775+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c005740 con 0x7f313410a270 2023-07-21T22:15:14.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.776+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31341a2b40 con 0x7f313410a270 2023-07-21T22:15:14.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.776+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f313410fed0 con 0x7f313410a270 2023-07-21T22:15:14.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.777+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f312c00dc60 con 0x7f313410a270 2023-07-21T22:15:14.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3131ffb700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31240a1260 0x7f31240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f312c0cc630 con 0x7f313410a270 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3138f6f700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31240a1260 0x7f31240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3131ffb700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] conn(0x7f31240aaa30 0x7f31240ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31240ad4f0 con 0x7f31240aaa30 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312c0ccad0 con 0x7f313410a270 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.778+0000 7f3138f6f700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31240a1260 0x7f31240a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3128005950 tx=0x7f312800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.779+0000 7f3133fff700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] conn(0x7f31240aaa30 0x7f31240ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.779+0000 7f3133fff700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] conn(0x7f31240aaa30 0x7f31240ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.780+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== osd.7 v2:172.21.15.134:6816/4117548000 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f31240ad4f0 con 0x7f31240aaa30 2023-07-21T22:15:14.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:14 smithi194 ceph-mon[98288]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-21T22:15:14.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.795+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 --> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f313404e210 con 0x7f31240aaa30 2023-07-21T22:15:14.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.796+0000 7f3131ffb700 1 -- 172.21.15.93:0/340432774 <== osd.7 v2:172.21.15.134:6816/4117548000 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f313404e210 con 0x7f31240aaa30 2023-07-21T22:15:14.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.796+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] conn(0x7f31240aaa30 msgr2=0x7f31240ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.796+0000 7f313b1d3700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] conn(0x7f31240aaa30 0x7f31240ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31240a1260 msgr2=0x7f31240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31240a1260 0x7f31240a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3128005950 tx=0x7f312800b410 comp rx=0 tx=0).stop 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 msgr2=0x7f313419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 --2- 172.21.15.93:0/340432774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f313410a270 0x7f313419a6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f312c007ee0 tx=0x7f312c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f3139770700 1 -- 172.21.15.93:0/340432774 reap_dead start 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 shutdown_connections 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 >> 172.21.15.93:0/340432774 conn(0x7f3134075b60 msgr2=0x7f31340fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:14.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 shutdown_connections 2023-07-21T22:15:14.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:14.797+0000 7f313b1d3700 1 -- 172.21.15.93:0/340432774 wait complete. 2023-07-21T22:15:14.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:14 smithi134 ceph-mon[97873]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-21T22:15:15.597 INFO:teuthology.orchestra.run.smithi093.stdout:296352743444 2023-07-21T22:15:15.598 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-21T22:15:16.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 -- 172.21.15.93:0/925311330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c8104ab0 msgr2=0x7f24c8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 --2- 172.21.15.93:0/925311330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c8104ab0 0x7f24c8104e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009990 tx=0x7f24b8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 -- 172.21.15.93:0/925311330 shutdown_connections 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 --2- 172.21.15.93:0/925311330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24c80ff460 0x7f24c810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 --2- 172.21.15.93:0/925311330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c8104ab0 0x7f24c8104e80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 --2- 172.21.15.93:0/925311330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c80feab0 0x7f24c80fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 -- 172.21.15.93:0/925311330 >> 172.21.15.93:0/925311330 conn(0x7f24c8075b70 msgr2=0x7f24c8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 -- 172.21.15.93:0/925311330 shutdown_connections 2023-07-21T22:15:16.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.083+0000 7f24cf47d700 1 -- 172.21.15.93:0/925311330 wait complete. 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 Processor -- start 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 -- start start 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c80feab0 0x7f24c8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 0x7f24c8196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24c8104ab0 0x7f24c81a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81978f0 con 0x7f24c80ff460 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81a2060 con 0x7f24c80feab0 2023-07-21T22:15:16.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cf47d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c81a21d0 con 0x7f24c8104ab0 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cca18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 0x7f24c8196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cca18700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 0x7f24c8196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42446/0 (socket says 172.21.15.93:42446) 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cca18700 1 -- 172.21.15.93:0/2890786712 learned_addr learned my addr 172.21.15.93:0/2890786712 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cd219700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c80feab0 0x7f24c8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.084+0000 7f24cca18700 1 -- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24c8104ab0 msgr2=0x7f24c81a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cca18700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24c8104ab0 0x7f24c81a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cca18700 1 -- 172.21.15.93:0/2890786712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c80feab0 msgr2=0x7f24c8196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cca18700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24c80feab0 0x7f24c8196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cca18700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80095f0 con 0x7f24c80ff460 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cca18700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 0x7f24c8196fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c920 tx=0x7f24c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4007b00 con 0x7f24c80ff460 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c4004cf0 con 0x7f24c80ff460 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c81a24b0 con 0x7f24c80ff460 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4005610 con 0x7f24c80ff460 2023-07-21T22:15:16.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.085+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c81a2a00 con 0x7f24c80ff460 2023-07-21T22:15:16.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.086+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24c804e210 con 0x7f24c80ff460 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.087+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f24c4004750 con 0x7f24c80ff460 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24be7fc700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24b40a1260 0x7f24b40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24cd219700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24b40a1260 0x7f24b40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24c40cca10 con 0x7f24c80ff460 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24be7fc700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] conn(0x7f24b40aaa30 0x7f24b40ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:16.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24b40ad4f0 con 0x7f24b40aaa30 2023-07-21T22:15:16.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c40cceb0 con 0x7f24c80ff460 2023-07-21T22:15:16.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24cd219700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24b40a1260 0x7f24b40a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24b8006010 tx=0x7f24b800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:16.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.088+0000 7f24cda1a700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] conn(0x7f24b40aaa30 0x7f24b40ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:16.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.089+0000 7f24cda1a700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] conn(0x7f24b40aaa30 0x7f24b40ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:16.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.089+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== osd.5 v2:172.21.15.194:6808/1989091547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f24b40ad4f0 con 0x7f24b40aaa30 2023-07-21T22:15:16.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.106+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 --> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f24c810ec50 con 0x7f24b40aaa30 2023-07-21T22:15:16.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.106+0000 7f24be7fc700 1 -- 172.21.15.93:0/2890786712 <== osd.5 v2:172.21.15.194:6808/1989091547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f24c0012040 con 0x7f24b40aaa30 2023-07-21T22:15:16.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] conn(0x7f24b40aaa30 msgr2=0x7f24b40ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] conn(0x7f24b40aaa30 0x7f24b40ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24b40a1260 msgr2=0x7f24b40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f24b40a1260 0x7f24b40a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24b8006010 tx=0x7f24b800b540 comp rx=0 tx=0).stop 2023-07-21T22:15:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 msgr2=0x7f24c8196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 --2- 172.21.15.93:0/2890786712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80ff460 0x7f24c8196fe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24c400c920 tx=0x7f24c400cce0 comp rx=0 tx=0).stop 2023-07-21T22:15:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cda1a700 1 -- 172.21.15.93:0/2890786712 reap_dead start 2023-07-21T22:15:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 shutdown_connections 2023-07-21T22:15:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 >> 172.21.15.93:0/2890786712 conn(0x7f24c8075b70 msgr2=0x7f24c80fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 shutdown_connections 2023-07-21T22:15:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.107+0000 7f24cf47d700 1 -- 172.21.15.93:0/2890786712 wait complete. 2023-07-21T22:15:16.547 INFO:teuthology.orchestra.run.smithi093.stdout:137438953518 2023-07-21T22:15:16.547 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-21T22:15:16.681 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:16 smithi093 ceph-mon[96579]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T22:15:16.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:16 smithi194 ceph-mon[98288]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T22:15:16.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:16 smithi134 ceph-mon[97873]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-07-21T22:15:17.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.974+0000 7f582897a700 1 -- 172.21.15.93:0/3049952523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f582410c490 msgr2=0x7f582410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.974+0000 7f582897a700 1 --2- 172.21.15.93:0/3049952523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f582410c490 0x7f582410e870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f581c00b0a0 tx=0x7f581c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 -- 172.21.15.93:0/3049952523 shutdown_connections 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 --2- 172.21.15.93:0/3049952523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f582410c490 0x7f582410e870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 --2- 172.21.15.93:0/3049952523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5824072850 0x7f5824072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 --2- 172.21.15.93:0/3049952523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824073160 0x7f582410bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 -- 172.21.15.93:0/3049952523 >> 172.21.15.93:0/3049952523 conn(0x7f58240fb710 msgr2=0x7f58240fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 -- 172.21.15.93:0/3049952523 shutdown_connections 2023-07-21T22:15:17.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 -- 172.21.15.93:0/3049952523 wait complete. 2023-07-21T22:15:17.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.975+0000 7f582897a700 1 Processor -- start 2023-07-21T22:15:17.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 -- start start 2023-07-21T22:15:17.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824072850 0x7f5824197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 0x7f5824198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f582410c490 0x7f582419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582419cf00 con 0x7f5824073160 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582419d070 con 0x7f5824072850 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582897a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582419d230 con 0x7f582410c490 2023-07-21T22:15:17.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 0x7f5824198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 0x7f5824198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42462/0 (socket says 172.21.15.93:42462) 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 -- 172.21.15.93:0/3289369926 learned_addr learned my addr 172.21.15.93:0/3289369926 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5822d9d700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f582410c490 0x7f582419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582259c700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824072850 0x7f5824197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 -- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f582410c490 msgr2=0x7f582419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f582410c490 0x7f582419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 -- 172.21.15.93:0/3289369926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824072850 msgr2=0x7f5824197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824072850 0x7f5824197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c009d00 con 0x7f5824073160 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f582259c700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5824072850 0x7f5824197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.976+0000 7f5821d9b700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 0x7f5824198480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f581800d950 tx=0x7f581800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.977+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818009b40 con 0x7f5824073160 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.977+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582419d510 con 0x7f5824073160 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.977+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58241a4d70 con 0x7f5824073160 2023-07-21T22:15:17.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.977+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5818009ca0 con 0x7f5824073160 2023-07-21T22:15:17.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.977+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581800f620 con 0x7f5824073160 2023-07-21T22:15:17.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.978+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5824111f30 con 0x7f5824073160 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5818010460 con 0x7f5824073160 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f580c0a11e0 0x7f580c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f582259c700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f580c0a11e0 0x7f580c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58180ccc00 con 0x7f5824073160 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] conn(0x7f580c0aa9b0 0x7f580c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f580c0ad470 con 0x7f580c0aa9b0 2023-07-21T22:15:17.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.979+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58180cd0a0 con 0x7f5824073160 2023-07-21T22:15:17.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.980+0000 7f582259c700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f580c0a11e0 0x7f580c0a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f5814005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.980+0000 7f5822d9d700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] conn(0x7f580c0aa9b0 0x7f580c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.980+0000 7f5822d9d700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] conn(0x7f580c0aa9b0 0x7f580c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.981+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== osd.11 v2:172.21.15.194:6824/4036125050 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f580c0ad470 con 0x7f580c0aa9b0 2023-07-21T22:15:17.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.998+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 --> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f582404e210 con 0x7f580c0aa9b0 2023-07-21T22:15:17.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f58137fe700 1 -- 172.21.15.93:0/3289369926 <== osd.11 v2:172.21.15.194:6824/4036125050 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f582404e210 con 0x7f580c0aa9b0 2023-07-21T22:15:17.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] conn(0x7f580c0aa9b0 msgr2=0x7f580c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] conn(0x7f580c0aa9b0 0x7f580c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f580c0a11e0 msgr2=0x7f580c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f580c0a11e0 0x7f580c0a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5814005fd0 tx=0x7f5814005ee0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 msgr2=0x7f5824198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 --2- 172.21.15.93:0/3289369926 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5824073160 0x7f5824198480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f581800d950 tx=0x7f581800dd10 comp rx=0 tx=0).stop 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5822d9d700 1 -- 172.21.15.93:0/3289369926 reap_dead start 2023-07-21T22:15:17.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 shutdown_connections 2023-07-21T22:15:17.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 >> 172.21.15.93:0/3289369926 conn(0x7f58240fb710 msgr2=0x7f5824103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 shutdown_connections 2023-07-21T22:15:17.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f582897a700 1 -- 172.21.15.93:0/3289369926 wait complete. 2023-07-21T22:15:17.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 -- 172.21.15.93:0/3024120940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 msgr2=0x7f5e58109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 --2- 172.21.15.93:0/3024120940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58109c60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009990 tx=0x7f5e48009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 -- 172.21.15.93:0/3024120940 shutdown_connections 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 --2- 172.21.15.93:0/3024120940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e5810a2d0 0x7f5e5810c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 --2- 172.21.15.93:0/3024120940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e58100e50 0x7f5e58101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 --2- 172.21.15.93:0/3024120940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58109c60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.997+0000 7f5e606b6700 1 -- 172.21.15.93:0/3024120940 >> 172.21.15.93:0/3024120940 conn(0x7f5e58075b60 msgr2=0x7f5e58075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.998+0000 7f5e606b6700 1 -- 172.21.15.93:0/3024120940 shutdown_connections 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.998+0000 7f5e606b6700 1 -- 172.21.15.93:0/3024120940 wait complete. 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.998+0000 7f5e606b6700 1 Processor -- start 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.998+0000 7f5e606b6700 1 -- start start 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e58100e50 0x7f5e58195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e5810a2d0 0x7f5e5819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819ad10 con 0x7f5e581017f0 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819ae80 con 0x7f5e58100e50 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e606b6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5819b040 con 0x7f5e5810a2d0 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42478/0 (socket says 172.21.15.93:42478) 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 -- 172.21.15.93:0/387844504 learned_addr learned my addr 172.21.15.93:0/387844504 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5ec53700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e5810a2d0 0x7f5e5819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5e452700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e58100e50 0x7f5e58195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 -- 172.21.15.93:0/387844504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e5810a2d0 msgr2=0x7f5e5819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e5810a2d0 0x7f5e5819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 -- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e58100e50 msgr2=0x7f5e58195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e58100e50 0x7f5e58195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480095f0 con 0x7f5e581017f0 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:16.999+0000 7f5e5dc51700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58196290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e48009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4801d070 con 0x7f5e581017f0 2023-07-21T22:15:17.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e48022470 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5819b2c0 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48017650 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e581a2b80 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.000+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e5804e210 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.002+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5e48022870 con 0x7f5e581017f0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.003+0000 7f5e4f7fe700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5e440a1260 0x7f5e440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.003+0000 7f5e5e452700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5e440a1260 0x7f5e440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e480ce2b0 con 0x7f5e581017f0 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e4f7fe700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] conn(0x7f5e440aaa30 0x7f5e440ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e440ad4f0 con 0x7f5e440aaa30 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e480ce750 con 0x7f5e581017f0 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e5e452700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5e440a1260 0x7f5e440a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009bd0 tx=0x7f5e54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.004+0000 7f5e5ec53700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] conn(0x7f5e440aaa30 0x7f5e440ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.005+0000 7f5e5ec53700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] conn(0x7f5e440aaa30 0x7f5e440ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.005+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== osd.1 v2:172.21.15.134:6800/2449999795 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e440ad4f0 con 0x7f5e440aaa30 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.021+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 --> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e5810fee0 con 0x7f5e440aaa30 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e4f7fe700 1 -- 172.21.15.93:0/387844504 <== osd.1 v2:172.21.15.134:6800/2449999795 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e5000f040 con 0x7f5e440aaa30 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] conn(0x7f5e440aaa30 msgr2=0x7f5e440ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] conn(0x7f5e440aaa30 0x7f5e440ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5e440a1260 msgr2=0x7f5e440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5e440a1260 0x7f5e440a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009bd0 tx=0x7f5e54009260 comp rx=0 tx=0).stop 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 msgr2=0x7f5e58196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 --2- 172.21.15.93:0/387844504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e581017f0 0x7f5e58196290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e48009580 comp rx=0 tx=0).stop 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e5ec53700 1 -- 172.21.15.93:0/387844504 reap_dead start 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 shutdown_connections 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 >> 172.21.15.93:0/387844504 conn(0x7f5e58075b60 msgr2=0x7f5e5810ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 shutdown_connections 2023-07-21T22:15:17.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.022+0000 7f5e606b6700 1 -- 172.21.15.93:0/387844504 wait complete. 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 -- 172.21.15.93:0/1168173428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 msgr2=0x7f4a240fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 --2- 172.21.15.93:0/1168173428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a240fef20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009990 tx=0x7f4a18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 -- 172.21.15.93:0/1168173428 shutdown_connections 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 --2- 172.21.15.93:0/1168173428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a240ff460 0x7f4a2410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 --2- 172.21.15.93:0/1168173428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a24104ab0 0x7f4a24104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 --2- 172.21.15.93:0/1168173428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a240fef20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 -- 172.21.15.93:0/1168173428 >> 172.21.15.93:0/1168173428 conn(0x7f4a24075b70 msgr2=0x7f4a24075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 -- 172.21.15.93:0/1168173428 shutdown_connections 2023-07-21T22:15:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.073+0000 7f4a291b7700 1 -- 172.21.15.93:0/1168173428 wait complete. 2023-07-21T22:15:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.074+0000 7f4a291b7700 1 Processor -- start 2023-07-21T22:15:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.074+0000 7f4a291b7700 1 -- start start 2023-07-21T22:15:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a2419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240ff460 0x7f4a2419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a24104ab0 0x7f4a24194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24194ff0 con 0x7f4a240feab0 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24195160 con 0x7f4a240ff460 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a291b7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24195320 con 0x7f4a24104ab0 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a2419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a2419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42492/0 (socket says 172.21.15.93:42492) 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 -- 172.21.15.93:0/4149809870 learned_addr learned my addr 172.21.15.93:0/4149809870 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a2259c700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240ff460 0x7f4a2419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a2359e700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a24104ab0 0x7f4a24194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 -- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a24104ab0 msgr2=0x7f4a24194980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a24104ab0 0x7f4a24194980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 -- 172.21.15.93:0/4149809870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240ff460 msgr2=0x7f4a2419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a240ff460 0x7f4a2419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180095f0 con 0x7f4a240feab0 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a2359e700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a24104ab0 0x7f4a24194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.075+0000 7f4a22d9d700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a2419a900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a140098f0 tx=0x7f4a14009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14012070 con 0x7f4a240feab0 2023-07-21T22:15:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a24195600 con 0x7f4a240feab0 2023-07-21T22:15:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a140044e0 con 0x7f4a240feab0 2023-07-21T22:15:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14007d10 con 0x7f4a240feab0 2023-07-21T22:15:17.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240681a0 con 0x7f4a240feab0 2023-07-21T22:15:17.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.076+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a2404e210 con 0x7f4a240feab0 2023-07-21T22:15:17.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.078+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4a14016030 con 0x7f4a240feab0 2023-07-21T22:15:17.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a0bfff700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4a0c0a11e0 0x7f4a0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a2259c700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4a0c0a11e0 0x7f4a0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4a140ce150 con 0x7f4a240feab0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a0bfff700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] conn(0x7f4a0c0aa9b0 0x7f4a0c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a0c0ad470 con 0x7f4a0c0aa9b0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a140ce5f0 con 0x7f4a240feab0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a2259c700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4a0c0a11e0 0x7f4a0c0a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a1800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.079+0000 7f4a2359e700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] conn(0x7f4a0c0aa9b0 0x7f4a0c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:17.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.080+0000 7f4a2359e700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] conn(0x7f4a0c0aa9b0 0x7f4a0c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:17.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.081+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== osd.2 v2:172.21.15.194:6800/866955250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4a0c0ad470 con 0x7f4a0c0aa9b0 2023-07-21T22:15:17.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.106+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 --> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a24102fc0 con 0x7f4a0c0aa9b0 2023-07-21T22:15:17.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.106+0000 7f4a0bfff700 1 -- 172.21.15.93:0/4149809870 <== osd.2 v2:172.21.15.194:6800/866955250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a24102fc0 con 0x7f4a0c0aa9b0 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] conn(0x7f4a0c0aa9b0 msgr2=0x7f4a0c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] conn(0x7f4a0c0aa9b0 0x7f4a0c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4a0c0a11e0 msgr2=0x7f4a0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4a0c0a11e0 0x7f4a0c0a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a1800b540 comp rx=0 tx=0).stop 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 msgr2=0x7f4a2419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a291b7700 1 --2- 172.21.15.93:0/4149809870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a240feab0 0x7f4a2419a900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a140098f0 tx=0x7f4a14009cb0 comp rx=0 tx=0).stop 2023-07-21T22:15:17.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.107+0000 7f4a2359e700 1 -- 172.21.15.93:0/4149809870 reap_dead start 2023-07-21T22:15:17.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.108+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 shutdown_connections 2023-07-21T22:15:17.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.108+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 >> 172.21.15.93:0/4149809870 conn(0x7f4a24075b70 msgr2=0x7f4a24102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:17.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.108+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 shutdown_connections 2023-07-21T22:15:17.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:17.108+0000 7f4a291b7700 1 -- 172.21.15.93:0/4149809870 wait complete. 2023-07-21T22:15:18.015 INFO:teuthology.orchestra.run.smithi093.stdout:158913789993 2023-07-21T22:15:18.016 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-21T22:15:18.191 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:18 smithi093 ceph-mon[96579]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-21T22:15:18.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:18 smithi194 ceph-mon[98288]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-21T22:15:18.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:18 smithi134 ceph-mon[97873]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-21T22:15:18.388 INFO:teuthology.orchestra.run.smithi093.stdout:214748364832 2023-07-21T22:15:18.389 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-21T22:15:18.899 INFO:teuthology.orchestra.run.smithi093.stdout:55834574907 2023-07-21T22:15:18.900 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-21T22:15:19.155 INFO:teuthology.orchestra.run.smithi093.stdout:330712481809 2023-07-21T22:15:19.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-21T22:15:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 -- 172.21.15.93:0/1862195731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fe0e0 msgr2=0x7f9fc00fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/1862195731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fe0e0 0x7f9fc00fe550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009990 tx=0x7f9fb4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 -- 172.21.15.93:0/1862195731 shutdown_connections 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/1862195731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9fc00fea90 0x7f9fc010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/1862195731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc0104b40 0x7f9fc0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/1862195731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fe0e0 0x7f9fc00fe550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 -- 172.21.15.93:0/1862195731 >> 172.21.15.93:0/1862195731 conn(0x7f9fc0075150 msgr2=0x7f9fc0075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 -- 172.21.15.93:0/1862195731 shutdown_connections 2023-07-21T22:15:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.462+0000 7f9fc52a7700 1 -- 172.21.15.93:0/1862195731 wait complete. 2023-07-21T22:15:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 Processor -- start 2023-07-21T22:15:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 -- start start 2023-07-21T22:15:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc00fe0e0 0x7f9fc019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 0x7f9fc019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9fc0104b40 0x7f9fc0195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc01962c0 con 0x7f9fc00fea90 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0196430 con 0x7f9fc00fe0e0 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fc52a7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc01965f0 con 0x7f9fc0104b40 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fbe7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 0x7f9fc019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fbe7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 0x7f9fc019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42510/0 (socket says 172.21.15.93:42510) 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.463+0000 7f9fbe7fc700 1 -- 172.21.15.93:0/2197376510 learned_addr learned my addr 172.21.15.93:0/2197376510 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbeffd700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc00fe0e0 0x7f9fc019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbf7fe700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9fc0104b40 0x7f9fc0195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 -- 172.21.15.93:0/2197376510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9fc0104b40 msgr2=0x7f9fc0195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9fc0104b40 0x7f9fc0195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 -- 172.21.15.93:0/2197376510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc00fe0e0 msgr2=0x7f9fc019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc00fe0e0 0x7f9fc019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40095f0 con 0x7f9fc00fea90 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fbe7fc700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 0x7f9fc019c060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb401d070 con 0x7f9fc00fea90 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fb400be40 con 0x7f9fc00fea90 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc0196870 con 0x7f9fc00fea90 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40173f0 con 0x7f9fc00fea90 2023-07-21T22:15:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.464+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc01a2aa0 con 0x7f9fc00fea90 2023-07-21T22:15:19.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.466+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fc010fe90 con 0x7f9fc00fea90 2023-07-21T22:15:19.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.466+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9fb4022470 con 0x7f9fc00fea90 2023-07-21T22:15:19.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fa7fff700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9fa80a11e0 0x7f9fa80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fbeffd700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9fa80a11e0 0x7f9fa80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9fb40cdb30 con 0x7f9fc00fea90 2023-07-21T22:15:19.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fa7fff700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] conn(0x7f9fa80aa9b0 0x7f9fa80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fa80ad470 con 0x7f9fa80aa9b0 2023-07-21T22:15:19.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.467+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fb40d2260 con 0x7f9fc00fea90 2023-07-21T22:15:19.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.468+0000 7f9fbeffd700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9fa80a11e0 0x7f9fa80a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009be0 tx=0x7f9fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.468+0000 7f9fbf7fe700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] conn(0x7f9fa80aa9b0 0x7f9fa80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.468+0000 7f9fbf7fe700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] conn(0x7f9fa80aa9b0 0x7f9fa80acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.469+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== osd.6 v2:172.21.15.93:6818/4017577032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9fa80ad470 con 0x7f9fa80aa9b0 2023-07-21T22:15:19.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 --> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fc004e210 con 0x7f9fa80aa9b0 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fa7fff700 1 -- 172.21.15.93:0/2197376510 <== osd.6 v2:172.21.15.93:6818/4017577032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9fc004e210 con 0x7f9fa80aa9b0 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] conn(0x7f9fa80aa9b0 msgr2=0x7f9fa80acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] conn(0x7f9fa80aa9b0 0x7f9fa80acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9fa80a11e0 msgr2=0x7f9fa80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9fa80a11e0 0x7f9fa80a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009be0 tx=0x7f9fb0009260 comp rx=0 tx=0).stop 2023-07-21T22:15:19.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 msgr2=0x7f9fc019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 --2- 172.21.15.93:0/2197376510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fc00fea90 0x7f9fc019c060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb400f690 comp rx=0 tx=0).stop 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fbf7fe700 1 -- 172.21.15.93:0/2197376510 reap_dead start 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 shutdown_connections 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 >> 172.21.15.93:0/2197376510 conn(0x7f9fc0075150 msgr2=0x7f9fc00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 shutdown_connections 2023-07-21T22:15:19.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.486+0000 7f9fc52a7700 1 -- 172.21.15.93:0/2197376510 wait complete. 2023-07-21T22:15:19.603 INFO:teuthology.orchestra.run.smithi093.stdout:77309411382 2023-07-21T22:15:19.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-21T22:15:19.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- 172.21.15.93:0/1271265471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf41005d0 msgr2=0x7f9bf4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 --2- 172.21.15.93:0/1271265471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf41005d0 0x7f9bf4100a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9be80099c0 tx=0x7f9be8009cd0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- 172.21.15.93:0/1271265471 shutdown_connections 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 --2- 172.21.15.93:0/1271265471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf4100f80 0x7f9bf410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 --2- 172.21.15.93:0/1271265471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf40fee80 0x7f9bf40ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 --2- 172.21.15.93:0/1271265471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf41005d0 0x7f9bf4100a40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- 172.21.15.93:0/1271265471 >> 172.21.15.93:0/1271265471 conn(0x7f9bf4075b60 msgr2=0x7f9bf4075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- 172.21.15.93:0/1271265471 shutdown_connections 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- 172.21.15.93:0/1271265471 wait complete. 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 Processor -- start 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.500+0000 7f9bfa631700 1 -- start start 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf40fee80 0x7f9bf4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf41005d0 0x7f9bf4196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 0x7f9bf419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419ad10 con 0x7f9bf4100f80 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419ae80 con 0x7f9bf41005d0 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf419b040 con 0x7f9bf40fee80 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 0x7f9bf419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 0x7f9bf419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42528/0 (socket says 172.21.15.93:42528) 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 -- 172.21.15.93:0/2693636959 learned_addr learned my addr 172.21.15.93:0/2693636959 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf37fe700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf41005d0 0x7f9bf4196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf3fff700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf40fee80 0x7f9bf4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 -- 172.21.15.93:0/2693636959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf40fee80 msgr2=0x7f9bf4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf40fee80 0x7f9bf4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 -- 172.21.15.93:0/2693636959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf41005d0 msgr2=0x7f9bf4196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf41005d0 0x7f9bf4196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bf4100f80 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf3fff700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bf40fee80 0x7f9bf4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:19.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf8bce700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 0x7f9bf419a700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00ada0 tx=0x7f9bec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00f070 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf419b320 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec007b30 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec005440 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.501+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41a2b80 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.502+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bf404e210 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9bec004ad0 con 0x7f9bf4100f80 2023-07-21T22:15:19.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9bdc0a11e0 0x7f9bdc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf3fff700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9bdc0a11e0 0x7f9bdc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9bec0cca90 con 0x7f9bf4100f80 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] conn(0x7f9bdc0aa9b0 0x7f9bdc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bdc0ad470 con 0x7f9bdc0aa9b0 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.504+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bec0cce00 con 0x7f9bf4100f80 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.505+0000 7f9bf37fe700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] conn(0x7f9bdc0aa9b0 0x7f9bdc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.505+0000 7f9bf3fff700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9bdc0a11e0 0x7f9bdc0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9be4005fd0 tx=0x7f9be4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.505+0000 7f9bf37fe700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] conn(0x7f9bdc0aa9b0 0x7f9bdc0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.508+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== osd.0 v2:172.21.15.93:6802/2127775933 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9bdc0ad470 con 0x7f9bdc0aa9b0 2023-07-21T22:15:19.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.532+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 --> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9bf410eda0 con 0x7f9bdc0aa9b0 2023-07-21T22:15:19.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.533+0000 7f9bf17fa700 1 -- 172.21.15.93:0/2693636959 <== osd.0 v2:172.21.15.93:6802/2127775933 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9be801e040 con 0x7f9bdc0aa9b0 2023-07-21T22:15:19.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] conn(0x7f9bdc0aa9b0 msgr2=0x7f9bdc0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] conn(0x7f9bdc0aa9b0 0x7f9bdc0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9bdc0a11e0 msgr2=0x7f9bdc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9bdc0a11e0 0x7f9bdc0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9be4005fd0 tx=0x7f9be4005ee0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 msgr2=0x7f9bf419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 --2- 172.21.15.93:0/2693636959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9bf4100f80 0x7f9bf419a700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00ada0 tx=0x7f9bec00c330 comp rx=0 tx=0).stop 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bf8bce700 1 -- 172.21.15.93:0/2693636959 reap_dead start 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 shutdown_connections 2023-07-21T22:15:19.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 >> 172.21.15.93:0/2693636959 conn(0x7f9bf4075b60 msgr2=0x7f9bf40688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 shutdown_connections 2023-07-21T22:15:19.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.535+0000 7f9bfa631700 1 -- 172.21.15.93:0/2693636959 wait complete. 2023-07-21T22:15:19.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.920+0000 7fa0d5644700 1 -- 172.21.15.93:0/2489596952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d00fea90 msgr2=0x7fa0d010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.920+0000 7fa0d5644700 1 --2- 172.21.15.93:0/2489596952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d00fea90 0x7fa0d010ca00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b0a0 tx=0x7fa0c800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 -- 172.21.15.93:0/2489596952 shutdown_connections 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 --2- 172.21.15.93:0/2489596952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d00fea90 0x7fa0d010ca00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 --2- 172.21.15.93:0/2489596952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d0104b40 0x7fa0d0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 --2- 172.21.15.93:0/2489596952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0d00fe0e0 0x7fa0d00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 -- 172.21.15.93:0/2489596952 >> 172.21.15.93:0/2489596952 conn(0x7fa0d0075150 msgr2=0x7fa0d0075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 -- 172.21.15.93:0/2489596952 shutdown_connections 2023-07-21T22:15:19.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.921+0000 7fa0d5644700 1 -- 172.21.15.93:0/2489596952 wait complete. 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 Processor -- start 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 -- start start 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0d00fe0e0 0x7fa0d019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d00fea90 0x7fa0d019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 0x7fa0d0195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01961b0 con 0x7fa0d0104b40 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d0196320 con 0x7fa0d00fea90 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0d5644700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01964e0 con 0x7fa0d00fe0e0 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 0x7fa0d0195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 0x7fa0d0195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42544/0 (socket says 172.21.15.93:42544) 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 -- 172.21.15.93:0/3736780749 learned_addr learned my addr 172.21.15.93:0/3736780749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0ce7fc700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d00fea90 0x7fa0d019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0ceffd700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0d00fe0e0 0x7fa0d019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 -- 172.21.15.93:0/3736780749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0d00fe0e0 msgr2=0x7fa0d019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0d00fe0e0 0x7fa0d019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 -- 172.21.15.93:0/3736780749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d00fea90 msgr2=0x7fa0d019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0d00fea90 0x7fa0d019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.922+0000 7fa0cf7fe700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8009d00 con 0x7fa0d0104b40 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0cf7fe700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 0x7fa0d0195b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b830 tx=0x7fa0c80079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800e040 con 0x7fa0d0104b40 2023-07-21T22:15:19.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0c8004ce0 con 0x7fa0d0104b40 2023-07-21T22:15:19.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d0196760 con 0x7fa0d0104b40 2023-07-21T22:15:19.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801d480 con 0x7fa0d0104b40 2023-07-21T22:15:19.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.923+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d01a2ab0 con 0x7fa0d0104b40 2023-07-21T22:15:19.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.924+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0d010fdf0 con 0x7fa0d0104b40 2023-07-21T22:15:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.925+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa0c8004000 con 0x7fa0d0104b40 2023-07-21T22:15:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0b7fff700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa0b80a1260 0x7fa0b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0ceffd700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa0b80a1260 0x7fa0b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0c80ce9b0 con 0x7fa0d0104b40 2023-07-21T22:15:19.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0b7fff700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] conn(0x7fa0b80aa900 0x7fa0b80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b80ad3c0 con 0x7fa0b80aa900 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0c80cee50 con 0x7fa0d0104b40 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0ce7fc700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] conn(0x7fa0b80aa900 0x7fa0b80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0ceffd700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa0b80a1260 0x7fa0b80a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005950 tx=0x7fa0c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.926+0000 7fa0ce7fc700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] conn(0x7fa0b80aa900 0x7fa0b80acd10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:19.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.927+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== osd.3 v2:172.21.15.93:6810/921753281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa0b80ad3c0 con 0x7fa0b80aa900 2023-07-21T22:15:19.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.943+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 --> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0d004e210 con 0x7fa0b80aa900 2023-07-21T22:15:19.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.943+0000 7fa0b7fff700 1 -- 172.21.15.93:0/3736780749 <== osd.3 v2:172.21.15.93:6810/921753281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa0d004e210 con 0x7fa0b80aa900 2023-07-21T22:15:19.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] conn(0x7fa0b80aa900 msgr2=0x7fa0b80acd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] conn(0x7fa0b80aa900 0x7fa0b80acd10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa0b80a1260 msgr2=0x7fa0b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa0b80a1260 0x7fa0b80a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005950 tx=0x7fa0c000b410 comp rx=0 tx=0).stop 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 msgr2=0x7fa0d0195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 --2- 172.21.15.93:0/3736780749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d0104b40 0x7fa0d0195b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b830 tx=0x7fa0c80079b0 comp rx=0 tx=0).stop 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0cf7fe700 1 -- 172.21.15.93:0/3736780749 reap_dead start 2023-07-21T22:15:19.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 shutdown_connections 2023-07-21T22:15:19.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 >> 172.21.15.93:0/3736780749 conn(0x7fa0d0075150 msgr2=0x7fa0d00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:19.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 shutdown_connections 2023-07-21T22:15:19.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:19.944+0000 7fa0d5644700 1 -- 172.21.15.93:0/3736780749 wait complete. 2023-07-21T22:15:20.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- 172.21.15.93:0/3264109996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d00104ab0 msgr2=0x7f9d00104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 --2- 172.21.15.93:0/3264109996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d00104ab0 0x7f9d00104e80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000b0a0 tx=0x7f9cf000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- 172.21.15.93:0/3264109996 shutdown_connections 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 --2- 172.21.15.93:0/3264109996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d000ff460 0x7f9d0010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 --2- 172.21.15.93:0/3264109996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d00104ab0 0x7f9d00104e80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 --2- 172.21.15.93:0/3264109996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 0x7f9d000fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- 172.21.15.93:0/3264109996 >> 172.21.15.93:0/3264109996 conn(0x7f9d00075b70 msgr2=0x7f9d00075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- 172.21.15.93:0/3264109996 shutdown_connections 2023-07-21T22:15:20.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- 172.21.15.93:0/3264109996 wait complete. 2023-07-21T22:15:20.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 Processor -- start 2023-07-21T22:15:20.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.044+0000 7f9d0792e700 1 -- start start 2023-07-21T22:15:20.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 0x7f9d00196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 0x7f9d00197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d00104ab0 0x7f9d001a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00197a80 con 0x7f9d000ff460 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001a21c0 con 0x7f9d00104ab0 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001a2380 con 0x7f9d000feab0 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 0x7f9d00197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 0x7f9d00197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42558/0 (socket says 172.21.15.93:42558) 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 -- 172.21.15.93:0/1875888784 learned_addr learned my addr 172.21.15.93:0/1875888784 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:20.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d056ca700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 0x7f9d00196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 -- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 msgr2=0x7f9d00196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 0x7f9d00196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 -- 172.21.15.93:0/1875888784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d00104ab0 msgr2=0x7f9d001a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d05ecb700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d00104ab0 0x7f9d001a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d00104ab0 0x7f9d001a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009d00 con 0x7f9d000ff460 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d056ca700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d000feab0 0x7f9d00196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:20.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d04ec9700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 0x7f9d00197140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b530 tx=0x7f9cf400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4011830 con 0x7f9d000ff460 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf4011e50 con 0x7f9d000ff460 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d001a2660 con 0x7f9d000ff460 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400f2e0 con 0x7f9d000ff460 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.045+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d001a2c30 con 0x7f9d000ff460 2023-07-21T22:15:20.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.046+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d0004e210 con 0x7f9d000ff460 2023-07-21T22:15:20.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9cf4010580 con 0x7f9d000ff460 2023-07-21T22:15:20.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ce80a1260 0x7f9ce80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9d056ca700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ce80a1260 0x7f9ce80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9cf40cc9c0 con 0x7f9d000ff460 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] conn(0x7f9ce80aaa30 0x7f9ce80ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ce80ad4f0 con 0x7f9ce80aaa30 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.048+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf40ccd30 con 0x7f9d000ff460 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.049+0000 7f9d056ca700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ce80a1260 0x7f9ce80a3710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009fd0 tx=0x7f9cf000b7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.049+0000 7f9d05ecb700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] conn(0x7f9ce80aaa30 0x7f9ce80ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.049+0000 7f9d05ecb700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] conn(0x7f9ce80aaa30 0x7f9ce80ace40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:20.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.050+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== osd.8 v2:172.21.15.194:6816/773360431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9ce80ad4f0 con 0x7f9ce80aaa30 2023-07-21T22:15:20.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.067+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 --> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d0010ed00 con 0x7f9ce80aaa30 2023-07-21T22:15:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.067+0000 7f9cfe7fc700 1 -- 172.21.15.93:0/1875888784 <== osd.8 v2:172.21.15.194:6816/773360431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9cf800bb40 con 0x7f9ce80aaa30 2023-07-21T22:15:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] conn(0x7f9ce80aaa30 msgr2=0x7f9ce80ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] conn(0x7f9ce80aaa30 0x7f9ce80ace40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ce80a1260 msgr2=0x7f9ce80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ce80a1260 0x7f9ce80a3710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009fd0 tx=0x7f9cf000b7d0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 msgr2=0x7f9d00197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d0792e700 1 --2- 172.21.15.93:0/1875888784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d000ff460 0x7f9d00197140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400b530 tx=0x7f9cf400b8f0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.068+0000 7f9d05ecb700 1 -- 172.21.15.93:0/1875888784 reap_dead start 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.069+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 shutdown_connections 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.069+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 >> 172.21.15.93:0/1875888784 conn(0x7f9d00075b70 msgr2=0x7f9d000fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:20.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.069+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 shutdown_connections 2023-07-21T22:15:20.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.069+0000 7f9d0792e700 1 -- 172.21.15.93:0/1875888784 wait complete. 2023-07-21T22:15:20.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:20 smithi194 ceph-mon[98288]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-07-21T22:15:20.606 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:20 smithi093 ceph-mon[96579]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-07-21T22:15:20.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:20 smithi134 ceph-mon[97873]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-07-21T22:15:20.833 INFO:teuthology.orchestra.run.smithi093.stdout:34359738432 2023-07-21T22:15:20.833 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-21T22:15:21.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 -- 172.21.15.93:0/4054457680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 msgr2=0x7fb88c0fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 --2- 172.21.15.93:0/4054457680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c0fe750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0099c0 tx=0x7fb87c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 -- 172.21.15.93:0/4054457680 shutdown_connections 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 --2- 172.21.15.93:0/4054457680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 0x7fb88c10ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 --2- 172.21.15.93:0/4054457680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c0fe750 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 --2- 172.21.15.93:0/4054457680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 0x7fb88c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 -- 172.21.15.93:0/4054457680 >> 172.21.15.93:0/4054457680 conn(0x7fb88c075150 msgr2=0x7fb88c075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 -- 172.21.15.93:0/4054457680 shutdown_connections 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 -- 172.21.15.93:0/4054457680 wait complete. 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.523+0000 7fb8913e2700 1 Processor -- start 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 -- start start 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 0x7fb88c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 0x7fb88c195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:21.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c195fd0 con 0x7fb88c0fe380 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c196140 con 0x7fb88c104ee0 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb8913e2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c196300 con 0x7fb88c0fed20 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42572/0 (socket says 172.21.15.93:42572) 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 -- 172.21.15.93:0/3388051346 learned_addr learned my addr 172.21.15.93:0/3388051346 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88b7fe700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 0x7fb88c195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:21.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 -- 172.21.15.93:0/3388051346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 msgr2=0x7fb88c19be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88a7fc700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 0x7fb88c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 0x7fb88c19be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 -- 172.21.15.93:0/3388051346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 msgr2=0x7fb88c195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 0x7fb88c195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88affd700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0095f0 con 0x7fb88c0fe380 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88b7fe700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb88c104ee0 0x7fb88c195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.524+0000 7fb88a7fc700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c0fed20 0x7fb88c19be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:21.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb88affd700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c19b8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005910 tx=0x7fb87c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c01d070 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c196580 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87c004200 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c00f460 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.525+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c1a28d0 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.526+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb88c04e210 con 0x7fb88c0fe380 2023-07-21T22:15:21.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb87c0226c0 con 0x7fb88c0fe380 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb8740a11e0 0x7fb8740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb88a7fc700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb8740a11e0 0x7fb8740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb87c0cde70 con 0x7fb88c0fe380 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] conn(0x7fb8740aa9b0 0x7fb8740acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8740ad470 con 0x7fb8740aa9b0 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.527+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb87c0d0580 con 0x7fb88c0fe380 2023-07-21T22:15:21.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.528+0000 7fb88b7fe700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] conn(0x7fb8740aa9b0 0x7fb8740acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:21.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.528+0000 7fb88a7fc700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb8740a11e0 0x7fb8740a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb880005fd0 tx=0x7fb880005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:21.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.528+0000 7fb88b7fe700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] conn(0x7fb8740aa9b0 0x7fb8740acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:21.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.528+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== osd.9 v2:172.21.15.93:6826/692701270 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb8740ad470 con 0x7fb8740aa9b0 2023-07-21T22:15:21.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 --> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb88c1062d0 con 0x7fb8740aa9b0 2023-07-21T22:15:21.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb873fff700 1 -- 172.21.15.93:0/3388051346 <== osd.9 v2:172.21.15.93:6826/692701270 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb88c1062d0 con 0x7fb8740aa9b0 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] conn(0x7fb8740aa9b0 msgr2=0x7fb8740acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] conn(0x7fb8740aa9b0 0x7fb8740acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb8740a11e0 msgr2=0x7fb8740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb8740a11e0 0x7fb8740a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb880005fd0 tx=0x7fb880005e20 comp rx=0 tx=0).stop 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 msgr2=0x7fb88c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 --2- 172.21.15.93:0/3388051346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb88c0fe380 0x7fb88c19b8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005910 tx=0x7fb87c0036f0 comp rx=0 tx=0).stop 2023-07-21T22:15:21.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb88b7fe700 1 -- 172.21.15.93:0/3388051346 reap_dead start 2023-07-21T22:15:21.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 shutdown_connections 2023-07-21T22:15:21.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.544+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 >> 172.21.15.93:0/3388051346 conn(0x7fb88c075150 msgr2=0x7fb88c105980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:21.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.545+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 shutdown_connections 2023-07-21T22:15:21.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:20.545+0000 7fb8913e2700 1 -- 172.21.15.93:0/3388051346 wait complete. 2023-07-21T22:15:21.269 INFO:teuthology.orchestra.run.smithi093.stdout:244813135901 2023-07-21T22:15:21.269 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-21T22:15:21.329 INFO:teuthology.orchestra.run.smithi093.stdout:193273528358 2023-07-21T22:15:21.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-21T22:15:21.574 INFO:teuthology.orchestra.run.smithi093.stdout:274877906970 2023-07-21T22:15:21.575 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-21T22:15:21.721 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:21.755 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:22.354 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:22.357 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:22.362 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:22.422 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:22.423 INFO:teuthology.orchestra.run.smithi093.stdout:107374182451 2023-07-21T22:15:22.423 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-21T22:15:22.453 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:22 smithi093 ceph-mon[96579]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2023-07-21T22:15:22.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:22 smithi194 ceph-mon[98288]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2023-07-21T22:15:22.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:22 smithi134 ceph-mon[97873]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 9 op/s 2023-07-21T22:15:22.757 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:24.252 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:24.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:24 smithi194 ceph-mon[98288]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-21T22:15:24.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:24 smithi134 ceph-mon[97873]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-21T22:15:24.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:24 smithi093 ceph-mon[96579]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-21T22:15:24.762 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:25.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/3648227972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0feab0 msgr2=0x7fd29c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/3648227972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0feab0 0x7fd29c0fef20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd290009990 tx=0x7fd290009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/3648227972 shutdown_connections 2023-07-21T22:15:25.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/3648227972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c0ff460 0x7fd29c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/3648227972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c104ab0 0x7fd29c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/3648227972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0feab0 0x7fd29c0fef20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.100 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/3648227972 >> 172.21.15.93:0/3648227972 conn(0x7fd29c075b70 msgr2=0x7fd29c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:25.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/3648227972 shutdown_connections 2023-07-21T22:15:25.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.055+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/3648227972 wait complete. 2023-07-21T22:15:25.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.056+0000 7fd2a0d9b700 1 Processor -- start 2023-07-21T22:15:25.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.056+0000 7fd2a0d9b700 1 -- start start 2023-07-21T22:15:25.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c0feab0 0x7fd29c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c104ab0 0x7fd29c194910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c194f80 con 0x7fd29c0ff460 2023-07-21T22:15:25.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1950f0 con 0x7fd29c104ab0 2023-07-21T22:15:25.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1952b0 con 0x7fd29c0feab0 2023-07-21T22:15:25.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42590/0 (socket says 172.21.15.93:42590) 2023-07-21T22:15:25.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 -- 172.21.15.93:0/1959222595 learned_addr learned my addr 172.21.15.93:0/1959222595 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:25.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd29ad9d700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c104ab0 0x7fd29c194910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd29a59c700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c0feab0 0x7fd29c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 -- 172.21.15.93:0/1959222595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c0feab0 msgr2=0x7fd29c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c0feab0 0x7fd29c19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 -- 172.21.15.93:0/1959222595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c104ab0 msgr2=0x7fd29c194910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c104ab0 0x7fd29c194910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 -- 172.21.15.93:0/1959222595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900095f0 con 0x7fd29c0ff460 2023-07-21T22:15:25.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd299d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2900095c0 tx=0x7fd29001a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:25.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29001f070 con 0x7fd29c0ff460 2023-07-21T22:15:25.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29000f460 con 0x7fd29c0ff460 2023-07-21T22:15:25.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.057+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c195530 con 0x7fd29c0ff460 2023-07-21T22:15:25.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.058+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290025680 con 0x7fd29c0ff460 2023-07-21T22:15:25.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.058+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c0681a0 con 0x7fd29c0ff460 2023-07-21T22:15:25.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.059+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c04e210 con 0x7fd29c0ff460 2023-07-21T22:15:25.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.060+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd290023070 con 0x7fd29c0ff460 2023-07-21T22:15:25.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.060+0000 7fd28b7fe700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 0x7fd2840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.060+0000 7fd29a59c700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 0x7fd2840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.060+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2900cf8a0 con 0x7fd29c0ff460 2023-07-21T22:15:25.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.061+0000 7fd29a59c700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 0x7fd2840a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005950 tx=0x7fd28c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:25.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.065+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd2900990d0 con 0x7fd29c0ff460 2023-07-21T22:15:25.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 -- 172.21.15.93:0/2604813494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 msgr2=0x7fcc3010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2604813494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc3010c580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b0a0 tx=0x7fcc2800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 -- 172.21.15.93:0/2604813494 shutdown_connections 2023-07-21T22:15:25.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2604813494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc3010c580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2604813494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 0x7fcc301011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2604813494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 0x7fcc30109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 -- 172.21.15.93:0/2604813494 >> 172.21.15.93:0/2604813494 conn(0x7fcc30075b70 msgr2=0x7fcc30075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:25.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 -- 172.21.15.93:0/2604813494 shutdown_connections 2023-07-21T22:15:25.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.240+0000 7fcc3723f700 1 -- 172.21.15.93:0/2604813494 wait complete. 2023-07-21T22:15:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 Processor -- start 2023-07-21T22:15:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 -- start start 2023-07-21T22:15:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 0x7fcc300fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 0x7fcc300fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301a4240 con 0x7fcc3010a140 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301a43b0 con 0x7fcc30101700 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc3723f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301a4570 con 0x7fcc30100df0 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42606/0 (socket says 172.21.15.93:42606) 2023-07-21T22:15:25.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 -- 172.21.15.93:0/2817241739 learned_addr learned my addr 172.21.15.93:0/2817241739 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc2ffff700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 0x7fcc300fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc34fdb700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 0x7fcc300fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 -- 172.21.15.93:0/2817241739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 msgr2=0x7fcc300fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 0x7fcc300fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 -- 172.21.15.93:0/2817241739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 msgr2=0x7fcc300fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 0x7fcc300fef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.241+0000 7fcc357dc700 1 -- 172.21.15.93:0/2817241739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28009d00 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc357dc700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b780 tx=0x7fcc28007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800e040 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc28004c50 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc301a47f0 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280052a0 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.242+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc301a4cf0 con 0x7fcc3010a140 2023-07-21T22:15:25.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.243+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc30104990 con 0x7fcc3010a140 2023-07-21T22:15:25.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.244+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcc2801d470 con 0x7fcc3010a140 2023-07-21T22:15:25.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.244+0000 7fcc2dffb700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 0x7fcc200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:25.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.245+0000 7fcc34fdb700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 0x7fcc200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:25.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.245+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc280cde00 con 0x7fcc3010a140 2023-07-21T22:15:25.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.245+0000 7fcc34fdb700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 0x7fcc200a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009bf0 tx=0x7fcc24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:25.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.249+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc28097630 con 0x7fcc3010a140 2023-07-21T22:15:25.297 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:25.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.273+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd29c195e20 con 0x7fd29c0ff460 2023-07-21T22:15:25.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.274+0000 7fd28b7fe700 1 -- 172.21.15.93:0/1959222595 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd29009c6f0 con 0x7fd29c0ff460 2023-07-21T22:15:25.317 INFO:teuthology.orchestra.run.smithi093.stdout:214748364834 2023-07-21T22:15:25.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 msgr2=0x7fd2840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 0x7fd2840a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005950 tx=0x7fd28c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 msgr2=0x7fd29c19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd2900095c0 tx=0x7fd29001a8c0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 shutdown_connections 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fd2840a1260 0x7fd2840a3710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd29c104ab0 0x7fd29c194910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd29c0feab0 0x7fd29c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 --2- 172.21.15.93:0/1959222595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd29c0ff460 0x7fd29c19add0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 >> 172.21.15.93:0/1959222595 conn(0x7fd29c075b70 msgr2=0x7fd29c0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:25.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 shutdown_connections 2023-07-21T22:15:25.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.276+0000 7fd2a0d9b700 1 -- 172.21.15.93:0/1959222595 wait complete. 2023-07-21T22:15:25.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.453+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcc3004e210 con 0x7fcc3010a140 2023-07-21T22:15:25.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.454+0000 7fcc2dffb700 1 -- 172.21.15.93:0/2817241739 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc2809ac50 con 0x7fcc3010a140 2023-07-21T22:15:25.498 INFO:teuthology.orchestra.run.smithi093.stdout:137438953520 2023-07-21T22:15:25.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 msgr2=0x7fcc200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 0x7fcc200a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009bf0 tx=0x7fcc24009260 comp rx=0 tx=0).stop 2023-07-21T22:15:25.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 msgr2=0x7fcc300ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:25.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b780 tx=0x7fcc28007a40 comp rx=0 tx=0).stop 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 shutdown_connections 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc3010a140 0x7fcc300ff510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc30100df0 0x7fcc300fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcc200a11e0 0x7fcc200a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 --2- 172.21.15.93:0/2817241739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc30101700 0x7fcc300fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.456+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 >> 172.21.15.93:0/2817241739 conn(0x7fcc30075b70 msgr2=0x7fcc30101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.457+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 shutdown_connections 2023-07-21T22:15:25.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:25.457+0000 7fcc3723f700 1 -- 172.21.15.93:0/2817241739 wait complete. 2023-07-21T22:15:25.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:25 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1959222595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:15:25.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:25 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1959222595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:15:25.661 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:25 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1959222595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:15:26.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364832 got 214748364834 for osd.7 2023-07-21T22:15:26.020 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:26 smithi194 ceph-mon[98288]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T22:15:26.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:26 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2817241739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:15:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:26 smithi134 ceph-mon[97873]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T22:15:26.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:26 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2817241739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:15:26.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:26 smithi093 ceph-mon[96579]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-21T22:15:26.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:26 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2817241739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:15:27.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- 172.21.15.93:0/283463079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 msgr2=0x7f025010ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 --2- 172.21.15.93:0/283463079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f025010ba10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f024800b0d0 tx=0x7f024800b3e0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- 172.21.15.93:0/283463079 shutdown_connections 2023-07-21T22:15:27.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 --2- 172.21.15.93:0/283463079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f025010ba10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 --2- 172.21.15.93:0/283463079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 0x7f02500ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 --2- 172.21.15.93:0/283463079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 0x7f0250100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- 172.21.15.93:0/283463079 >> 172.21.15.93:0/283463079 conn(0x7f0250075b60 msgr2=0x7f0250075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- 172.21.15.93:0/283463079 shutdown_connections 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- 172.21.15.93:0/283463079 wait complete. 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 Processor -- start 2023-07-21T22:15:27.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.417+0000 7f0258758700 1 -- start start 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 0x7f0250196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 0x7f02501971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250197ae0 con 0x7f0250100f80 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501a2220 con 0x7f02500fee80 2023-07-21T22:15:27.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501a23e0 con 0x7f02501005d0 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42620/0 (socket says 172.21.15.93:42620) 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 -- 172.21.15.93:0/2872781785 learned_addr learned my addr 172.21.15.93:0/2872781785 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0255cf3700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 0x7f02501971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f02564f4700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 0x7f0250196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 -- 172.21.15.93:0/2872781785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 msgr2=0x7f02501971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 0x7f02501971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 -- 172.21.15.93:0/2872781785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 msgr2=0x7f0250196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 0x7f0250196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 -- 172.21.15.93:0/2872781785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0248009d00 con 0x7f0250100f80 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0256cf5700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0248009450 tx=0x7f0248009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024800e040 con 0x7f0250100f80 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02501a2660 con 0x7f0250100f80 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0248003e10 con 0x7f0250100f80 2023-07-21T22:15:27.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.418+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024801ba20 con 0x7f0250100f80 2023-07-21T22:15:27.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.419+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02501a2b00 con 0x7f0250100f80 2023-07-21T22:15:27.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.419+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025004e210 con 0x7f0250100f80 2023-07-21T22:15:27.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.420+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0248005730 con 0x7f0250100f80 2023-07-21T22:15:27.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.421+0000 7f02477fe700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 0x7f023c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.421+0000 7f02564f4700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 0x7f023c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.421+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02480ce010 con 0x7f0250100f80 2023-07-21T22:15:27.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.421+0000 7f02564f4700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 0x7f023c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f024c009a10 tx=0x7f024c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:27.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.425+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0248097840 con 0x7f0250100f80 2023-07-21T22:15:27.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.621+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f02501a2d20 con 0x7f0250100f80 2023-07-21T22:15:27.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.621+0000 7f02477fe700 1 -- 172.21.15.93:0/2872781785 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f024809ae60 con 0x7f0250100f80 2023-07-21T22:15:27.666 INFO:teuthology.orchestra.run.smithi093.stdout:34359738433 2023-07-21T22:15:27.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 msgr2=0x7f023c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 0x7f023c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f024c009a10 tx=0x7f024c009190 comp rx=0 tx=0).stop 2023-07-21T22:15:27.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 msgr2=0x7f02501a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0248009450 tx=0x7f0248009480 comp rx=0 tx=0).stop 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 shutdown_connections 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0250100f80 0x7f02501a1ce0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f023c0a11e0 0x7f023c0a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02500fee80 0x7f0250196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 --2- 172.21.15.93:0/2872781785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02501005d0 0x7f02501971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.624+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 >> 172.21.15.93:0/2872781785 conn(0x7f0250075b60 msgr2=0x7f0250101750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:27.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.625+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 shutdown_connections 2023-07-21T22:15:27.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.625+0000 7f0258758700 1 -- 172.21.15.93:0/2872781785 wait complete. 2023-07-21T22:15:27.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953518 got 137438953520 for osd.4 2023-07-21T22:15:27.687 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:27.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 -- 172.21.15.93:0/3343409519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 msgr2=0x7ff81410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/3343409519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff81410bff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff804009990 tx=0x7ff804009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 -- 172.21.15.93:0/3343409519 shutdown_connections 2023-07-21T22:15:27.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/3343409519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff81410c530 0x7ff81410e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/3343409519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8140fe980 0x7ff8140fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/3343409519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff81410bff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 -- 172.21.15.93:0/3343409519 >> 172.21.15.93:0/3343409519 conn(0x7ff814075b70 msgr2=0x7ff814075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:27.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 -- 172.21.15.93:0/3343409519 shutdown_connections 2023-07-21T22:15:27.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.913+0000 7ff81bfbd700 1 -- 172.21.15.93:0/3343409519 wait complete. 2023-07-21T22:15:27.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 Processor -- start 2023-07-21T22:15:27.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 -- start start 2023-07-21T22:15:27.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 0x7ff814197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff81410c530 0x7ff81419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419ce10 con 0x7ff8140ff290 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419cf80 con 0x7ff81410c530 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff81bfbd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81419d140 con 0x7ff8140fe980 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff819558700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff819558700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42630/0 (socket says 172.21.15.93:42630) 2023-07-21T22:15:27.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.914+0000 7ff819558700 1 -- 172.21.15.93:0/1694565377 learned_addr learned my addr 172.21.15.93:0/1694565377 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819d59700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 0x7ff814197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff81a55a700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff81410c530 0x7ff81419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 -- 172.21.15.93:0/1694565377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 msgr2=0x7ff814197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 0x7ff814197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 -- 172.21.15.93:0/1694565377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff81410c530 msgr2=0x7ff81419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:27.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff81410c530 0x7ff81419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 -- 172.21.15.93:0/1694565377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040095f0 con 0x7ff8140ff290 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819d59700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 0x7ff814197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff819558700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff80400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80401d070 con 0x7ff8140ff290 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff804005510 con 0x7ff8140ff290 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81419d3c0 con 0x7ff8140ff290 2023-07-21T22:15:27.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804004c90 con 0x7ff8140ff290 2023-07-21T22:15:27.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.915+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8141a4c80 con 0x7ff8140ff290 2023-07-21T22:15:27.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.916+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81404e210 con 0x7ff8140ff290 2023-07-21T22:15:27.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.917+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff80400fc10 con 0x7ff8140ff290 2023-07-21T22:15:27.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.917+0000 7ff80affd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 0x7ff8000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:27.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.918+0000 7ff819d59700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 0x7ff8000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:27.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.918+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff8040cd730 con 0x7ff8140ff290 2023-07-21T22:15:27.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.918+0000 7ff819d59700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 0x7ff8000a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff810005950 tx=0x7ff8100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:27.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:27.922+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff80405b770 con 0x7ff8140ff290 2023-07-21T22:15:28.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.007+0000 7f8a482e3700 1 -- 172.21.15.93:0/1731323993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40104ea0 msgr2=0x7f8a4010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- 172.21.15.93:0/1731323993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40104ea0 0x7f8a4010ca20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800b0d0 tx=0x7f8a3800b3e0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- 172.21.15.93:0/1731323993 shutdown_connections 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- 172.21.15.93:0/1731323993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40104ea0 0x7f8a4010ca20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- 172.21.15.93:0/1731323993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ffe00 0x7f8a401001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- 172.21.15.93:0/1731323993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a401007a0 0x7f8a40104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- 172.21.15.93:0/1731323993 >> 172.21.15.93:0/1731323993 conn(0x7f8a400fb660 msgr2=0x7f8a400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- 172.21.15.93:0/1731323993 shutdown_connections 2023-07-21T22:15:28.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- 172.21.15.93:0/1731323993 wait complete. 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 Processor -- start 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- start start 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a401007a0 0x7f8a401962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 0x7f8a4019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.008+0000 7f8a482e3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019ad60 con 0x7f8a400ffe00 2023-07-21T22:15:28.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a482e3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019aed0 con 0x7f8a40104ea0 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a482e3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019b090 con 0x7f8a401007a0 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42642/0 (socket says 172.21.15.93:42642) 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 -- 172.21.15.93:0/4058833691 learned_addr learned my addr 172.21.15.93:0/4058833691 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4587e700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a401007a0 0x7f8a401962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a46880700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 0x7f8a4019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 -- 172.21.15.93:0/4058833691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a401007a0 msgr2=0x7f8a401962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a401007a0 0x7f8a401962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 -- 172.21.15.93:0/4058833691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 msgr2=0x7f8a4019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 0x7f8a4019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 -- 172.21.15.93:0/4058833691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38009d00 con 0x7f8a400ffe00 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a46880700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 0x7f8a4019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a4607f700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b530 tx=0x7f8a3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:28.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c011830 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a3c011e50 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4019b370 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00f2e0 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.009+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401a2bd0 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.010+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4004e210 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.012+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8a3c010380 con 0x7f8a400ffe00 2023-07-21T22:15:28.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.013+0000 7f8a377fe700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 0x7f8a2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.013+0000 7f8a4587e700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 0x7f8a2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.013+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a3c0cc9e0 con 0x7f8a400ffe00 2023-07-21T22:15:28.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.013+0000 7f8a4587e700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 0x7f8a2c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a401973c0 tx=0x7f8a300073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:28.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.016+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8a3c096190 con 0x7f8a400ffe00 2023-07-21T22:15:28.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.127+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ff8141a4e10 con 0x7ff8140ff290 2023-07-21T22:15:28.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.128+0000 7ff80affd700 1 -- 172.21.15.93:0/1694565377 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff804096f60 con 0x7ff8140ff290 2023-07-21T22:15:28.171 INFO:teuthology.orchestra.run.smithi093.stdout:330712481811 2023-07-21T22:15:28.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 msgr2=0x7ff8000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 0x7ff8000a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff810005950 tx=0x7ff8100058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 msgr2=0x7ff814198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff80400f720 comp rx=0 tx=0).stop 2023-07-21T22:15:28.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 shutdown_connections 2023-07-21T22:15:28.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff8000a11e0 0x7ff8000a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff81410c530 0x7ff81419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8140fe980 0x7ff814197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 --2- 172.21.15.93:0/1694565377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8140ff290 0x7ff814198390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 >> 172.21.15.93:0/1694565377 conn(0x7ff814075b70 msgr2=0x7ff8140fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:28.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 shutdown_connections 2023-07-21T22:15:28.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.130+0000 7ff81bfbd700 1 -- 172.21.15.93:0/1694565377 wait complete. 2023-07-21T22:15:28.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.218+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f8a40197100 con 0x7f8a400ffe00 2023-07-21T22:15:28.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.219+0000 7f8a377fe700 1 -- 172.21.15.93:0/4058833691 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a3c0997b0 con 0x7f8a400ffe00 2023-07-21T22:15:28.263 INFO:teuthology.orchestra.run.smithi093.stdout:274877906971 2023-07-21T22:15:28.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 msgr2=0x7f8a2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 0x7f8a2c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a401973c0 tx=0x7f8a300073d0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 msgr2=0x7f8a40195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b530 tx=0x7f8a3c00b840 comp rx=0 tx=0).stop 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 shutdown_connections 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8a2c0a11e0 0x7f8a2c0a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a40104ea0 0x7f8a4019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.265 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a400ffe00 0x7f8a40195da0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 --2- 172.21.15.93:0/4058833691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a401007a0 0x7f8a401962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 >> 172.21.15.93:0/4058833691 conn(0x7f8a400fb660 msgr2=0x7f8a400fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:28.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 shutdown_connections 2023-07-21T22:15:28.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.221+0000 7f8a482e3700 1 -- 172.21.15.93:0/4058833691 wait complete. 2023-07-21T22:15:28.296 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:28 smithi093 ceph-mon[96579]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-21T22:15:28.297 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:28 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2872781785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:15:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:28 smithi194 ceph-mon[98288]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-21T22:15:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:28 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2872781785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:15:28.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:28 smithi134 ceph-mon[97873]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-21T22:15:28.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:28 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2872781785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:15:28.669 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:28.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- 172.21.15.93:0/1272941011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3780feab0 msgr2=0x7fe3780fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/1272941011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3780feab0 0x7fe3780fef20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe368009990 tx=0x7fe368009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- 172.21.15.93:0/1272941011 shutdown_connections 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/1272941011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 0x7fe37810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/1272941011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe378104ab0 0x7fe378104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/1272941011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3780feab0 0x7fe3780fef20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- 172.21.15.93:0/1272941011 >> 172.21.15.93:0/1272941011 conn(0x7fe378075b70 msgr2=0x7fe378075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- 172.21.15.93:0/1272941011 shutdown_connections 2023-07-21T22:15:28.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- 172.21.15.93:0/1272941011 wait complete. 2023-07-21T22:15:28.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 Processor -- start 2023-07-21T22:15:28.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.885+0000 7fe37d8ee700 1 -- start start 2023-07-21T22:15:28.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780feab0 0x7fe378197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 0x7fe3781983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.931 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819ce20 con 0x7fe378104ab0 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819cf90 con 0x7fe3780feab0 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37819d150 con 0x7fe3780ff460 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42674/0 (socket says 172.21.15.93:42674) 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 -- 172.21.15.93:0/516933016 learned_addr learned my addr 172.21.15.93:0/516933016 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:28.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3767fc700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 0x7fe3781983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe376ffd700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780feab0 0x7fe378197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 -- 172.21.15.93:0/516933016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 msgr2=0x7fe3781983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 0x7fe3781983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 -- 172.21.15.93:0/516933016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780feab0 msgr2=0x7fe378197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780feab0 0x7fe378197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 -- 172.21.15.93:0/516933016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680095f0 con 0x7fe378104ab0 2023-07-21T22:15:28.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe3777fe700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe370007f10 tx=0x7fe37000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009ea0 con 0x7fe378104ab0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37819d430 con 0x7fe378104ab0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37000f040 con 0x7fe378104ab0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370004810 con 0x7fe378104ab0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.886+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3781a4c90 con 0x7fe378104ab0 2023-07-21T22:15:28.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.887+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37804e210 con 0x7fe378104ab0 2023-07-21T22:15:28.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.888+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe370003a00 con 0x7fe378104ab0 2023-07-21T22:15:28.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.889+0000 7fe37c8ec700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 0x7fe3600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:28.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.889+0000 7fe376ffd700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 0x7fe3600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:28.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.889+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe3700cc1f0 con 0x7fe378104ab0 2023-07-21T22:15:28.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.889+0000 7fe376ffd700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 0x7fe3600a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005950 tx=0x7fe36c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:28.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.893+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe37004e770 con 0x7fe378104ab0 2023-07-21T22:15:29.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.613+0000 7fe3608e4700 1 -- 172.21.15.93:0/3717127698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c0feab0 msgr2=0x7fe35c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.613+0000 7fe3608e4700 1 --2- 172.21.15.93:0/3717127698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c0feab0 0x7fe35c0fef20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe350009990 tx=0x7fe350009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.613+0000 7fe3608e4700 1 -- 172.21.15.93:0/3717127698 shutdown_connections 2023-07-21T22:15:29.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 --2- 172.21.15.93:0/3717127698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0ff460 0x7fe35c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 --2- 172.21.15.93:0/3717127698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c104ab0 0x7fe35c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 --2- 172.21.15.93:0/3717127698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c0feab0 0x7fe35c0fef20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 -- 172.21.15.93:0/3717127698 >> 172.21.15.93:0/3717127698 conn(0x7fe35c075b70 msgr2=0x7fe35c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 -- 172.21.15.93:0/3717127698 shutdown_connections 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 -- 172.21.15.93:0/3717127698 wait complete. 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 Processor -- start 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.614+0000 7fe3608e4700 1 -- start start 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 0x7fe35c1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 0x7fe35c100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:29.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a4120 con 0x7fe35c104ab0 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a4290 con 0x7fe35c0ff460 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a4450 con 0x7fe35c0feab0 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42660/0 (socket says 172.21.15.93:42660) 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 -- 172.21.15.93:0/1188186254 learned_addr learned my addr 172.21.15.93:0/1188186254 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:29.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe359d9b700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 0x7fe35c100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 -- 172.21.15.93:0/1188186254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 msgr2=0x7fe35c1006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35a59c700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 0x7fe35c1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 0x7fe35c1006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 -- 172.21.15.93:0/1188186254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 msgr2=0x7fe35c100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 0x7fe35c100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 -- 172.21.15.93:0/1188186254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500095f0 con 0x7fe35c104ab0 2023-07-21T22:15:29.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe359d9b700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 0x7fe35c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35a59c700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 0x7fe35c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe35ad9d700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe35400b9e0 tx=0x7fe35400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400c950 con 0x7fe35c104ab0 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe354014440 con 0x7fe35c104ab0 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c1a4730 con 0x7fe35c104ab0 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540075d0 con 0x7fe35c104ab0 2023-07-21T22:15:29.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.615+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c1a4cb0 con 0x7fe35c104ab0 2023-07-21T22:15:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.616+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c04e210 con 0x7fe35c104ab0 2023-07-21T22:15:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.617+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe3540145b0 con 0x7fe35c104ab0 2023-07-21T22:15:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.618+0000 7fe34b7fe700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 0x7fe3440a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:29.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.618+0000 7fe35a59c700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 0x7fe3440a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:29.039 INFO:teuthology.orchestra.run.smithi093.stdout:296352743446 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.618+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe354006e40 con 0x7fe35c104ab0 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.618+0000 7fe35a59c700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 0x7fe3440a3790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0ffbf0 tx=0x7fe34c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.622+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe354096230 con 0x7fe35c104ab0 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.820+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fe35c101bf0 con 0x7fe35c104ab0 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.821+0000 7fe34b7fe700 1 -- 172.21.15.93:0/1188186254 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe354019090 con 0x7fe35c104ab0 2023-07-21T22:15:29.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.822+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 msgr2=0x7fe3440a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 0x7fe3440a3790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0ffbf0 tx=0x7fe34c008040 comp rx=0 tx=0).stop 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 msgr2=0x7fe35c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe35400b9e0 tx=0x7fe35400bda0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 shutdown_connections 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe35c104ab0 0x7fe35c101180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe35c0feab0 0x7fe35c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3440a12e0 0x7fe3440a3790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 --2- 172.21.15.93:0/1188186254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe35c0ff460 0x7fe35c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 >> 172.21.15.93:0/1188186254 conn(0x7fe35c075b70 msgr2=0x7fe35c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:29.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 shutdown_connections 2023-07-21T22:15:29.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:28.823+0000 7fe3608e4700 1 -- 172.21.15.93:0/1188186254 wait complete. 2023-07-21T22:15:29.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738433 for osd.0 2023-07-21T22:15:29.048 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:29.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.114+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe3781a4ee0 con 0x7fe378104ab0 2023-07-21T22:15:29.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.115+0000 7fe37c8ec700 1 -- 172.21.15.93:0/516933016 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe370095990 con 0x7fe378104ab0 2023-07-21T22:15:29.159 INFO:teuthology.orchestra.run.smithi093.stdout:158913789996 2023-07-21T22:15:29.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 msgr2=0x7fe3600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 0x7fe3600a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005950 tx=0x7fe36c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 msgr2=0x7fe37819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:29.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe370007f10 tx=0x7fe37000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 shutdown_connections 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe378104ab0 0x7fe37819c810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3600a11e0 0x7fe3600a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780feab0 0x7fe378197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 --2- 172.21.15.93:0/516933016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3780ff460 0x7fe3781983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:29.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 >> 172.21.15.93:0/516933016 conn(0x7fe378075b70 msgr2=0x7fe3780fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:29.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 shutdown_connections 2023-07-21T22:15:29.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:29.118+0000 7fe37d8ee700 1 -- 172.21.15.93:0/516933016 wait complete. 2023-07-21T22:15:29.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:29 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1694565377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:15:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:29 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4058833691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:15:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:29 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1188186254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:15:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:29 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1694565377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:15:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:29 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4058833691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:15:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:29 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1188186254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:15:29.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:29 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1694565377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:15:29.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:29 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4058833691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:15:29.406 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:29 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1188186254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:15:29.856 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906971 for osd.9 2023-07-21T22:15:29.857 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:29.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481811 for osd.11 2023-07-21T22:15:29.893 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:30.005 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743444 got 296352743446 for osd.10 2023-07-21T22:15:30.005 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:30.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.072+0000 7f2b0364f700 1 -- 172.21.15.93:0/3526712666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 msgr2=0x7f2afc100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.072+0000 7f2b0364f700 1 --2- 172.21.15.93:0/3526712666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc100a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2af00099c0 tx=0x7f2af0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- 172.21.15.93:0/3526712666 shutdown_connections 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- 172.21.15.93:0/3526712666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 0x7f2afc10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- 172.21.15.93:0/3526712666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- 172.21.15.93:0/3526712666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc100a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- 172.21.15.93:0/3526712666 >> 172.21.15.93:0/3526712666 conn(0x7f2afc075b60 msgr2=0x7f2afc075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- 172.21.15.93:0/3526712666 shutdown_connections 2023-07-21T22:15:30.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- 172.21.15.93:0/3526712666 wait complete. 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 Processor -- start 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- start start 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 0x7f2afc1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1a21c0 con 0x7f2afc1005d0 2023-07-21T22:15:30.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1a2300 con 0x7f2afc100f80 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.073+0000 7f2b0364f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1a2440 con 0x7f2afc0fee80 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42684/0 (socket says 172.21.15.93:42684) 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 -- 172.21.15.93:0/4183630518 learned_addr learned my addr 172.21.15.93:0/4183630518 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 -- 172.21.15.93:0/4183630518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 msgr2=0x7f2afc196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b01bec700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 0x7f2afc1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b013eb700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 -- 172.21.15.93:0/4183630518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 msgr2=0x7f2afc1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 0x7f2afc1a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 -- 172.21.15.93:0/4183630518 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00095f0 con 0x7f2afc1005d0 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b013eb700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b00bea700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005910 tx=0x7f2af00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:30.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af001d070 con 0x7f2afc1005d0 2023-07-21T22:15:30.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc1a2670 con 0x7f2afc1005d0 2023-07-21T22:15:30.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af0004200 con 0x7f2afc1005d0 2023-07-21T22:15:30.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000f460 con 0x7f2afc1005d0 2023-07-21T22:15:30.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.074+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc1a2b10 con 0x7f2afc1005d0 2023-07-21T22:15:30.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.075+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04e210 con 0x7f2afc1005d0 2023-07-21T22:15:30.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.077+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2af00226c0 con 0x7f2afc1005d0 2023-07-21T22:15:30.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.078+0000 7f2afa7fc700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 0x7f2ae40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.078+0000 7f2b013eb700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 0x7f2ae40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.078+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2af00ce5c0 con 0x7f2afc1005d0 2023-07-21T22:15:30.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.078+0000 7f2b013eb700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 0x7f2ae40a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007900 tx=0x7f2af4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:30.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.081+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2af0097df0 con 0x7f2afc1005d0 2023-07-21T22:15:30.252 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:30 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/516933016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:15:30.253 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:30 smithi093 ceph-mon[96579]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:15:30.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.286+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f2afc1a2df0 con 0x7f2afc1005d0 2023-07-21T22:15:30.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.286+0000 7f2afa7fc700 1 -- 172.21.15.93:0/4183630518 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2af009b410 con 0x7f2afc1005d0 2023-07-21T22:15:30.330 INFO:teuthology.orchestra.run.smithi093.stdout:244813135903 2023-07-21T22:15:30.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 msgr2=0x7f2ae40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 0x7f2ae40a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2af4007900 tx=0x7f2af4008040 comp rx=0 tx=0).stop 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 msgr2=0x7f2afc197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2af0005910 tx=0x7f2af00036f0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 shutdown_connections 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2ae40a11e0 0x7f2ae40a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2afc100f80 0x7f2afc1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2afc0fee80 0x7f2afc196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 --2- 172.21.15.93:0/4183630518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2afc1005d0 0x7f2afc197140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 >> 172.21.15.93:0/4183630518 conn(0x7f2afc075b60 msgr2=0x7f2afc103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:30.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 shutdown_connections 2023-07-21T22:15:30.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.288+0000 7f2b0364f700 1 -- 172.21.15.93:0/4183630518 wait complete. 2023-07-21T22:15:30.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:30 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/516933016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:15:30.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:30 smithi194 ceph-mon[98288]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:15:30.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:30 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/516933016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:15:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:30 smithi134 ceph-mon[97873]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:15:30.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- 172.21.15.93:0/4023033158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c100f80 msgr2=0x7f598c10ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 --2- 172.21.15.93:0/4023033158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c100f80 0x7f598c10ba10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f598400b0d0 tx=0x7f598400b3e0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- 172.21.15.93:0/4023033158 shutdown_connections 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 --2- 172.21.15.93:0/4023033158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c100f80 0x7f598c10ba10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 --2- 172.21.15.93:0/4023033158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c0fee80 0x7f598c0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 --2- 172.21.15.93:0/4023033158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c1005d0 0x7f598c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- 172.21.15.93:0/4023033158 >> 172.21.15.93:0/4023033158 conn(0x7f598c075b60 msgr2=0x7f598c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- 172.21.15.93:0/4023033158 shutdown_connections 2023-07-21T22:15:30.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- 172.21.15.93:0/4023033158 wait complete. 2023-07-21T22:15:30.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 Processor -- start 2023-07-21T22:15:30.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.385+0000 7f59939d1700 1 -- start start 2023-07-21T22:15:30.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c0fee80 0x7f598c196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c100f80 0x7f598c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c197af0 con 0x7f598c1005d0 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1a2260 con 0x7f598c100f80 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1a23d0 con 0x7f598c0fee80 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42698/0 (socket says 172.21.15.93:42698) 2023-07-21T22:15:30.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 -- 172.21.15.93:0/467748110 learned_addr learned my addr 172.21.15.93:0/467748110 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f599176d700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c0fee80 0x7f598c196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5991f6e700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c100f80 0x7f598c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 -- 172.21.15.93:0/467748110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c0fee80 msgr2=0x7f598c196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c0fee80 0x7f598c196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 -- 172.21.15.93:0/467748110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c100f80 msgr2=0x7f598c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c100f80 0x7f598c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 -- 172.21.15.93:0/467748110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009d00 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f5990f6c700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f597c00d950 tx=0x7f597c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c009940 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c1a26b0 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f597c010460 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00f620 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.386+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c1a2c00 con 0x7f598c1005d0 2023-07-21T22:15:30.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.388+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c04e210 con 0x7f598c1005d0 2023-07-21T22:15:30.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.388+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f597c0108d0 con 0x7f598c1005d0 2023-07-21T22:15:30.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.389+0000 7f59827fc700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 0x7f59780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:30.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.389+0000 7f599176d700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 0x7f59780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:30.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.389+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f597c0cca40 con 0x7f598c1005d0 2023-07-21T22:15:30.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.390+0000 7f599176d700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 0x7f59780a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5988005950 tx=0x7f598800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:30.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.394+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f597c0961e0 con 0x7f598c1005d0 2023-07-21T22:15:30.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.600+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f598c1a2ee0 con 0x7f598c1005d0 2023-07-21T22:15:30.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.601+0000 7f59827fc700 1 -- 172.21.15.93:0/467748110 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f597c020090 con 0x7f598c1005d0 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 msgr2=0x7f59780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 0x7f59780a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5988005950 tx=0x7f598800a300 comp rx=0 tx=0).stop 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 msgr2=0x7f598c1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f597c00d950 tx=0x7f597c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 shutdown_connections 2023-07-21T22:15:30.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f59780a11e0 0x7f59780a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c100f80 0x7f598c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f598c0fee80 0x7f598c196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 --2- 172.21.15.93:0/467748110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f598c1005d0 0x7f598c1971e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 >> 172.21.15.93:0/467748110 conn(0x7f598c075b60 msgr2=0x7f598c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 shutdown_connections 2023-07-21T22:15:30.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:30.603+0000 7f59939d1700 1 -- 172.21.15.93:0/467748110 wait complete. 2023-07-21T22:15:30.673 INFO:teuthology.orchestra.run.smithi093.stdout:55834574909 2023-07-21T22:15:30.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789993 got 158913789996 for osd.5 2023-07-21T22:15:30.998 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:31.196 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:31.213 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:31 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4183630518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:15:31.213 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:31 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/467748110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:15:31.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:31 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4183630518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:15:31.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:31 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/467748110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:15:31.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:31 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4183630518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:15:31.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:31 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/467748110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:15:31.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135901 got 244813135903 for osd.8 2023-07-21T22:15:31.623 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 -- 172.21.15.93:0/1607838072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc104ab0 msgr2=0x7feedc104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 --2- 172.21.15.93:0/1607838072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc104ab0 0x7feedc104e80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feecc009990 tx=0x7feecc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 -- 172.21.15.93:0/1607838072 shutdown_connections 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 --2- 172.21.15.93:0/1607838072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feedc0ff460 0x7feedc10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 --2- 172.21.15.93:0/1607838072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc104ab0 0x7feedc104e80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 --2- 172.21.15.93:0/1607838072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc0feab0 0x7feedc0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 -- 172.21.15.93:0/1607838072 >> 172.21.15.93:0/1607838072 conn(0x7feedc075b70 msgr2=0x7feedc075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 -- 172.21.15.93:0/1607838072 shutdown_connections 2023-07-21T22:15:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.602+0000 7feee2388700 1 -- 172.21.15.93:0/1607838072 wait complete. 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 Processor -- start 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 -- start start 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feedc0ff460 0x7feedc19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc104ab0 0x7feedc1949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc195020 con 0x7feedc0feab0 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc195190 con 0x7feedc104ab0 2023-07-21T22:15:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feee2388700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc195350 con 0x7feedc0ff460 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feedbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feedbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42710/0 (socket says 172.21.15.93:42710) 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.603+0000 7feedbfff700 1 -- 172.21.15.93:0/3243261136 learned_addr learned my addr 172.21.15.93:0/3243261136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 -- 172.21.15.93:0/3243261136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feedc0ff460 msgr2=0x7feedc19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feedc0ff460 0x7feedc19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 -- 172.21.15.93:0/3243261136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc104ab0 msgr2=0x7feedc1949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc104ab0 0x7feedc1949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 -- 172.21.15.93:0/3243261136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0095f0 con 0x7feedc0feab0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feedbfff700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feecc000c00 tx=0x7feecc00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc01d070 con 0x7feedc0feab0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc1955d0 con 0x7feedc0feab0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feecc00fca0 con 0x7feedc0feab0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc017760 con 0x7feedc0feab0 2023-07-21T22:15:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.604+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc1a1920 con 0x7feedc0feab0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.605+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc1957e0 con 0x7feedc0feab0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.606+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feecc0178c0 con 0x7feedc0feab0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.607+0000 7feed97fa700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 0x7feec40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.607+0000 7feedb7fe700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 0x7feec40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.607+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feecc0ce770 con 0x7feedc0feab0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.607+0000 7feedb7fe700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 0x7feec40a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feedc0fc720 tx=0x7feed0005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.610+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feecc097fa0 con 0x7feedc0feab0 2023-07-21T22:15:31.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.802+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feedc04e210 con 0x7feedc0feab0 2023-07-21T22:15:31.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.802+0000 7feed97fa700 1 -- 172.21.15.93:0/3243261136 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feecc09b5c0 con 0x7feedc0feab0 2023-07-21T22:15:31.846 INFO:teuthology.orchestra.run.smithi093.stdout:77309411385 2023-07-21T22:15:31.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.804+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 msgr2=0x7feec40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:31.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.804+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 0x7feec40a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feedc0fc720 tx=0x7feed0005c80 comp rx=0 tx=0).stop 2023-07-21T22:15:31.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.804+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 msgr2=0x7feedc19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:31.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.804+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feecc000c00 tx=0x7feecc00f7b0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 shutdown_connections 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7feec40a11e0 0x7feec40a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feedc104ab0 0x7feedc1949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc0feab0 0x7feedc19a930 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 --2- 172.21.15.93:0/3243261136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feedc0ff460 0x7feedc19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 >> 172.21.15.93:0/3243261136 conn(0x7feedc075b70 msgr2=0x7feedc0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:31.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 shutdown_connections 2023-07-21T22:15:31.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:31.805+0000 7feee2388700 1 -- 172.21.15.93:0/3243261136 wait complete. 2023-07-21T22:15:31.880 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574909 for osd.1 2023-07-21T22:15:31.880 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:32.294 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:32 smithi093 ceph-mon[96579]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:32.295 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:32 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3243261136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:15:32.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 -- 172.21.15.93:0/685668233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980fe780 msgr2=0x7ff3980feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:32.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 --2- 172.21.15.93:0/685668233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980fe780 0x7ff3980feb50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff388009990 tx=0x7ff388009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 -- 172.21.15.93:0/685668233 shutdown_connections 2023-07-21T22:15:32.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 --2- 172.21.15.93:0/685668233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff39810c330 0x7ff39810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 --2- 172.21.15.93:0/685668233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980fe780 0x7ff3980feb50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 --2- 172.21.15.93:0/685668233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3980ff090 0x7ff39810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 -- 172.21.15.93:0/685668233 >> 172.21.15.93:0/685668233 conn(0x7ff3980fa450 msgr2=0x7ff3980fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:32.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 -- 172.21.15.93:0/685668233 shutdown_connections 2023-07-21T22:15:32.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.271+0000 7ff39759e700 1 -- 172.21.15.93:0/685668233 wait complete. 2023-07-21T22:15:32.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 Processor -- start 2023-07-21T22:15:32.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 -- start start 2023-07-21T22:15:32.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3980fe780 0x7ff398197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:32.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:32.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff39810c330 0x7ff39819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:32.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819bba0 con 0x7ff3980ff090 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819bd10 con 0x7ff3980fe780 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39759e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819bed0 con 0x7ff39810c330 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42726/0 (socket says 172.21.15.93:42726) 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 -- 172.21.15.93:0/2870836679 learned_addr learned my addr 172.21.15.93:0/2870836679 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff39659c700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3980fe780 0x7ff398197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:32.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff396d9d700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff39810c330 0x7ff39819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 -- 172.21.15.93:0/2870836679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff39810c330 msgr2=0x7ff39819b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff39810c330 0x7ff39819b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 -- 172.21.15.93:0/2870836679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3980fe780 msgr2=0x7ff398197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3980fe780 0x7ff398197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.272+0000 7ff395d9b700 1 -- 172.21.15.93:0/2870836679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880095f0 con 0x7ff3980ff090 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff395d9b700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00d950 tx=0x7ff38c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c009940 con 0x7ff3980ff090 2023-07-21T22:15:32.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38c010460 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00f620 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39819c1b0 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.273+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3981a4b00 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.274+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39804ea60 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.275+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff38c0108d0 con 0x7ff3980ff090 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.275+0000 7ff3877fe700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 0x7ff3800a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:32.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.276+0000 7ff39659c700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 0x7ff3800a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:32.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.276+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff38c0cc7f0 con 0x7ff3980ff090 2023-07-21T22:15:32.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.276+0000 7ff39659c700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 0x7ff3800a3790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b2e0 tx=0x7ff388005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:32.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.279+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff38c095e90 con 0x7ff3980ff090 2023-07-21T22:15:32.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:32 smithi194 ceph-mon[98288]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:32.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:32 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3243261136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:15:32.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:32 smithi134 ceph-mon[97873]: pgmap v272: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:32.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:32 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3243261136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:15:32.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.479+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff398198fc0 con 0x7ff3980ff090 2023-07-21T22:15:32.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.479+0000 7ff3877fe700 1 -- 172.21.15.93:0/2870836679 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff38c020090 con 0x7ff3980ff090 2023-07-21T22:15:32.523 INFO:teuthology.orchestra.run.smithi093.stdout:193273528361 2023-07-21T22:15:32.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.481+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 msgr2=0x7ff3800a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:32.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.481+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 0x7ff3800a3790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b2e0 tx=0x7ff388005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.481+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 msgr2=0x7ff3981981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:32.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.481+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00d950 tx=0x7ff38c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 shutdown_connections 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff39810c330 0x7ff39819b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff3800a12e0 0x7ff3800a3790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3980fe780 0x7ff398197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 --2- 172.21.15.93:0/2870836679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3980ff090 0x7ff3981981e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 >> 172.21.15.93:0/2870836679 conn(0x7ff3980fa450 msgr2=0x7ff39810ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 shutdown_connections 2023-07-21T22:15:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:32.482+0000 7ff39759e700 1 -- 172.21.15.93:0/2870836679 wait complete. 2023-07-21T22:15:32.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411382 got 77309411385 for osd.2 2023-07-21T22:15:32.941 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:33.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:33 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2870836679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:15:33.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528358 got 193273528361 for osd.6 2023-07-21T22:15:33.113 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:33.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:33 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2870836679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:15:33.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:33 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2870836679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:15:33.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- 172.21.15.93:0/3015734356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740feab0 msgr2=0x7f1b740fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:33.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/3015734356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740feab0 0x7f1b740fef20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009990 tx=0x7f1b60009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- 172.21.15.93:0/3015734356 shutdown_connections 2023-07-21T22:15:33.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/3015734356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b7410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/3015734356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b74104ab0 0x7f1b74104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/3015734356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740feab0 0x7f1b740fef20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- 172.21.15.93:0/3015734356 >> 172.21.15.93:0/3015734356 conn(0x7f1b74075b70 msgr2=0x7f1b74075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- 172.21.15.93:0/3015734356 shutdown_connections 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- 172.21.15.93:0/3015734356 wait complete. 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 Processor -- start 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.394+0000 7f1b7b17b700 1 -- start start 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b740feab0 0x7f1b741969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:33.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b74196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74197820 con 0x7f1b74104ab0 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b741a1f90 con 0x7f1b740ff460 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b741a2100 con 0x7f1b740feab0 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42742/0 (socket says 172.21.15.93:42742) 2023-07-21T22:15:33.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 -- 172.21.15.93:0/2623968196 learned_addr learned my addr 172.21.15.93:0/2623968196 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b78f17700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b740feab0 0x7f1b741969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 -- 172.21.15.93:0/2623968196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b740feab0 msgr2=0x7f1b741969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b740feab0 0x7f1b741969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 -- 172.21.15.93:0/2623968196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 msgr2=0x7f1b74196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b73fff700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b74196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b74196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 -- 172.21.15.93:0/2623968196 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600095f0 con 0x7f1b74104ab0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b73fff700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b74196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b79718700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c007f10 tx=0x7f1b6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c017070 con 0x7f1b74104ab0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b741a23e0 con 0x7f1b74104ab0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b6c00f040 con 0x7f1b74104ab0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.395+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c005740 con 0x7f1b74104ab0 2023-07-21T22:15:33.443 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.396+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b741a2890 con 0x7f1b74104ab0 2023-07-21T22:15:33.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.396+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7404e210 con 0x7f1b74104ab0 2023-07-21T22:15:33.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.397+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1b6c00dc60 con 0x7f1b74104ab0 2023-07-21T22:15:33.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.398+0000 7f1b71ffb700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 0x7f1b640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:33.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.398+0000 7f1b78f17700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 0x7f1b640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:33.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.398+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b6c0cc070 con 0x7f1b74104ab0 2023-07-21T22:15:33.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.398+0000 7f1b78f17700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 0x7f1b640a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b740ffbf0 tx=0x7f1b68005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:33.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.402+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1b6c095810 con 0x7f1b74104ab0 2023-07-21T22:15:33.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.590+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1b741a2c40 con 0x7f1b74104ab0 2023-07-21T22:15:33.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.591+0000 7f1b71ffb700 1 -- 172.21.15.93:0/2623968196 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b6c098e30 con 0x7f1b74104ab0 2023-07-21T22:15:33.634 INFO:teuthology.orchestra.run.smithi093.stdout:107374182454 2023-07-21T22:15:33.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 msgr2=0x7f1b640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:33.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 0x7f1b640a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b740ffbf0 tx=0x7f1b68005d70 comp rx=0 tx=0).stop 2023-07-21T22:15:33.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 msgr2=0x7f1b741a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c007f10 tx=0x7f1b6c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 shutdown_connections 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b74104ab0 0x7f1b741a1a50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b740feab0 0x7f1b741969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f1b640a11e0 0x7f1b640a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 --2- 172.21.15.93:0/2623968196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b740ff460 0x7f1b74196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 >> 172.21.15.93:0/2623968196 conn(0x7f1b74075b70 msgr2=0x7f1b74105930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:33.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 shutdown_connections 2023-07-21T22:15:33.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:33.593+0000 7f1b7b17b700 1 -- 172.21.15.93:0/2623968196 wait complete. 2023-07-21T22:15:34.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182451 got 107374182454 for osd.3 2023-07-21T22:15:34.061 DEBUG:teuthology.parallel:result is None 2023-07-21T22:15:34.061 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-21T22:15:34.061 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:15:34.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:34 smithi194 ceph-mon[98288]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:34.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:34 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2623968196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:15:34.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:34 smithi134 ceph-mon[97873]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:34.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:34 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2623968196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:15:34.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:34 smithi093 ceph-mon[96579]: pgmap v273: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:34.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:34 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2623968196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:15:34.617 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:35.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 -- 172.21.15.93:0/3824035974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0ff460 msgr2=0x7f4f9c10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:35.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/3824035974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c10b8f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b0a0 tx=0x7f4f9400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 -- 172.21.15.93:0/3824035974 shutdown_connections 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/3824035974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c10b8f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/3824035974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c104ab0 0x7f4f9c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/3824035974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 -- 172.21.15.93:0/3824035974 >> 172.21.15.93:0/3824035974 conn(0x7f4f9c075b70 msgr2=0x7f4f9c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 -- 172.21.15.93:0/3824035974 shutdown_connections 2023-07-21T22:15:35.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.794+0000 7f4fa3e37700 1 -- 172.21.15.93:0/3824035974 wait complete. 2023-07-21T22:15:35.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 Processor -- start 2023-07-21T22:15:35.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 -- start start 2023-07-21T22:15:35.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:35.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:35.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c104ab0 0x7f4f9c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:35.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19aeb0 con 0x7f4f9c0feab0 2023-07-21T22:15:35.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19b020 con 0x7f4f9c0ff460 2023-07-21T22:15:35.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa3e37700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19b1e0 con 0x7f4f9c104ab0 2023-07-21T22:15:35.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:35.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42752/0 (socket says 172.21.15.93:42752) 2023-07-21T22:15:35.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 -- 172.21.15.93:0/1949934834 learned_addr learned my addr 172.21.15.93:0/1949934834 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:35.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa13d2700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:35.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa23d4700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c104ab0 0x7f4f9c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:35.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 -- 172.21.15.93:0/1949934834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c104ab0 msgr2=0x7f4f9c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:35.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c104ab0 0x7f4f9c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 -- 172.21.15.93:0/1949934834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c0ff460 msgr2=0x7f4f9c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:35.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:35.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.795+0000 7f4fa1bd3700 1 -- 172.21.15.93:0/1949934834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009d00 con 0x7f4f9c0feab0 2023-07-21T22:15:35.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4fa1bd3700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b500 tx=0x7f4f8c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:35.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011830 con 0x7f4f9c0feab0 2023-07-21T22:15:35.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c011e50 con 0x7f4f9c0feab0 2023-07-21T22:15:35.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00f2e0 con 0x7f4f9c0feab0 2023-07-21T22:15:35.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c19b4c0 con 0x7f4f9c0feab0 2023-07-21T22:15:35.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.796+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c1a2d20 con 0x7f4f9c0feab0 2023-07-21T22:15:35.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.797+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c04e210 con 0x7f4f9c0feab0 2023-07-21T22:15:35.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.798+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4f8c010580 con 0x7f4f9c0feab0 2023-07-21T22:15:35.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.798+0000 7f4f92ffd700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 0x7f4f880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:35.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.798+0000 7f4fa13d2700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 0x7f4f880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:35.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.798+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4f8c0cc240 con 0x7f4f9c0feab0 2023-07-21T22:15:35.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.799+0000 7f4fa13d2700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 0x7f4f880a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c197510 tx=0x7f4f98009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:35.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.803+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4f8c095af0 con 0x7f4f9c0feab0 2023-07-21T22:15:36.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.984+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f9c067fd0 con 0x7f4f880a11e0 2023-07-21T22:15:36.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.994+0000 7f4f92ffd700 1 -- 172.21.15.93:0/1949934834 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339880 (secure 0 0 0) 0x7f4f9c067fd0 con 0x7f4f880a11e0 2023-07-21T22:15:36.041 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 msgr2=0x7f4f880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 0x7f4f880a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c197510 tx=0x7f4f98009190 comp rx=0 tx=0).stop 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 msgr2=0x7f4f9c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b500 tx=0x7f4f8c00b810 comp rx=0 tx=0).stop 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 shutdown_connections 2023-07-21T22:15:36.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f9c104ab0 0x7f4f9c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f9c0feab0 0x7f4f9c195ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f4f880a11e0 0x7f4f880a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 --2- 172.21.15.93:0/1949934834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f9c0ff460 0x7f4f9c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 >> 172.21.15.93:0/1949934834 conn(0x7f4f9c075b70 msgr2=0x7f4f9c102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:36.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 shutdown_connections 2023-07-21T22:15:36.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:35.997+0000 7f4fa3e37700 1 -- 172.21.15.93:0/1949934834 wait complete. 2023-07-21T22:15:36.045 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:15:36.370 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-07-21T22:15:35.219213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654342,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":536,"num_read_kb":392,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336036,"kb_used_data":12612,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605788,"statfs":{"total":1151940427776,"available":1151596326912,"internally_reserved":0,"allocated":12914688,"data_stored":7035801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":57,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002741"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249025+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:14:52.249025+0000","last_peered":"2023-07-21T22:14:52.249025+0000","last_clean":"2023-07-21T22:14:52.249025+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:14:52.249025+0000","last_undegraded":"2023-07-21T22:14:52.249025+0000","last_fullsized":"2023-07-21T22:14:52.249025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:33.953568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324185+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:14:52.324185+0000","last_peered":"2023-07-21T22:14:52.324185+0000","last_clean":"2023-07-21T22:14:52.324185+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:14:52.324185+0000","last_undegraded":"2023-07-21T22:14:52.324185+0000","last_fullsized":"2023-07-21T22:14:52.324185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:29.855484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323763+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.323763+0000","last_peered":"2023-07-21T22:14:52.323763+0000","last_clean":"2023-07-21T22:14:52.323763+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:14:52.323763+0000","last_undegraded":"2023-07-21T22:14:52.323763+0000","last_fullsized":"2023-07-21T22:14:52.323763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:27.021961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324294+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:14:52.324294+0000","last_peered":"2023-07-21T22:14:52.324294+0000","last_clean":"2023-07-21T22:14:52.324294+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:14:52.324294+0000","last_undegraded":"2023-07-21T22:14:52.324294+0000","last_fullsized":"2023-07-21T22:14:52.324294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:59:03.712988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550902+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:14:53.550902+0000","last_peered":"2023-07-21T22:14:53.550902+0000","last_clean":"2023-07-21T22:14:53.550902+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:14:53.550902+0000","last_undegraded":"2023-07-21T22:14:53.550902+0000","last_fullsized":"2023-07-21T22:14:53.550902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:00.017709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323849+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:14:52.323849+0000","last_peered":"2023-07-21T22:14:52.323849+0000","last_clean":"2023-07-21T22:14:52.323849+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:14:52.323849+0000","last_undegraded":"2023-07-21T22:14:52.323849+0000","last_fullsized":"2023-07-21T22:14:52.323849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:58.487301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323957+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:14:52.323957+0000","last_peered":"2023-07-21T22:14:52.323957+0000","last_clean":"2023-07-21T22:14:52.323957+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:14:52.323957+0000","last_undegraded":"2023-07-21T22:14:52.323957+0000","last_fullsized":"2023-07-21T22:14:52.323957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:02.172487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248538+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:14:52.248538+0000","last_peered":"2023-07-21T22:14:52.248538+0000","last_clean":"2023-07-21T22:14:52.248538+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:14:52.248538+0000","last_undegraded":"2023-07-21T22:14:52.248538+0000","last_fullsized":"2023-07-21T22:14:52.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:54.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249436+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:14:52.249436+0000","last_peered":"2023-07-21T22:14:52.249436+0000","last_clean":"2023-07-21T22:14:52.249436+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:14:52.249436+0000","last_undegraded":"2023-07-21T22:14:52.249436+0000","last_fullsized":"2023-07-21T22:14:52.249436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:33:03.512159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324232+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:14:52.324232+0000","last_peered":"2023-07-21T22:14:52.324232+0000","last_clean":"2023-07-21T22:14:52.324232+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:14:52.324232+0000","last_undegraded":"2023-07-21T22:14:52.324232+0000","last_fullsized":"2023-07-21T22:14:52.324232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:33.917374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324415+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:14:52.324415+0000","last_peered":"2023-07-21T22:14:52.324415+0000","last_clean":"2023-07-21T22:14:52.324415+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:14:52.324415+0000","last_undegraded":"2023-07-21T22:14:52.324415+0000","last_fullsized":"2023-07-21T22:14:52.324415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:42.648086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324156+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.324156+0000","last_peered":"2023-07-21T22:14:52.324156+0000","last_clean":"2023-07-21T22:14:52.324156+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:14:52.324156+0000","last_undegraded":"2023-07-21T22:14:52.324156+0000","last_fullsized":"2023-07-21T22:14:52.324156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:12.914647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323062+0000","last_change":"2023-07-21T22:14:51.245228+0000","last_active":"2023-07-21T22:14:52.323062+0000","last_peered":"2023-07-21T22:14:52.323062+0000","last_clean":"2023-07-21T22:14:52.323062+0000","last_became_active":"2023-07-21T22:14:51.244919+0000","last_became_peered":"2023-07-21T22:14:51.244919+0000","last_unstale":"2023-07-21T22:14:52.323062+0000","last_undegraded":"2023-07-21T22:14:52.323062+0000","last_fullsized":"2023-07-21T22:14:52.323062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:21:11.471385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323500+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:14:52.323500+0000","last_peered":"2023-07-21T22:14:52.323500+0000","last_clean":"2023-07-21T22:14:52.323500+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:14:52.323500+0000","last_undegraded":"2023-07-21T22:14:52.323500+0000","last_fullsized":"2023-07-21T22:14:52.323500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:05:33.532014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323093+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:14:52.323093+0000","last_peered":"2023-07-21T22:14:52.323093+0000","last_clean":"2023-07-21T22:14:52.323093+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:14:52.323093+0000","last_undegraded":"2023-07-21T22:14:52.323093+0000","last_fullsized":"2023-07-21T22:14:52.323093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:57.567468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248840+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:14:52.248840+0000","last_peered":"2023-07-21T22:14:52.248840+0000","last_clean":"2023-07-21T22:14:52.248840+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:14:52.248840+0000","last_undegraded":"2023-07-21T22:14:52.248840+0000","last_fullsized":"2023-07-21T22:14:52.248840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:28:05.983432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249184+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:14:52.249184+0000","last_peered":"2023-07-21T22:14:52.249184+0000","last_clean":"2023-07-21T22:14:52.249184+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:14:52.249184+0000","last_undegraded":"2023-07-21T22:14:52.249184+0000","last_fullsized":"2023-07-21T22:14:52.249184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:35.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323352+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:14:52.323352+0000","last_peered":"2023-07-21T22:14:52.323352+0000","last_clean":"2023-07-21T22:14:52.323352+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:14:52.323352+0000","last_undegraded":"2023-07-21T22:14:52.323352+0000","last_fullsized":"2023-07-21T22:14:52.323352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:31.713111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323930+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:14:52.323930+0000","last_peered":"2023-07-21T22:14:52.323930+0000","last_clean":"2023-07-21T22:14:52.323930+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:14:52.323930+0000","last_undegraded":"2023-07-21T22:14:52.323930+0000","last_fullsized":"2023-07-21T22:14:52.323930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:45.482633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.066194+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:15:32.066194+0000","last_peered":"2023-07-21T22:15:32.066194+0000","last_clean":"2023-07-21T22:15:32.066194+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:15:32.066194+0000","last_undegraded":"2023-07-21T22:15:32.066194+0000","last_fullsized":"2023-07-21T22:15:32.066194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:42.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248992+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:14:52.248992+0000","last_peered":"2023-07-21T22:14:52.248992+0000","last_clean":"2023-07-21T22:14:52.248992+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:14:52.248992+0000","last_undegraded":"2023-07-21T22:14:52.248992+0000","last_fullsized":"2023-07-21T22:14:52.248992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:57.089271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249080+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:14:52.249080+0000","last_peered":"2023-07-21T22:14:52.249080+0000","last_clean":"2023-07-21T22:14:52.249080+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:14:52.249080+0000","last_undegraded":"2023-07-21T22:14:52.249080+0000","last_fullsized":"2023-07-21T22:14:52.249080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:17.931445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324904+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:14:52.324904+0000","last_peered":"2023-07-21T22:14:52.324904+0000","last_clean":"2023-07-21T22:14:52.324904+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:14:52.324904+0000","last_undegraded":"2023-07-21T22:14:52.324904+0000","last_fullsized":"2023-07-21T22:14:52.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:07.388393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.007909+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:14:57.007909+0000","last_peered":"2023-07-21T22:14:57.007909+0000","last_clean":"2023-07-21T22:14:57.007909+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:14:57.007909+0000","last_undegraded":"2023-07-21T22:14:57.007909+0000","last_fullsized":"2023-07-21T22:14:57.007909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:45.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248983+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:14:52.248983+0000","last_peered":"2023-07-21T22:14:52.248983+0000","last_clean":"2023-07-21T22:14:52.248983+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:14:52.248983+0000","last_undegraded":"2023-07-21T22:14:52.248983+0000","last_fullsized":"2023-07-21T22:14:52.248983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:33.576139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.007984+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:14:57.007984+0000","last_peered":"2023-07-21T22:14:57.007984+0000","last_clean":"2023-07-21T22:14:57.007984+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:14:57.007984+0000","last_undegraded":"2023-07-21T22:14:57.007984+0000","last_fullsized":"2023-07-21T22:14:57.007984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:04:03.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324394+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.324394+0000","last_peered":"2023-07-21T22:14:52.324394+0000","last_clean":"2023-07-21T22:14:52.324394+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:14:52.324394+0000","last_undegraded":"2023-07-21T22:14:52.324394+0000","last_fullsized":"2023-07-21T22:14:52.324394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:10:13.245800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:31.049706+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:15:31.049706+0000","last_peered":"2023-07-21T22:15:31.049706+0000","last_clean":"2023-07-21T22:15:31.049706+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:15:31.049706+0000","last_undegraded":"2023-07-21T22:15:31.049706+0000","last_fullsized":"2023-07-21T22:15:31.049706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:15:58.146646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551008+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:14:53.551008+0000","last_peered":"2023-07-21T22:14:53.551008+0000","last_clean":"2023-07-21T22:14:53.551008+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:14:53.551008+0000","last_undegraded":"2023-07-21T22:14:53.551008+0000","last_fullsized":"2023-07-21T22:14:53.551008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:04.181744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285879+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:14:56.285879+0000","last_peered":"2023-07-21T22:14:56.285879+0000","last_clean":"2023-07-21T22:14:56.285879+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:14:56.285879+0000","last_undegraded":"2023-07-21T22:14:56.285879+0000","last_fullsized":"2023-07-21T22:14:56.285879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:46:03.474285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324013+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:14:52.324013+0000","last_peered":"2023-07-21T22:14:52.324013+0000","last_clean":"2023-07-21T22:14:52.324013+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:14:52.324013+0000","last_undegraded":"2023-07-21T22:14:52.324013+0000","last_fullsized":"2023-07-21T22:14:52.324013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:15:10.615938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.280021+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:14:55.280021+0000","last_peered":"2023-07-21T22:14:55.280021+0000","last_clean":"2023-07-21T22:14:55.280021+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:14:55.280021+0000","last_undegraded":"2023-07-21T22:14:55.280021+0000","last_fullsized":"2023-07-21T22:14:55.280021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:01.057949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248598+0000","last_change":"2023-07-21T22:14:51.248592+0000","last_active":"2023-07-21T22:14:52.248598+0000","last_peered":"2023-07-21T22:14:52.248598+0000","last_clean":"2023-07-21T22:14:52.248598+0000","last_became_active":"2023-07-21T22:14:51.247932+0000","last_became_peered":"2023-07-21T22:14:51.247932+0000","last_unstale":"2023-07-21T22:14:52.248598+0000","last_undegraded":"2023-07-21T22:14:52.248598+0000","last_fullsized":"2023-07-21T22:14:52.248598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:01.815385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001359+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:14:57.001359+0000","last_peered":"2023-07-21T22:14:57.001359+0000","last_clean":"2023-07-21T22:14:57.001359+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:14:57.001359+0000","last_undegraded":"2023-07-21T22:14:57.001359+0000","last_fullsized":"2023-07-21T22:14:57.001359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:40.285840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322887+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:14:52.322887+0000","last_peered":"2023-07-21T22:14:52.322887+0000","last_clean":"2023-07-21T22:14:52.322887+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:14:52.322887+0000","last_undegraded":"2023-07-21T22:14:52.322887+0000","last_fullsized":"2023-07-21T22:14:52.322887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:55:36.899847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.050257+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:15:32.050257+0000","last_peered":"2023-07-21T22:15:32.050257+0000","last_clean":"2023-07-21T22:15:32.050257+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:15:32.050257+0000","last_undegraded":"2023-07-21T22:15:32.050257+0000","last_fullsized":"2023-07-21T22:15:32.050257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:14.250309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249338+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:14:52.249338+0000","last_peered":"2023-07-21T22:14:52.249338+0000","last_clean":"2023-07-21T22:14:52.249338+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:14:52.249338+0000","last_undegraded":"2023-07-21T22:14:52.249338+0000","last_fullsized":"2023-07-21T22:14:52.249338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:06.815314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006887+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:14:57.006887+0000","last_peered":"2023-07-21T22:14:57.006887+0000","last_clean":"2023-07-21T22:14:57.006887+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:14:57.006887+0000","last_undegraded":"2023-07-21T22:14:57.006887+0000","last_fullsized":"2023-07-21T22:14:57.006887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:38:48.158170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324814+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:14:52.324814+0000","last_peered":"2023-07-21T22:14:52.324814+0000","last_clean":"2023-07-21T22:14:52.324814+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:14:52.324814+0000","last_undegraded":"2023-07-21T22:14:52.324814+0000","last_fullsized":"2023-07-21T22:14:52.324814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:53:48.638340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284307+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:14:56.284307+0000","last_peered":"2023-07-21T22:14:56.284307+0000","last_clean":"2023-07-21T22:14:56.284307+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:14:56.284307+0000","last_undegraded":"2023-07-21T22:14:56.284307+0000","last_fullsized":"2023-07-21T22:14:56.284307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:15.490891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249055+0000","last_change":"2023-07-21T22:14:51.249768+0000","last_active":"2023-07-21T22:14:52.249055+0000","last_peered":"2023-07-21T22:14:52.249055+0000","last_clean":"2023-07-21T22:14:52.249055+0000","last_became_active":"2023-07-21T22:14:51.249503+0000","last_became_peered":"2023-07-21T22:14:51.249503+0000","last_unstale":"2023-07-21T22:14:52.249055+0000","last_undegraded":"2023-07-21T22:14:52.249055+0000","last_fullsized":"2023-07-21T22:14:52.249055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:39:15.294044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.050257+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:15:32.050257+0000","last_peered":"2023-07-21T22:15:32.050257+0000","last_clean":"2023-07-21T22:15:32.050257+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:15:32.050257+0000","last_undegraded":"2023-07-21T22:15:32.050257+0000","last_fullsized":"2023-07-21T22:15:32.050257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:38.675168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57498,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":97,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551147+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:14:53.551147+0000","last_peered":"2023-07-21T22:14:53.551147+0000","last_clean":"2023-07-21T22:14:53.551147+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:14:53.551147+0000","last_undegraded":"2023-07-21T22:14:53.551147+0000","last_fullsized":"2023-07-21T22:14:53.551147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:51.431054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006578+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:14:57.006578+0000","last_peered":"2023-07-21T22:14:57.006578+0000","last_clean":"2023-07-21T22:14:57.006578+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:14:57.006578+0000","last_undegraded":"2023-07-21T22:14:57.006578+0000","last_fullsized":"2023-07-21T22:14:57.006578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:55.930555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323112+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:14:52.323112+0000","last_peered":"2023-07-21T22:14:52.323112+0000","last_clean":"2023-07-21T22:14:52.323112+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:14:52.323112+0000","last_undegraded":"2023-07-21T22:14:52.323112+0000","last_fullsized":"2023-07-21T22:14:52.323112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:57:57.488142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006376+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:14:57.006376+0000","last_peered":"2023-07-21T22:14:57.006376+0000","last_clean":"2023-07-21T22:14:57.006376+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:14:57.006376+0000","last_undegraded":"2023-07-21T22:14:57.006376+0000","last_fullsized":"2023-07-21T22:14:57.006376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:28.876044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324774+0000","last_change":"2023-07-21T22:14:51.258940+0000","last_active":"2023-07-21T22:14:52.324774+0000","last_peered":"2023-07-21T22:14:52.324774+0000","last_clean":"2023-07-21T22:14:52.324774+0000","last_became_active":"2023-07-21T22:14:51.258595+0000","last_became_peered":"2023-07-21T22:14:51.258595+0000","last_unstale":"2023-07-21T22:14:52.324774+0000","last_undegraded":"2023-07-21T22:14:52.324774+0000","last_fullsized":"2023-07-21T22:14:52.324774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:39.297921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:30.047983+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:15:30.047983+0000","last_peered":"2023-07-21T22:15:30.047983+0000","last_clean":"2023-07-21T22:15:30.047983+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:15:30.047983+0000","last_undegraded":"2023-07-21T22:15:30.047983+0000","last_fullsized":"2023-07-21T22:15:30.047983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:39.148790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323808+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:14:52.323808+0000","last_peered":"2023-07-21T22:14:52.323808+0000","last_clean":"2023-07-21T22:14:52.323808+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:14:52.323808+0000","last_undegraded":"2023-07-21T22:14:52.323808+0000","last_fullsized":"2023-07-21T22:14:52.323808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:38.761579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.997294+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:14:56.997294+0000","last_peered":"2023-07-21T22:14:56.997294+0000","last_clean":"2023-07-21T22:14:56.997294+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:14:56.997294+0000","last_undegraded":"2023-07-21T22:14:56.997294+0000","last_fullsized":"2023-07-21T22:14:56.997294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:01:49.310753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323584+0000","last_change":"2023-07-21T22:14:51.252408+0000","last_active":"2023-07-21T22:14:52.323584+0000","last_peered":"2023-07-21T22:14:52.323584+0000","last_clean":"2023-07-21T22:14:52.323584+0000","last_became_active":"2023-07-21T22:14:51.252074+0000","last_became_peered":"2023-07-21T22:14:51.252074+0000","last_unstale":"2023-07-21T22:14:52.323584+0000","last_undegraded":"2023-07-21T22:14:52.323584+0000","last_fullsized":"2023-07-21T22:14:52.323584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.877759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.997626+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:14:56.997626+0000","last_peered":"2023-07-21T22:14:56.997626+0000","last_clean":"2023-07-21T22:14:56.997626+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:14:56.997626+0000","last_undegraded":"2023-07-21T22:14:56.997626+0000","last_fullsized":"2023-07-21T22:14:56.997626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:36.286445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249168+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:14:52.249168+0000","last_peered":"2023-07-21T22:14:52.249168+0000","last_clean":"2023-07-21T22:14:52.249168+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:14:52.249168+0000","last_undegraded":"2023-07-21T22:14:52.249168+0000","last_fullsized":"2023-07-21T22:14:52.249168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:12.075583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.101685+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:14:53.101685+0000","last_peered":"2023-07-21T22:14:53.101685+0000","last_clean":"2023-07-21T22:14:53.101685+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:14:53.101685+0000","last_undegraded":"2023-07-21T22:14:53.101685+0000","last_fullsized":"2023-07-21T22:14:53.101685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:03.657128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.530952+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:14:52.530952+0000","last_peered":"2023-07-21T22:14:52.530952+0000","last_clean":"2023-07-21T22:14:52.530952+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:14:52.530952+0000","last_undegraded":"2023-07-21T22:14:52.530952+0000","last_fullsized":"2023-07-21T22:14:52.530952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:11:20.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:33.067987+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:15:33.067987+0000","last_peered":"2023-07-21T22:15:33.067987+0000","last_clean":"2023-07-21T22:15:33.067987+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:15:33.067987+0000","last_undegraded":"2023-07-21T22:15:33.067987+0000","last_fullsized":"2023-07-21T22:15:33.067987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:37.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324083+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:14:52.324083+0000","last_peered":"2023-07-21T22:14:52.324083+0000","last_clean":"2023-07-21T22:14:52.324083+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:14:52.324083+0000","last_undegraded":"2023-07-21T22:14:52.324083+0000","last_fullsized":"2023-07-21T22:14:52.324083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:57:09.279672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323831+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:14:52.323831+0000","last_peered":"2023-07-21T22:14:52.323831+0000","last_clean":"2023-07-21T22:14:52.323831+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:14:52.323831+0000","last_undegraded":"2023-07-21T22:14:52.323831+0000","last_fullsized":"2023-07-21T22:14:52.323831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:31.048621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:33.051320+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:15:33.051320+0000","last_peered":"2023-07-21T22:15:33.051320+0000","last_clean":"2023-07-21T22:15:33.051320+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:15:33.051320+0000","last_undegraded":"2023-07-21T22:15:33.051320+0000","last_fullsized":"2023-07-21T22:15:33.051320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:39:31.032230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248803+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:14:52.248803+0000","last_peered":"2023-07-21T22:14:52.248803+0000","last_clean":"2023-07-21T22:14:52.248803+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:14:52.248803+0000","last_undegraded":"2023-07-21T22:14:52.248803+0000","last_fullsized":"2023-07-21T22:14:52.248803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:06.145398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002580+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:57.002580+0000","last_peered":"2023-07-21T22:14:57.002580+0000","last_clean":"2023-07-21T22:14:57.002580+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:14:57.002580+0000","last_undegraded":"2023-07-21T22:14:57.002580+0000","last_fullsized":"2023-07-21T22:14:57.002580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:48.946604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323556+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:14:52.323556+0000","last_peered":"2023-07-21T22:14:52.323556+0000","last_clean":"2023-07-21T22:14:52.323556+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:14:52.323556+0000","last_undegraded":"2023-07-21T22:14:52.323556+0000","last_fullsized":"2023-07-21T22:14:52.323556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:04.146166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323206+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:14:52.323206+0000","last_peered":"2023-07-21T22:14:52.323206+0000","last_clean":"2023-07-21T22:14:52.323206+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:14:52.323206+0000","last_undegraded":"2023-07-21T22:14:52.323206+0000","last_fullsized":"2023-07-21T22:14:52.323206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:14:26.470268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.279353+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:14:55.279353+0000","last_peered":"2023-07-21T22:14:55.279353+0000","last_clean":"2023-07-21T22:14:55.279353+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:14:55.279353+0000","last_undegraded":"2023-07-21T22:14:55.279353+0000","last_fullsized":"2023-07-21T22:14:55.279353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:41.327520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996715+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:14:56.996715+0000","last_peered":"2023-07-21T22:14:56.996715+0000","last_clean":"2023-07-21T22:14:56.996715+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:14:56.996715+0000","last_undegraded":"2023-07-21T22:14:56.996715+0000","last_fullsized":"2023-07-21T22:14:56.996715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:00.386436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323125+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:14:52.323125+0000","last_peered":"2023-07-21T22:14:52.323125+0000","last_clean":"2023-07-21T22:14:52.323125+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:14:52.323125+0000","last_undegraded":"2023-07-21T22:14:52.323125+0000","last_fullsized":"2023-07-21T22:14:52.323125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:25.036430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:03.550435+0000","last_change":"2023-07-21T22:14:50.231962+0000","last_active":"2023-07-21T22:15:03.550435+0000","last_peered":"2023-07-21T22:15:03.550435+0000","last_clean":"2023-07-21T22:15:03.550435+0000","last_became_active":"2023-07-21T22:14:50.231638+0000","last_became_peered":"2023-07-21T22:14:50.231638+0000","last_unstale":"2023-07-21T22:15:03.550435+0000","last_undegraded":"2023-07-21T22:15:03.550435+0000","last_fullsized":"2023-07-21T22:15:03.550435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:18.134292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249093+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:14:52.249093+0000","last_peered":"2023-07-21T22:14:52.249093+0000","last_clean":"2023-07-21T22:14:52.249093+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:14:52.249093+0000","last_undegraded":"2023-07-21T22:14:52.249093+0000","last_fullsized":"2023-07-21T22:14:52.249093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:15.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002623+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:14:57.002623+0000","last_peered":"2023-07-21T22:14:57.002623+0000","last_clean":"2023-07-21T22:14:57.002623+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:14:57.002623+0000","last_undegraded":"2023-07-21T22:14:57.002623+0000","last_fullsized":"2023-07-21T22:14:57.002623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:43.076462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324274+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:14:52.324274+0000","last_peered":"2023-07-21T22:14:52.324274+0000","last_clean":"2023-07-21T22:14:52.324274+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:14:52.324274+0000","last_undegraded":"2023-07-21T22:14:52.324274+0000","last_fullsized":"2023-07-21T22:14:52.324274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:16.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270269+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:14:54.270269+0000","last_peered":"2023-07-21T22:14:54.270269+0000","last_clean":"2023-07-21T22:14:54.270269+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:14:54.270269+0000","last_undegraded":"2023-07-21T22:14:54.270269+0000","last_fullsized":"2023-07-21T22:14:54.270269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:06:47.274452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550789+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:14:53.550789+0000","last_peered":"2023-07-21T22:14:53.550789+0000","last_clean":"2023-07-21T22:14:53.550789+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:14:53.550789+0000","last_undegraded":"2023-07-21T22:14:53.550789+0000","last_fullsized":"2023-07-21T22:14:53.550789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:02:40.632894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.005576+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:14:57.005576+0000","last_peered":"2023-07-21T22:14:57.005576+0000","last_clean":"2023-07-21T22:14:57.005576+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:14:57.005576+0000","last_undegraded":"2023-07-21T22:14:57.005576+0000","last_fullsized":"2023-07-21T22:14:57.005576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:31.455547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248754+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:14:52.248754+0000","last_peered":"2023-07-21T22:14:52.248754+0000","last_clean":"2023-07-21T22:14:52.248754+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:14:52.248754+0000","last_undegraded":"2023-07-21T22:14:52.248754+0000","last_fullsized":"2023-07-21T22:14:52.248754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:25.484174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:59.922284+0000","last_change":"2023-07-21T22:14:50.229923+0000","last_active":"2023-07-21T22:14:59.922284+0000","last_peered":"2023-07-21T22:14:59.922284+0000","last_clean":"2023-07-21T22:14:59.922284+0000","last_became_active":"2023-07-21T22:14:50.229645+0000","last_became_peered":"2023-07-21T22:14:50.229645+0000","last_unstale":"2023-07-21T22:14:59.922284+0000","last_undegraded":"2023-07-21T22:14:59.922284+0000","last_fullsized":"2023-07-21T22:14:59.922284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:36:56.239739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248731+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:14:52.248731+0000","last_peered":"2023-07-21T22:14:52.248731+0000","last_clean":"2023-07-21T22:14:52.248731+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:14:52.248731+0000","last_undegraded":"2023-07-21T22:14:52.248731+0000","last_fullsized":"2023-07-21T22:14:52.248731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:20.211619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270798+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:14:54.270798+0000","last_peered":"2023-07-21T22:14:54.270798+0000","last_clean":"2023-07-21T22:14:54.270798+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:14:54.270798+0000","last_undegraded":"2023-07-21T22:14:54.270798+0000","last_fullsized":"2023-07-21T22:14:54.270798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:01.500609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324713+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:14:52.324713+0000","last_peered":"2023-07-21T22:14:52.324713+0000","last_clean":"2023-07-21T22:14:52.324713+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:14:52.324713+0000","last_undegraded":"2023-07-21T22:14:52.324713+0000","last_fullsized":"2023-07-21T22:14:52.324713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:17.553733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:31.049519+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:15:31.049519+0000","last_peered":"2023-07-21T22:15:31.049519+0000","last_clean":"2023-07-21T22:15:31.049519+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:15:31.049519+0000","last_undegraded":"2023-07-21T22:15:31.049519+0000","last_fullsized":"2023-07-21T22:15:31.049519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:15.167223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324532+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:14:52.324532+0000","last_peered":"2023-07-21T22:14:52.324532+0000","last_clean":"2023-07-21T22:14:52.324532+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:14:52.324532+0000","last_undegraded":"2023-07-21T22:14:52.324532+0000","last_fullsized":"2023-07-21T22:14:52.324532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:33.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996861+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:14:56.996861+0000","last_peered":"2023-07-21T22:14:56.996861+0000","last_clean":"2023-07-21T22:14:56.996861+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:14:56.996861+0000","last_undegraded":"2023-07-21T22:14:56.996861+0000","last_fullsized":"2023-07-21T22:14:56.996861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.897090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.993320+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:14:54.993320+0000","last_peered":"2023-07-21T22:14:54.993320+0000","last_clean":"2023-07-21T22:14:54.993320+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:14:54.993320+0000","last_undegraded":"2023-07-21T22:14:54.993320+0000","last_fullsized":"2023-07-21T22:14:54.993320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:52.794435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001568+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:14:57.001568+0000","last_peered":"2023-07-21T22:14:57.001568+0000","last_clean":"2023-07-21T22:14:57.001568+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:14:57.001568+0000","last_undegraded":"2023-07-21T22:14:57.001568+0000","last_fullsized":"2023-07-21T22:14:57.001568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:56.943909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324655+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:14:52.324655+0000","last_peered":"2023-07-21T22:14:52.324655+0000","last_clean":"2023-07-21T22:14:52.324655+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:14:52.324655+0000","last_undegraded":"2023-07-21T22:14:52.324655+0000","last_fullsized":"2023-07-21T22:14:52.324655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:05:57.078423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248843+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:14:52.248843+0000","last_peered":"2023-07-21T22:14:52.248843+0000","last_clean":"2023-07-21T22:14:52.248843+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:14:52.248843+0000","last_undegraded":"2023-07-21T22:14:52.248843+0000","last_fullsized":"2023-07-21T22:14:52.248843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:49:10.848487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324305+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:14:52.324305+0000","last_peered":"2023-07-21T22:14:52.324305+0000","last_clean":"2023-07-21T22:14:52.324305+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:14:52.324305+0000","last_undegraded":"2023-07-21T22:14:52.324305+0000","last_fullsized":"2023-07-21T22:14:52.324305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:16:56.773704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323602+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:14:52.323602+0000","last_peered":"2023-07-21T22:14:52.323602+0000","last_clean":"2023-07-21T22:14:52.323602+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:14:52.323602+0000","last_undegraded":"2023-07-21T22:14:52.323602+0000","last_fullsized":"2023-07-21T22:14:52.323602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:58.733025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324433+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:14:52.324433+0000","last_peered":"2023-07-21T22:14:52.324433+0000","last_clean":"2023-07-21T22:14:52.324433+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:14:52.324433+0000","last_undegraded":"2023-07-21T22:14:52.324433+0000","last_fullsized":"2023-07-21T22:14:52.324433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:51.861062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322982+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:14:52.322982+0000","last_peered":"2023-07-21T22:14:52.322982+0000","last_clean":"2023-07-21T22:14:52.322982+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:14:52.322982+0000","last_undegraded":"2023-07-21T22:14:52.322982+0000","last_fullsized":"2023-07-21T22:14:52.322982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:29.650653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248998+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:14:52.248998+0000","last_peered":"2023-07-21T22:14:52.248998+0000","last_clean":"2023-07-21T22:14:52.248998+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:14:52.248998+0000","last_undegraded":"2023-07-21T22:14:52.248998+0000","last_fullsized":"2023-07-21T22:14:52.248998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:30:39.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324003+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:14:52.324003+0000","last_peered":"2023-07-21T22:14:52.324003+0000","last_clean":"2023-07-21T22:14:52.324003+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:14:52.324003+0000","last_undegraded":"2023-07-21T22:14:52.324003+0000","last_fullsized":"2023-07-21T22:14:52.324003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:23.704220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249061+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:14:52.249061+0000","last_peered":"2023-07-21T22:14:52.249061+0000","last_clean":"2023-07-21T22:14:52.249061+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:14:52.249061+0000","last_undegraded":"2023-07-21T22:14:52.249061+0000","last_fullsized":"2023-07-21T22:14:52.249061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:22.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323571+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:14:52.323571+0000","last_peered":"2023-07-21T22:14:52.323571+0000","last_clean":"2023-07-21T22:14:52.323571+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:14:52.323571+0000","last_undegraded":"2023-07-21T22:14:52.323571+0000","last_fullsized":"2023-07-21T22:14:52.323571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:35:26.714623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248660+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:14:52.248660+0000","last_peered":"2023-07-21T22:14:52.248660+0000","last_clean":"2023-07-21T22:14:52.248660+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:14:52.248660+0000","last_undegraded":"2023-07-21T22:14:52.248660+0000","last_fullsized":"2023-07-21T22:14:52.248660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:27:40.710691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249020+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.249020+0000","last_peered":"2023-07-21T22:14:52.249020+0000","last_clean":"2023-07-21T22:14:52.249020+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:14:52.249020+0000","last_undegraded":"2023-07-21T22:14:52.249020+0000","last_fullsized":"2023-07-21T22:14:52.249020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:45.083477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323134+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:14:52.323134+0000","last_peered":"2023-07-21T22:14:52.323134+0000","last_clean":"2023-07-21T22:14:52.323134+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:14:52.323134+0000","last_undegraded":"2023-07-21T22:14:52.323134+0000","last_fullsized":"2023-07-21T22:14:52.323134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:19:31.888652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248874+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:14:52.248874+0000","last_peered":"2023-07-21T22:14:52.248874+0000","last_clean":"2023-07-21T22:14:52.248874+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:14:52.248874+0000","last_undegraded":"2023-07-21T22:14:52.248874+0000","last_fullsized":"2023-07-21T22:14:52.248874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:28.191291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63728,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":400,"num_read_kb":273,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.62},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.55500000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.69399999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.755}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1589248,"data_stored":1072491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.498},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74399999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.69399999999999995}]},{"osd":11,"last update":"Fri Jul 21 22:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.78200000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.308,"15min":0.308},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.879}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.746},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.84299999999999997}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":423983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.9480000000000004,"5min":5.9480000000000004,"15min":5.9480000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.2400000000000002,"5min":4.2400000000000002,"15min":4.2400000000000002},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":0.629},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1820000000000004,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.754}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.69299999999999995,"15min":0.63400000000000001},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.4550000000000001,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.91100000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.874},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.97699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1015972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86799999999999999,"5min":1.024,"15min":1.024},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.434}]},{"osd":5,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.628}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.56100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93200000000000005,"5min":1.173,"15min":1.173},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94699999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.64600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.49399999999999999,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":13.989000000000001,"5min":13.989000000000001,"15min":13.989000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.28399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57099999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.308}]},{"osd":1,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.70799999999999996,"15min":0.622},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.68600000000000005,"15min":0.59399999999999997},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":13.789999999999999,"5min":13.789999999999999,"15min":13.789999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.56899999999999995,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":14.023,"5min":14.023,"15min":14.023},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.64500000000000002,"15min":0.59999999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":7.1189999999999998,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.75800000000000001,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":13.956,"5min":13.956,"15min":13.956},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.65600000000000003,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78200000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.77900000000000003,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":7.2489999999999997,"5min":7.2489999999999997,"15min":7.2489999999999997},"last":0.52200000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.434}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":482016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.61899999999999999,"15min":0.57099999999999995},"min":{"1min":0.52300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.502}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.40100000000000002,"15min":0.371},"min":{"1min":0.33300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42699999999999999,"15min":0.39700000000000002},"min":{"1min":0.313,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67700000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90800000000000003,"5min":0.995,"15min":0.995},"last":0.753},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72799999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":424944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Jul 21 22:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.505},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.436},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.39300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442}]},{"osd":5,"last update":"Fri Jul 21 22:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78200000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.75,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74099999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40899999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:14:29 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.317},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57299999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.66200000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.629},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54300000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":482188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.39700000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.754},{"interface":"front","average":{"1min":0.88,"5min":0.63600000000000001,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.7210000000000001,"5min":4.7210000000000001,"15min":4.7210000000000001},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.46400000000000002,"15min":0.438},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.495},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.45800000000000002,"15min":0.44500000000000001},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.443}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":423159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35099999999999998,"15min":0.32300000000000001},"min":{"1min":0.255,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.37},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.378,"15min":0.33600000000000002},"min":{"1min":0.28000000000000003,"5min":0.19,"15min":0.19},"max":{"1min":1.232,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":1.512,"15min":1.512},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.034,"5min":0.63300000000000001,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.545,"5min":10.545,"15min":10.545},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.434,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92000000000000004,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.625},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.67000000000000004,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":10.446999999999999,"5min":10.446999999999999,"15min":10.446999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":1.839,"15min":1.839},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0409999999999999,"5min":1.653,"15min":1.653},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71799999999999997,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79400000000000004,"5min":0.998,"15min":0.998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.68100000000000005,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.869999999999999,"5min":10.869999999999999,"15min":10.869999999999999},"last":0.45200000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.32700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.51400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94199999999999995,"5min":1.22,"15min":1.22},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.49199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.55500000000000005,"5min":0.372,"15min":0.372},"max":{"1min":0.95099999999999996,"5min":1.105,"15min":1.105},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57799999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.5,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0169999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.69199999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:14:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.81799999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1523712,"data_stored":1016077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83199999999999996,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.5,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.439}]},{"osd":3,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.82599999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54900000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.876}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:15:36.378 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:15:36.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:36 smithi194 ceph-mon[98288]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:36.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:36 smithi134 ceph-mon[97873]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:36.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:36 smithi093 ceph-mon[96579]: pgmap v274: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:36.930 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:37.401 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:37 smithi093 ceph-mon[96579]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:37.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:37 smithi194 ceph-mon[98288]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:37.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:37 smithi134 ceph-mon[97873]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:37.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 -- 172.21.15.93:0/468270468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa800ff460 msgr2=0x7ffa8010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:37.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/468270468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa800ff460 0x7ffa8010b8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b0a0 tx=0x7ffa7800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 -- 172.21.15.93:0/468270468 shutdown_connections 2023-07-21T22:15:37.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/468270468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa800ff460 0x7ffa8010b8f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/468270468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa80104ab0 0x7ffa80104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/468270468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 0x7ffa800fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 -- 172.21.15.93:0/468270468 >> 172.21.15.93:0/468270468 conn(0x7ffa80075b70 msgr2=0x7ffa80075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 -- 172.21.15.93:0/468270468 shutdown_connections 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.942+0000 7ffa84b2e700 1 -- 172.21.15.93:0/468270468 wait complete. 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 Processor -- start 2023-07-21T22:15:37.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 -- start start 2023-07-21T22:15:37.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 0x7ffa8019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:37.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa800ff460 0x7ffa8019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:37.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:37.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80195050 con 0x7ffa80104ab0 2023-07-21T22:15:37.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa801951c0 con 0x7ffa800ff460 2023-07-21T22:15:37.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa84b2e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80195380 con 0x7ffa800feab0 2023-07-21T22:15:37.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:37.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42770/0 (socket says 172.21.15.93:42770) 2023-07-21T22:15:37.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 -- 172.21.15.93:0/2728144367 learned_addr learned my addr 172.21.15.93:0/2728144367 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7dd9b700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa800ff460 0x7ffa8019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7e59c700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 0x7ffa8019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 -- 172.21.15.93:0/2728144367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 msgr2=0x7ffa8019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 0x7ffa8019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 -- 172.21.15.93:0/2728144367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa800ff460 msgr2=0x7ffa8019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa800ff460 0x7ffa8019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:37.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.943+0000 7ffa7ed9d700 1 -- 172.21.15.93:0/2728144367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78009d00 con 0x7ffa80104ab0 2023-07-21T22:15:37.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa7ed9d700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa780062a0 tx=0x7ffa7800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:37.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800e040 con 0x7ffa80104ab0 2023-07-21T22:15:37.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa80195600 con 0x7ffa80104ab0 2023-07-21T22:15:37.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa78003c00 con 0x7ffa80104ab0 2023-07-21T22:15:37.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7801a700 con 0x7ffa80104ab0 2023-07-21T22:15:37.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.944+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa800681a0 con 0x7ffa80104ab0 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.945+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa80195820 con 0x7ffa80104ab0 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.946+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ffa78004020 con 0x7ffa80104ab0 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.946+0000 7ffa6f7fe700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 0x7ffa680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.947+0000 7ffa7e59c700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 0x7ffa680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.947+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffa780cfa80 con 0x7ffa80104ab0 2023-07-21T22:15:37.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.947+0000 7ffa7e59c700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 0x7ffa680a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005fd0 tx=0x7ffa74005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:37.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:37.951+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffa780991b0 con 0x7ffa80104ab0 2023-07-21T22:15:38.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.135+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa80105fd0 con 0x7ffa680a11e0 2023-07-21T22:15:38.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.145+0000 7ffa6f7fe700 1 -- 172.21.15.93:0/2728144367 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339954 (secure 0 0 0) 0x7ffa80105fd0 con 0x7ffa680a11e0 2023-07-21T22:15:38.193 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:38.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 msgr2=0x7ffa680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:38.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 0x7ffa680a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005fd0 tx=0x7ffa74005dd0 comp rx=0 tx=0).stop 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 msgr2=0x7ffa801949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffa780062a0 tx=0x7ffa7800cf00 comp rx=0 tx=0).stop 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 shutdown_connections 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa80104ab0 0x7ffa801949e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa800feab0 0x7ffa8019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:38.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa680a11e0 0x7ffa680a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:38.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 --2- 172.21.15.93:0/2728144367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa800ff460 0x7ffa8019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:38.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 >> 172.21.15.93:0/2728144367 conn(0x7ffa80075b70 msgr2=0x7ffa801058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:38.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 shutdown_connections 2023-07-21T22:15:38.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:38.148+0000 7ffa84b2e700 1 -- 172.21.15.93:0/2728144367 wait complete. 2023-07-21T22:15:38.197 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:15:38.583 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:38 smithi093 ceph-mon[96579]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:38.586 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2023-07-21T22:15:37.219639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654342,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":546,"num_read_kb":392,"num_write":416,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336036,"kb_used_data":12612,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605788,"statfs":{"total":1151940427776,"available":1151596326912,"internally_reserved":0,"allocated":12914688,"data_stored":7035801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":52,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002726"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249025+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:14:52.249025+0000","last_peered":"2023-07-21T22:14:52.249025+0000","last_clean":"2023-07-21T22:14:52.249025+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:14:52.249025+0000","last_undegraded":"2023-07-21T22:14:52.249025+0000","last_fullsized":"2023-07-21T22:14:52.249025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:33.953568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324185+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:14:52.324185+0000","last_peered":"2023-07-21T22:14:52.324185+0000","last_clean":"2023-07-21T22:14:52.324185+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:14:52.324185+0000","last_undegraded":"2023-07-21T22:14:52.324185+0000","last_fullsized":"2023-07-21T22:14:52.324185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:29.855484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323763+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.323763+0000","last_peered":"2023-07-21T22:14:52.323763+0000","last_clean":"2023-07-21T22:14:52.323763+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:14:52.323763+0000","last_undegraded":"2023-07-21T22:14:52.323763+0000","last_fullsized":"2023-07-21T22:14:52.323763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:27.021961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324294+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:14:52.324294+0000","last_peered":"2023-07-21T22:14:52.324294+0000","last_clean":"2023-07-21T22:14:52.324294+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:14:52.324294+0000","last_undegraded":"2023-07-21T22:14:52.324294+0000","last_fullsized":"2023-07-21T22:14:52.324294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:59:03.712988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550902+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:14:53.550902+0000","last_peered":"2023-07-21T22:14:53.550902+0000","last_clean":"2023-07-21T22:14:53.550902+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:14:53.550902+0000","last_undegraded":"2023-07-21T22:14:53.550902+0000","last_fullsized":"2023-07-21T22:14:53.550902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:00.017709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323849+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:14:52.323849+0000","last_peered":"2023-07-21T22:14:52.323849+0000","last_clean":"2023-07-21T22:14:52.323849+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:14:52.323849+0000","last_undegraded":"2023-07-21T22:14:52.323849+0000","last_fullsized":"2023-07-21T22:14:52.323849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:58.487301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323957+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:14:52.323957+0000","last_peered":"2023-07-21T22:14:52.323957+0000","last_clean":"2023-07-21T22:14:52.323957+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:14:52.323957+0000","last_undegraded":"2023-07-21T22:14:52.323957+0000","last_fullsized":"2023-07-21T22:14:52.323957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:02.172487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248538+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:14:52.248538+0000","last_peered":"2023-07-21T22:14:52.248538+0000","last_clean":"2023-07-21T22:14:52.248538+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:14:52.248538+0000","last_undegraded":"2023-07-21T22:14:52.248538+0000","last_fullsized":"2023-07-21T22:14:52.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:54.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249436+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:14:52.249436+0000","last_peered":"2023-07-21T22:14:52.249436+0000","last_clean":"2023-07-21T22:14:52.249436+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:14:52.249436+0000","last_undegraded":"2023-07-21T22:14:52.249436+0000","last_fullsized":"2023-07-21T22:14:52.249436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:33:03.512159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324232+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:14:52.324232+0000","last_peered":"2023-07-21T22:14:52.324232+0000","last_clean":"2023-07-21T22:14:52.324232+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:14:52.324232+0000","last_undegraded":"2023-07-21T22:14:52.324232+0000","last_fullsized":"2023-07-21T22:14:52.324232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:33.917374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324415+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:14:52.324415+0000","last_peered":"2023-07-21T22:14:52.324415+0000","last_clean":"2023-07-21T22:14:52.324415+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:14:52.324415+0000","last_undegraded":"2023-07-21T22:14:52.324415+0000","last_fullsized":"2023-07-21T22:14:52.324415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:44:42.648086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324156+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:14:52.324156+0000","last_peered":"2023-07-21T22:14:52.324156+0000","last_clean":"2023-07-21T22:14:52.324156+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:14:52.324156+0000","last_undegraded":"2023-07-21T22:14:52.324156+0000","last_fullsized":"2023-07-21T22:14:52.324156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:31:12.914647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323062+0000","last_change":"2023-07-21T22:14:51.245228+0000","last_active":"2023-07-21T22:14:52.323062+0000","last_peered":"2023-07-21T22:14:52.323062+0000","last_clean":"2023-07-21T22:14:52.323062+0000","last_became_active":"2023-07-21T22:14:51.244919+0000","last_became_peered":"2023-07-21T22:14:51.244919+0000","last_unstale":"2023-07-21T22:14:52.323062+0000","last_undegraded":"2023-07-21T22:14:52.323062+0000","last_fullsized":"2023-07-21T22:14:52.323062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:21:11.471385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323500+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:14:52.323500+0000","last_peered":"2023-07-21T22:14:52.323500+0000","last_clean":"2023-07-21T22:14:52.323500+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:14:52.323500+0000","last_undegraded":"2023-07-21T22:14:52.323500+0000","last_fullsized":"2023-07-21T22:14:52.323500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:05:33.532014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323093+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:14:52.323093+0000","last_peered":"2023-07-21T22:14:52.323093+0000","last_clean":"2023-07-21T22:14:52.323093+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:14:52.323093+0000","last_undegraded":"2023-07-21T22:14:52.323093+0000","last_fullsized":"2023-07-21T22:14:52.323093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:57.567468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248840+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:14:52.248840+0000","last_peered":"2023-07-21T22:14:52.248840+0000","last_clean":"2023-07-21T22:14:52.248840+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:14:52.248840+0000","last_undegraded":"2023-07-21T22:14:52.248840+0000","last_fullsized":"2023-07-21T22:14:52.248840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:28:05.983432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249184+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:14:52.249184+0000","last_peered":"2023-07-21T22:14:52.249184+0000","last_clean":"2023-07-21T22:14:52.249184+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:14:52.249184+0000","last_undegraded":"2023-07-21T22:14:52.249184+0000","last_fullsized":"2023-07-21T22:14:52.249184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:35.085962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323352+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:14:52.323352+0000","last_peered":"2023-07-21T22:14:52.323352+0000","last_clean":"2023-07-21T22:14:52.323352+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:14:52.323352+0000","last_undegraded":"2023-07-21T22:14:52.323352+0000","last_fullsized":"2023-07-21T22:14:52.323352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:31.713111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323930+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:14:52.323930+0000","last_peered":"2023-07-21T22:14:52.323930+0000","last_clean":"2023-07-21T22:14:52.323930+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:14:52.323930+0000","last_undegraded":"2023-07-21T22:14:52.323930+0000","last_fullsized":"2023-07-21T22:14:52.323930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:45.482633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.066194+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:15:32.066194+0000","last_peered":"2023-07-21T22:15:32.066194+0000","last_clean":"2023-07-21T22:15:32.066194+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:15:32.066194+0000","last_undegraded":"2023-07-21T22:15:32.066194+0000","last_fullsized":"2023-07-21T22:15:32.066194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:42.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248992+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:14:52.248992+0000","last_peered":"2023-07-21T22:14:52.248992+0000","last_clean":"2023-07-21T22:14:52.248992+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:14:52.248992+0000","last_undegraded":"2023-07-21T22:14:52.248992+0000","last_fullsized":"2023-07-21T22:14:52.248992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:57.089271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249080+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:14:52.249080+0000","last_peered":"2023-07-21T22:14:52.249080+0000","last_clean":"2023-07-21T22:14:52.249080+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:14:52.249080+0000","last_undegraded":"2023-07-21T22:14:52.249080+0000","last_fullsized":"2023-07-21T22:14:52.249080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:37:17.931445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324904+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:14:52.324904+0000","last_peered":"2023-07-21T22:14:52.324904+0000","last_clean":"2023-07-21T22:14:52.324904+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:14:52.324904+0000","last_undegraded":"2023-07-21T22:14:52.324904+0000","last_fullsized":"2023-07-21T22:14:52.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:07.388393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.007909+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:14:57.007909+0000","last_peered":"2023-07-21T22:14:57.007909+0000","last_clean":"2023-07-21T22:14:57.007909+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:14:57.007909+0000","last_undegraded":"2023-07-21T22:14:57.007909+0000","last_fullsized":"2023-07-21T22:14:57.007909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:45.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248983+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:14:52.248983+0000","last_peered":"2023-07-21T22:14:52.248983+0000","last_clean":"2023-07-21T22:14:52.248983+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:14:52.248983+0000","last_undegraded":"2023-07-21T22:14:52.248983+0000","last_fullsized":"2023-07-21T22:14:52.248983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:33.576139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.007984+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:14:57.007984+0000","last_peered":"2023-07-21T22:14:57.007984+0000","last_clean":"2023-07-21T22:14:57.007984+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:14:57.007984+0000","last_undegraded":"2023-07-21T22:14:57.007984+0000","last_fullsized":"2023-07-21T22:14:57.007984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:04:03.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324394+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.324394+0000","last_peered":"2023-07-21T22:14:52.324394+0000","last_clean":"2023-07-21T22:14:52.324394+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:14:52.324394+0000","last_undegraded":"2023-07-21T22:14:52.324394+0000","last_fullsized":"2023-07-21T22:14:52.324394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:10:13.245800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:36.057110+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:15:36.057110+0000","last_peered":"2023-07-21T22:15:36.057110+0000","last_clean":"2023-07-21T22:15:36.057110+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:15:36.057110+0000","last_undegraded":"2023-07-21T22:15:36.057110+0000","last_fullsized":"2023-07-21T22:15:36.057110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:15:58.146646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551008+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:14:53.551008+0000","last_peered":"2023-07-21T22:14:53.551008+0000","last_clean":"2023-07-21T22:14:53.551008+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:14:53.551008+0000","last_undegraded":"2023-07-21T22:14:53.551008+0000","last_fullsized":"2023-07-21T22:14:53.551008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:04.181744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.285879+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:14:56.285879+0000","last_peered":"2023-07-21T22:14:56.285879+0000","last_clean":"2023-07-21T22:14:56.285879+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:14:56.285879+0000","last_undegraded":"2023-07-21T22:14:56.285879+0000","last_fullsized":"2023-07-21T22:14:56.285879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:46:03.474285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324013+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:14:52.324013+0000","last_peered":"2023-07-21T22:14:52.324013+0000","last_clean":"2023-07-21T22:14:52.324013+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:14:52.324013+0000","last_undegraded":"2023-07-21T22:14:52.324013+0000","last_fullsized":"2023-07-21T22:14:52.324013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:15:10.615938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.280021+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:14:55.280021+0000","last_peered":"2023-07-21T22:14:55.280021+0000","last_clean":"2023-07-21T22:14:55.280021+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:14:55.280021+0000","last_undegraded":"2023-07-21T22:14:55.280021+0000","last_fullsized":"2023-07-21T22:14:55.280021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:01.057949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248598+0000","last_change":"2023-07-21T22:14:51.248592+0000","last_active":"2023-07-21T22:14:52.248598+0000","last_peered":"2023-07-21T22:14:52.248598+0000","last_clean":"2023-07-21T22:14:52.248598+0000","last_became_active":"2023-07-21T22:14:51.247932+0000","last_became_peered":"2023-07-21T22:14:51.247932+0000","last_unstale":"2023-07-21T22:14:52.248598+0000","last_undegraded":"2023-07-21T22:14:52.248598+0000","last_fullsized":"2023-07-21T22:14:52.248598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:01.815385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001359+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:14:57.001359+0000","last_peered":"2023-07-21T22:14:57.001359+0000","last_clean":"2023-07-21T22:14:57.001359+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:14:57.001359+0000","last_undegraded":"2023-07-21T22:14:57.001359+0000","last_fullsized":"2023-07-21T22:14:57.001359+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:40.285840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322887+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:14:52.322887+0000","last_peered":"2023-07-21T22:14:52.322887+0000","last_clean":"2023-07-21T22:14:52.322887+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:14:52.322887+0000","last_undegraded":"2023-07-21T22:14:52.322887+0000","last_fullsized":"2023-07-21T22:14:52.322887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:55:36.899847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.050257+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:15:32.050257+0000","last_peered":"2023-07-21T22:15:32.050257+0000","last_clean":"2023-07-21T22:15:32.050257+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:15:32.050257+0000","last_undegraded":"2023-07-21T22:15:32.050257+0000","last_fullsized":"2023-07-21T22:15:32.050257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:14.250309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249338+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:14:52.249338+0000","last_peered":"2023-07-21T22:14:52.249338+0000","last_clean":"2023-07-21T22:14:52.249338+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:14:52.249338+0000","last_undegraded":"2023-07-21T22:14:52.249338+0000","last_fullsized":"2023-07-21T22:14:52.249338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:06.815314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006887+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:14:57.006887+0000","last_peered":"2023-07-21T22:14:57.006887+0000","last_clean":"2023-07-21T22:14:57.006887+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:14:57.006887+0000","last_undegraded":"2023-07-21T22:14:57.006887+0000","last_fullsized":"2023-07-21T22:14:57.006887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:38:48.158170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324814+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:14:52.324814+0000","last_peered":"2023-07-21T22:14:52.324814+0000","last_clean":"2023-07-21T22:14:52.324814+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:14:52.324814+0000","last_undegraded":"2023-07-21T22:14:52.324814+0000","last_fullsized":"2023-07-21T22:14:52.324814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:53:48.638340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.284307+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:14:56.284307+0000","last_peered":"2023-07-21T22:14:56.284307+0000","last_clean":"2023-07-21T22:14:56.284307+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:14:56.284307+0000","last_undegraded":"2023-07-21T22:14:56.284307+0000","last_fullsized":"2023-07-21T22:14:56.284307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:15.490891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249055+0000","last_change":"2023-07-21T22:14:51.249768+0000","last_active":"2023-07-21T22:14:52.249055+0000","last_peered":"2023-07-21T22:14:52.249055+0000","last_clean":"2023-07-21T22:14:52.249055+0000","last_became_active":"2023-07-21T22:14:51.249503+0000","last_became_peered":"2023-07-21T22:14:51.249503+0000","last_unstale":"2023-07-21T22:14:52.249055+0000","last_undegraded":"2023-07-21T22:14:52.249055+0000","last_fullsized":"2023-07-21T22:14:52.249055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:39:15.294044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":125,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:32.050257+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:15:32.050257+0000","last_peered":"2023-07-21T22:15:32.050257+0000","last_clean":"2023-07-21T22:15:32.050257+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:15:32.050257+0000","last_undegraded":"2023-07-21T22:15:32.050257+0000","last_fullsized":"2023-07-21T22:15:32.050257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:38.675168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57498,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":97,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.551147+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:14:53.551147+0000","last_peered":"2023-07-21T22:14:53.551147+0000","last_clean":"2023-07-21T22:14:53.551147+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:14:53.551147+0000","last_undegraded":"2023-07-21T22:14:53.551147+0000","last_fullsized":"2023-07-21T22:14:53.551147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:47:51.431054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006578+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:14:57.006578+0000","last_peered":"2023-07-21T22:14:57.006578+0000","last_clean":"2023-07-21T22:14:57.006578+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:14:57.006578+0000","last_undegraded":"2023-07-21T22:14:57.006578+0000","last_fullsized":"2023-07-21T22:14:57.006578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:22:55.930555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323112+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:14:52.323112+0000","last_peered":"2023-07-21T22:14:52.323112+0000","last_clean":"2023-07-21T22:14:52.323112+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:14:52.323112+0000","last_undegraded":"2023-07-21T22:14:52.323112+0000","last_fullsized":"2023-07-21T22:14:52.323112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:57:57.488142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.006376+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:14:57.006376+0000","last_peered":"2023-07-21T22:14:57.006376+0000","last_clean":"2023-07-21T22:14:57.006376+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:14:57.006376+0000","last_undegraded":"2023-07-21T22:14:57.006376+0000","last_fullsized":"2023-07-21T22:14:57.006376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:28.876044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324774+0000","last_change":"2023-07-21T22:14:51.258940+0000","last_active":"2023-07-21T22:14:52.324774+0000","last_peered":"2023-07-21T22:14:52.324774+0000","last_clean":"2023-07-21T22:14:52.324774+0000","last_became_active":"2023-07-21T22:14:51.258595+0000","last_became_peered":"2023-07-21T22:14:51.258595+0000","last_unstale":"2023-07-21T22:14:52.324774+0000","last_undegraded":"2023-07-21T22:14:52.324774+0000","last_fullsized":"2023-07-21T22:14:52.324774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:39.297921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:35.054846+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:15:35.054846+0000","last_peered":"2023-07-21T22:15:35.054846+0000","last_clean":"2023-07-21T22:15:35.054846+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:15:35.054846+0000","last_undegraded":"2023-07-21T22:15:35.054846+0000","last_fullsized":"2023-07-21T22:15:35.054846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:39.148790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323808+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:14:52.323808+0000","last_peered":"2023-07-21T22:14:52.323808+0000","last_clean":"2023-07-21T22:14:52.323808+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:14:52.323808+0000","last_undegraded":"2023-07-21T22:14:52.323808+0000","last_fullsized":"2023-07-21T22:14:52.323808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:38.761579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.997294+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:14:56.997294+0000","last_peered":"2023-07-21T22:14:56.997294+0000","last_clean":"2023-07-21T22:14:56.997294+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:14:56.997294+0000","last_undegraded":"2023-07-21T22:14:56.997294+0000","last_fullsized":"2023-07-21T22:14:56.997294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:01:49.310753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323584+0000","last_change":"2023-07-21T22:14:51.252408+0000","last_active":"2023-07-21T22:14:52.323584+0000","last_peered":"2023-07-21T22:14:52.323584+0000","last_clean":"2023-07-21T22:14:52.323584+0000","last_became_active":"2023-07-21T22:14:51.252074+0000","last_became_peered":"2023-07-21T22:14:51.252074+0000","last_unstale":"2023-07-21T22:14:52.323584+0000","last_undegraded":"2023-07-21T22:14:52.323584+0000","last_fullsized":"2023-07-21T22:14:52.323584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.877759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.997626+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:14:56.997626+0000","last_peered":"2023-07-21T22:14:56.997626+0000","last_clean":"2023-07-21T22:14:56.997626+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:14:56.997626+0000","last_undegraded":"2023-07-21T22:14:56.997626+0000","last_fullsized":"2023-07-21T22:14:56.997626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:36.286445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249168+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:14:52.249168+0000","last_peered":"2023-07-21T22:14:52.249168+0000","last_clean":"2023-07-21T22:14:52.249168+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:14:52.249168+0000","last_undegraded":"2023-07-21T22:14:52.249168+0000","last_fullsized":"2023-07-21T22:14:52.249168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:12.075583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.101685+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:14:53.101685+0000","last_peered":"2023-07-21T22:14:53.101685+0000","last_clean":"2023-07-21T22:14:53.101685+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:14:53.101685+0000","last_undegraded":"2023-07-21T22:14:53.101685+0000","last_fullsized":"2023-07-21T22:14:53.101685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:03.657128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.530952+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:14:52.530952+0000","last_peered":"2023-07-21T22:14:52.530952+0000","last_clean":"2023-07-21T22:14:52.530952+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:14:52.530952+0000","last_undegraded":"2023-07-21T22:14:52.530952+0000","last_fullsized":"2023-07-21T22:14:52.530952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:11:20.224848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:33.067987+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:15:33.067987+0000","last_peered":"2023-07-21T22:15:33.067987+0000","last_clean":"2023-07-21T22:15:33.067987+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:15:33.067987+0000","last_undegraded":"2023-07-21T22:15:33.067987+0000","last_fullsized":"2023-07-21T22:15:33.067987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:37.949470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324083+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:14:52.324083+0000","last_peered":"2023-07-21T22:14:52.324083+0000","last_clean":"2023-07-21T22:14:52.324083+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:14:52.324083+0000","last_undegraded":"2023-07-21T22:14:52.324083+0000","last_fullsized":"2023-07-21T22:14:52.324083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:57:09.279672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323831+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:14:52.323831+0000","last_peered":"2023-07-21T22:14:52.323831+0000","last_clean":"2023-07-21T22:14:52.323831+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:14:52.323831+0000","last_undegraded":"2023-07-21T22:14:52.323831+0000","last_fullsized":"2023-07-21T22:14:52.323831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:31:31.048621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:33.051320+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:15:33.051320+0000","last_peered":"2023-07-21T22:15:33.051320+0000","last_clean":"2023-07-21T22:15:33.051320+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:15:33.051320+0000","last_undegraded":"2023-07-21T22:15:33.051320+0000","last_fullsized":"2023-07-21T22:15:33.051320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:39:31.032230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248803+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:14:52.248803+0000","last_peered":"2023-07-21T22:14:52.248803+0000","last_clean":"2023-07-21T22:14:52.248803+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:14:52.248803+0000","last_undegraded":"2023-07-21T22:14:52.248803+0000","last_fullsized":"2023-07-21T22:14:52.248803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:06.145398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002580+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:14:57.002580+0000","last_peered":"2023-07-21T22:14:57.002580+0000","last_clean":"2023-07-21T22:14:57.002580+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:14:57.002580+0000","last_undegraded":"2023-07-21T22:14:57.002580+0000","last_fullsized":"2023-07-21T22:14:57.002580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:48.946604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323556+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:14:52.323556+0000","last_peered":"2023-07-21T22:14:52.323556+0000","last_clean":"2023-07-21T22:14:52.323556+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:14:52.323556+0000","last_undegraded":"2023-07-21T22:14:52.323556+0000","last_fullsized":"2023-07-21T22:14:52.323556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:04.146166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323206+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:14:52.323206+0000","last_peered":"2023-07-21T22:14:52.323206+0000","last_clean":"2023-07-21T22:14:52.323206+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:14:52.323206+0000","last_undegraded":"2023-07-21T22:14:52.323206+0000","last_fullsized":"2023-07-21T22:14:52.323206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:14:26.470268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:55.279353+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:14:55.279353+0000","last_peered":"2023-07-21T22:14:55.279353+0000","last_clean":"2023-07-21T22:14:55.279353+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:14:55.279353+0000","last_undegraded":"2023-07-21T22:14:55.279353+0000","last_fullsized":"2023-07-21T22:14:55.279353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:41.327520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996715+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:14:56.996715+0000","last_peered":"2023-07-21T22:14:56.996715+0000","last_clean":"2023-07-21T22:14:56.996715+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:14:56.996715+0000","last_undegraded":"2023-07-21T22:14:56.996715+0000","last_fullsized":"2023-07-21T22:14:56.996715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:53:00.386436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323125+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:14:52.323125+0000","last_peered":"2023-07-21T22:14:52.323125+0000","last_clean":"2023-07-21T22:14:52.323125+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:14:52.323125+0000","last_undegraded":"2023-07-21T22:14:52.323125+0000","last_fullsized":"2023-07-21T22:14:52.323125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:25.036430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:03.550435+0000","last_change":"2023-07-21T22:14:50.231962+0000","last_active":"2023-07-21T22:15:03.550435+0000","last_peered":"2023-07-21T22:15:03.550435+0000","last_clean":"2023-07-21T22:15:03.550435+0000","last_became_active":"2023-07-21T22:14:50.231638+0000","last_became_peered":"2023-07-21T22:14:50.231638+0000","last_unstale":"2023-07-21T22:15:03.550435+0000","last_undegraded":"2023-07-21T22:15:03.550435+0000","last_fullsized":"2023-07-21T22:15:03.550435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:18.134292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249093+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:14:52.249093+0000","last_peered":"2023-07-21T22:14:52.249093+0000","last_clean":"2023-07-21T22:14:52.249093+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:14:52.249093+0000","last_undegraded":"2023-07-21T22:14:52.249093+0000","last_fullsized":"2023-07-21T22:14:52.249093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:15.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.002623+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:14:57.002623+0000","last_peered":"2023-07-21T22:14:57.002623+0000","last_clean":"2023-07-21T22:14:57.002623+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:14:57.002623+0000","last_undegraded":"2023-07-21T22:14:57.002623+0000","last_fullsized":"2023-07-21T22:14:57.002623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:43.076462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324274+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:14:52.324274+0000","last_peered":"2023-07-21T22:14:52.324274+0000","last_clean":"2023-07-21T22:14:52.324274+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:14:52.324274+0000","last_undegraded":"2023-07-21T22:14:52.324274+0000","last_fullsized":"2023-07-21T22:14:52.324274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:16.004933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270269+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:14:54.270269+0000","last_peered":"2023-07-21T22:14:54.270269+0000","last_clean":"2023-07-21T22:14:54.270269+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:14:54.270269+0000","last_undegraded":"2023-07-21T22:14:54.270269+0000","last_fullsized":"2023-07-21T22:14:54.270269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:06:47.274452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:53.550789+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:14:53.550789+0000","last_peered":"2023-07-21T22:14:53.550789+0000","last_clean":"2023-07-21T22:14:53.550789+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:14:53.550789+0000","last_undegraded":"2023-07-21T22:14:53.550789+0000","last_fullsized":"2023-07-21T22:14:53.550789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:02:40.632894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.005576+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:14:57.005576+0000","last_peered":"2023-07-21T22:14:57.005576+0000","last_clean":"2023-07-21T22:14:57.005576+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:14:57.005576+0000","last_undegraded":"2023-07-21T22:14:57.005576+0000","last_fullsized":"2023-07-21T22:14:57.005576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:31.455547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248754+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:14:52.248754+0000","last_peered":"2023-07-21T22:14:52.248754+0000","last_clean":"2023-07-21T22:14:52.248754+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:14:52.248754+0000","last_undegraded":"2023-07-21T22:14:52.248754+0000","last_fullsized":"2023-07-21T22:14:52.248754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:25.484174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:59.922284+0000","last_change":"2023-07-21T22:14:50.229923+0000","last_active":"2023-07-21T22:14:59.922284+0000","last_peered":"2023-07-21T22:14:59.922284+0000","last_clean":"2023-07-21T22:14:59.922284+0000","last_became_active":"2023-07-21T22:14:50.229645+0000","last_became_peered":"2023-07-21T22:14:50.229645+0000","last_unstale":"2023-07-21T22:14:59.922284+0000","last_undegraded":"2023-07-21T22:14:59.922284+0000","last_fullsized":"2023-07-21T22:14:59.922284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:36:56.239739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248731+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:14:52.248731+0000","last_peered":"2023-07-21T22:14:52.248731+0000","last_clean":"2023-07-21T22:14:52.248731+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:14:52.248731+0000","last_undegraded":"2023-07-21T22:14:52.248731+0000","last_fullsized":"2023-07-21T22:14:52.248731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:20.211619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.270798+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:14:54.270798+0000","last_peered":"2023-07-21T22:14:54.270798+0000","last_clean":"2023-07-21T22:14:54.270798+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:14:54.270798+0000","last_undegraded":"2023-07-21T22:14:54.270798+0000","last_fullsized":"2023-07-21T22:14:54.270798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:01.500609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324713+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:14:52.324713+0000","last_peered":"2023-07-21T22:14:52.324713+0000","last_clean":"2023-07-21T22:14:52.324713+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:14:52.324713+0000","last_undegraded":"2023-07-21T22:14:52.324713+0000","last_fullsized":"2023-07-21T22:14:52.324713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:17.553733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:15:31.049519+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:15:31.049519+0000","last_peered":"2023-07-21T22:15:31.049519+0000","last_clean":"2023-07-21T22:15:31.049519+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:15:31.049519+0000","last_undegraded":"2023-07-21T22:15:31.049519+0000","last_fullsized":"2023-07-21T22:15:31.049519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:15.167223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324532+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:14:52.324532+0000","last_peered":"2023-07-21T22:14:52.324532+0000","last_clean":"2023-07-21T22:14:52.324532+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:14:52.324532+0000","last_undegraded":"2023-07-21T22:14:52.324532+0000","last_fullsized":"2023-07-21T22:14:52.324532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:47:33.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:56.996861+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:14:56.996861+0000","last_peered":"2023-07-21T22:14:56.996861+0000","last_clean":"2023-07-21T22:14:56.996861+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:14:56.996861+0000","last_undegraded":"2023-07-21T22:14:56.996861+0000","last_fullsized":"2023-07-21T22:14:56.996861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.897090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:54.993320+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:14:54.993320+0000","last_peered":"2023-07-21T22:14:54.993320+0000","last_clean":"2023-07-21T22:14:54.993320+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:14:54.993320+0000","last_undegraded":"2023-07-21T22:14:54.993320+0000","last_fullsized":"2023-07-21T22:14:54.993320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:52.794435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:57.001568+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:14:57.001568+0000","last_peered":"2023-07-21T22:14:57.001568+0000","last_clean":"2023-07-21T22:14:57.001568+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:14:57.001568+0000","last_undegraded":"2023-07-21T22:14:57.001568+0000","last_fullsized":"2023-07-21T22:14:57.001568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:56.943909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324655+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:14:52.324655+0000","last_peered":"2023-07-21T22:14:52.324655+0000","last_clean":"2023-07-21T22:14:52.324655+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:14:52.324655+0000","last_undegraded":"2023-07-21T22:14:52.324655+0000","last_fullsized":"2023-07-21T22:14:52.324655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:05:57.078423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248843+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:14:52.248843+0000","last_peered":"2023-07-21T22:14:52.248843+0000","last_clean":"2023-07-21T22:14:52.248843+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:14:52.248843+0000","last_undegraded":"2023-07-21T22:14:52.248843+0000","last_fullsized":"2023-07-21T22:14:52.248843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:49:10.848487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324305+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:14:52.324305+0000","last_peered":"2023-07-21T22:14:52.324305+0000","last_clean":"2023-07-21T22:14:52.324305+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:14:52.324305+0000","last_undegraded":"2023-07-21T22:14:52.324305+0000","last_fullsized":"2023-07-21T22:14:52.324305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:16:56.773704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323602+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:14:52.323602+0000","last_peered":"2023-07-21T22:14:52.323602+0000","last_clean":"2023-07-21T22:14:52.323602+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:14:52.323602+0000","last_undegraded":"2023-07-21T22:14:52.323602+0000","last_fullsized":"2023-07-21T22:14:52.323602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:58.733025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324433+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:14:52.324433+0000","last_peered":"2023-07-21T22:14:52.324433+0000","last_clean":"2023-07-21T22:14:52.324433+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:14:52.324433+0000","last_undegraded":"2023-07-21T22:14:52.324433+0000","last_fullsized":"2023-07-21T22:14:52.324433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:51.861062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.322982+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:14:52.322982+0000","last_peered":"2023-07-21T22:14:52.322982+0000","last_clean":"2023-07-21T22:14:52.322982+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:14:52.322982+0000","last_undegraded":"2023-07-21T22:14:52.322982+0000","last_fullsized":"2023-07-21T22:14:52.322982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:29.650653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248998+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:14:52.248998+0000","last_peered":"2023-07-21T22:14:52.248998+0000","last_clean":"2023-07-21T22:14:52.248998+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:14:52.248998+0000","last_undegraded":"2023-07-21T22:14:52.248998+0000","last_fullsized":"2023-07-21T22:14:52.248998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:30:39.513431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.324003+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:14:52.324003+0000","last_peered":"2023-07-21T22:14:52.324003+0000","last_clean":"2023-07-21T22:14:52.324003+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:14:52.324003+0000","last_undegraded":"2023-07-21T22:14:52.324003+0000","last_fullsized":"2023-07-21T22:14:52.324003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:23.704220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249061+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:14:52.249061+0000","last_peered":"2023-07-21T22:14:52.249061+0000","last_clean":"2023-07-21T22:14:52.249061+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:14:52.249061+0000","last_undegraded":"2023-07-21T22:14:52.249061+0000","last_fullsized":"2023-07-21T22:14:52.249061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:22.277878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323571+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:14:52.323571+0000","last_peered":"2023-07-21T22:14:52.323571+0000","last_clean":"2023-07-21T22:14:52.323571+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:14:52.323571+0000","last_undegraded":"2023-07-21T22:14:52.323571+0000","last_fullsized":"2023-07-21T22:14:52.323571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:35:26.714623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248660+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:14:52.248660+0000","last_peered":"2023-07-21T22:14:52.248660+0000","last_clean":"2023-07-21T22:14:52.248660+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:14:52.248660+0000","last_undegraded":"2023-07-21T22:14:52.248660+0000","last_fullsized":"2023-07-21T22:14:52.248660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:27:40.710691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.249020+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:14:52.249020+0000","last_peered":"2023-07-21T22:14:52.249020+0000","last_clean":"2023-07-21T22:14:52.249020+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:14:52.249020+0000","last_undegraded":"2023-07-21T22:14:52.249020+0000","last_fullsized":"2023-07-21T22:14:52.249020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:45.083477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.323134+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:14:52.323134+0000","last_peered":"2023-07-21T22:14:52.323134+0000","last_clean":"2023-07-21T22:14:52.323134+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:14:52.323134+0000","last_undegraded":"2023-07-21T22:14:52.323134+0000","last_fullsized":"2023-07-21T22:14:52.323134+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:19:31.888652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:14:52.248874+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:14:52.248874+0000","last_peered":"2023-07-21T22:14:52.248874+0000","last_clean":"2023-07-21T22:14:52.248874+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:14:52.248874+0000","last_undegraded":"2023-07-21T22:14:52.248874+0000","last_fullsized":"2023-07-21T22:14:52.248874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:28.191291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63728,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":410,"num_read_kb":273,"num_write":233,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79600000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":9,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.872}]},{"osd":10,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.82799999999999996}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1589248,"data_stored":1072491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.498},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74399999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.69399999999999995}]},{"osd":11,"last update":"Fri Jul 21 22:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.78200000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.308,"15min":0.308},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.879}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.746},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.85999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78300000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.84299999999999997}]}]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":423983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.9480000000000004,"5min":5.9480000000000004,"15min":5.9480000000000004},"last":0.621},{"interface":"front","average":{"1min":0.746,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.2400000000000002,"5min":4.2400000000000002,"15min":4.2400000000000002},"last":0.77900000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1820000000000004,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.505}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.69299999999999995,"15min":0.63400000000000001},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.4550000000000001,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.86799999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.74399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.92000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1015972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.437,"15min":0.42499999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32400000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.42499999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86799999999999999,"5min":1.024,"15min":1.024},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84299999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.434}]},{"osd":5,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79800000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.64900000000000002},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":1.036,"15min":1.036},"last":0.628}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.56100000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95199999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93200000000000005,"5min":1.173,"15min":1.173},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94699999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.64600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.49399999999999999,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":13.989000000000001,"5min":13.989000000000001,"15min":13.989000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.28399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57099999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.308}]},{"osd":1,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.70799999999999996,"15min":0.622},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.68600000000000005,"15min":0.59399999999999997},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":13.789999999999999,"5min":13.789999999999999,"15min":13.789999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.56899999999999995,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":14.023,"5min":14.023,"15min":14.023},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.64500000000000002,"15min":0.59999999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":7.1189999999999998,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.75800000000000001,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":13.956,"5min":13.956,"15min":13.956},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.65600000000000003,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:14:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64900000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78200000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.77900000000000003,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":7.2489999999999997,"5min":7.2489999999999997,"15min":7.2489999999999997},"last":0.52200000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.434}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":482016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73199999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52800000000000002,"15min":0.51700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73899999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.61899999999999999,"15min":0.57099999999999995},"min":{"1min":0.52300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.40100000000000002,"15min":0.371},"min":{"1min":0.33300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42699999999999999,"15min":0.39700000000000002},"min":{"1min":0.313,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Jul 21 22:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56999999999999995,"15min":0.53200000000000003},"min":{"1min":0.497,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82899999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.48699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.438},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.38500000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71799999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90800000000000003,"5min":0.995,"15min":0.995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77800000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.75700000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":424944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.624},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.50600000000000001},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38100000000000001},"min":{"1min":0.26300000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.443,"15min":0.436},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.39300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:15:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72099999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.441,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78200000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59199999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65000000000000002,"5min":0.877,"15min":0.877},"last":0.45100000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74099999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64900000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.436},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.747,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57599999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70799999999999996}]},{"osd":11,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":482188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51900000000000002,"15min":0.502},"min":{"1min":0.36399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.497,"15min":0.48699999999999999},"min":{"1min":0.35899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.53900000000000003,"15min":0.51000000000000001},"min":{"1min":0.39700000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.4489999999999998,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.58599999999999997,"15min":0.53500000000000003},"min":{"1min":0.46500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.754},{"interface":"front","average":{"1min":0.88,"5min":0.63600000000000001,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.7210000000000001,"5min":4.7210000000000001,"15min":4.7210000000000001},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.46400000000000002,"15min":0.438},"min":{"1min":0.33100000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.495},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.45800000000000002,"15min":0.44500000000000001},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.443}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.35299999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":423159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35099999999999998,"15min":0.32300000000000001},"min":{"1min":0.255,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.37},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.378,"15min":0.33600000000000002},"min":{"1min":0.28000000000000003,"5min":0.19,"15min":0.19},"max":{"1min":1.232,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":1.512,"15min":1.512},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.034,"5min":0.63300000000000001,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.545,"5min":10.545,"15min":10.545},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.434,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.92000000000000004,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.625},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.67000000000000004,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":10.446999999999999,"5min":10.446999999999999,"15min":10.446999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":1.839,"15min":1.839},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0409999999999999,"5min":1.653,"15min":1.653},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71799999999999997,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79400000000000004,"5min":0.998,"15min":0.998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.68100000000000005,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.869999999999999,"5min":10.869999999999999,"15min":10.869999999999999},"last":0.45200000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.38800000000000001,"15min":0.378},"min":{"1min":0.32700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28199999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.57799999999999996,"15min":0.57499999999999996},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.57599999999999996,"15min":0.55200000000000005},"min":{"1min":0.51400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94199999999999995,"5min":1.22,"15min":1.22},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:15:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.49199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48399999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.55500000000000005,"5min":0.372,"15min":0.372},"max":{"1min":0.95099999999999996,"5min":1.105,"15min":1.105},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56100000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.5,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0169999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":1.286,"15min":1.286},"last":0.85699999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:14:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1523712,"data_stored":1016077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83199999999999996,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.5,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.372}]},{"osd":3,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.625}]},{"osd":7,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.73399999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.505},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60199999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.59299999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88800000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.70899999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:15:38.595 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-21T22:15:38.595 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-21T22:15:38.596 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-21T22:15:38.596 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-21T22:15:38.597 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:38 smithi194 ceph-mon[98288]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:38.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:38 smithi134 ceph-mon[97873]: pgmap v275: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:15:39.145 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:39.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:39 smithi194 ceph-mon[98288]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:39.600 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:39 smithi093 ceph-mon[96579]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:39.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:39 smithi134 ceph-mon[97873]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:15:40.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 -- 172.21.15.93:0/3313194433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc100790 msgr2=0x7f12fc104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:40.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 --2- 172.21.15.93:0/3313194433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc100790 0x7f12fc104760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0099c0 tx=0x7f12ec009cd0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 -- 172.21.15.93:0/3313194433 shutdown_connections 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 --2- 172.21.15.93:0/3313194433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc104e90 0x7f12fc10ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 --2- 172.21.15.93:0/3313194433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc0ffdf0 0x7f12fc1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 --2- 172.21.15.93:0/3313194433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc100790 0x7f12fc104760 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 -- 172.21.15.93:0/3313194433 >> 172.21.15.93:0/3313194433 conn(0x7f12fc0fb670 msgr2=0x7f12fc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 -- 172.21.15.93:0/3313194433 shutdown_connections 2023-07-21T22:15:40.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 -- 172.21.15.93:0/3313194433 wait complete. 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.300+0000 7f130212b700 1 Processor -- start 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 -- start start 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc0ffdf0 0x7f12fc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc100790 0x7f12fc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1a4140 con 0x7f12fc104e90 2023-07-21T22:15:40.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1a42b0 con 0x7f12fc100790 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f130212b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1a4470 con 0x7f12fc0ffdf0 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42784/0 (socket says 172.21.15.93:42784) 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 -- 172.21.15.93:0/1902310465 learned_addr learned my addr 172.21.15.93:0/1902310465 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fb7fe700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc0ffdf0 0x7f12fc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12faffd700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc100790 0x7f12fc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 -- 172.21.15.93:0/1902310465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc0ffdf0 msgr2=0x7f12fc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:40.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc0ffdf0 0x7f12fc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 -- 172.21.15.93:0/1902310465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc100790 msgr2=0x7f12fc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc100790 0x7f12fc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 -- 172.21.15.93:0/1902310465 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0095f0 con 0x7f12fc104e90 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.301+0000 7f12fbfff700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b970 tx=0x7f12f400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.302+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400c930 con 0x7f12fc104e90 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.302+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc1a4750 con 0x7f12fc104e90 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.302+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f4014440 con 0x7f12fc104e90 2023-07-21T22:15:40.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.302+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40075d0 con 0x7f12fc104e90 2023-07-21T22:15:40.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.302+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc1a4bd0 con 0x7f12fc104e90 2023-07-21T22:15:40.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.303+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc04e210 con 0x7f12fc104e90 2023-07-21T22:15:40.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.304+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f12f40145b0 con 0x7f12fc104e90 2023-07-21T22:15:40.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.304+0000 7f12f8ff9700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 0x7f12e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:40.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.304+0000 7f12fb7fe700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 0x7f12e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:40.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.304+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f12f4006e40 con 0x7f12fc104e90 2023-07-21T22:15:40.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.304+0000 7f12fb7fe700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 0x7f12e40a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:40.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.308+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f12f404e770 con 0x7f12fc104e90 2023-07-21T22:15:40.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:40 smithi093 ceph-mon[96579]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:40.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.535+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12fc073d40 con 0x7f12fc104e90 2023-07-21T22:15:40.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.536+0000 7f12f8ff9700 1 -- 172.21.15.93:0/1902310465 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f12fc073d40 con 0x7f12fc104e90 2023-07-21T22:15:40.580 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:40.580 INFO:teuthology.orchestra.run.smithi093.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-21T22:15:40.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 msgr2=0x7f12e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:40.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 0x7f12e40a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005950 tx=0x7f12f000a300 comp rx=0 tx=0).stop 2023-07-21T22:15:40.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 msgr2=0x7f12fc0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b970 tx=0x7f12f400bc80 comp rx=0 tx=0).stop 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 shutdown_connections 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12fc104e90 0x7f12fc0732d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12fc0ffdf0 0x7f12fc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f12e40a11e0 0x7f12e40a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 --2- 172.21.15.93:0/1902310465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12fc100790 0x7f12fc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 >> 172.21.15.93:0/1902310465 conn(0x7f12fc0fb670 msgr2=0x7f12fc0fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:40.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 shutdown_connections 2023-07-21T22:15:40.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:40.538+0000 7f130212b700 1 -- 172.21.15.93:0/1902310465 wait complete. 2023-07-21T22:15:40.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:40 smithi194 ceph-mon[98288]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:40.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:40 smithi134 ceph-mon[97873]: pgmap v276: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:15:40.997 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-21T22:15:40.998 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:15:41.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 -- 172.21.15.93:0/342340012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880ffbe0 msgr2=0x7fdd88100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/342340012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880ffbe0 0x7fdd88100030 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009990 tx=0x7fdd78009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 -- 172.21.15.93:0/342340012 shutdown_connections 2023-07-21T22:15:41.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/342340012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd88100570 0x7fdd88106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/342340012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd880fe9f0 0x7fdd880fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/342340012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880ffbe0 0x7fdd88100030 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 -- 172.21.15.93:0/342340012 >> 172.21.15.93:0/342340012 conn(0x7fdd880f9ff0 msgr2=0x7fdd880fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 -- 172.21.15.93:0/342340012 shutdown_connections 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.080+0000 7fdd8d89b700 1 -- 172.21.15.93:0/342340012 wait complete. 2023-07-21T22:15:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 Processor -- start 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 -- start start 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd880ffbe0 0x7fdd88195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 0x7fdd8819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8819d340 con 0x7fdd880fe9f0 2023-07-21T22:15:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8819d480 con 0x7fdd88100570 2023-07-21T22:15:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd8d89b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8819d610 con 0x7fdd880ffbe0 2023-07-21T22:15:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42790/0 (socket says 172.21.15.93:42790) 2023-07-21T22:15:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd86ffd700 1 -- 172.21.15.93:0/3245832035 learned_addr learned my addr 172.21.15.93:0/3245832035 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd867fc700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd880ffbe0 0x7fdd88195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.081+0000 7fdd877fe700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 0x7fdd8819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 -- 172.21.15.93:0/3245832035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd880ffbe0 msgr2=0x7fdd88195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd880ffbe0 0x7fdd88195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 -- 172.21.15.93:0/3245832035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 msgr2=0x7fdd8819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 0x7fdd8819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 -- 172.21.15.93:0/3245832035 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780095f0 con 0x7fdd880fe9f0 2023-07-21T22:15:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd877fe700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 0x7fdd8819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd86ffd700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00d980 tx=0x7fdd7c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00bb60 con 0x7fdd880fe9f0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd7c007620 con 0x7fdd880fe9f0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8819d8f0 con 0x7fdd880fe9f0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00f620 con 0x7fdd880fe9f0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.082+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8819de00 con 0x7fdd880fe9f0 2023-07-21T22:15:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.083+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8810a1c0 con 0x7fdd880fe9f0 2023-07-21T22:15:41.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.084+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdd7c010460 con 0x7fdd880fe9f0 2023-07-21T22:15:41.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.084+0000 7fdd8c899700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 0x7fdd700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.085+0000 7fdd867fc700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 0x7fdd700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.085+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdd7c0cd250 con 0x7fdd880fe9f0 2023-07-21T22:15:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.085+0000 7fdd867fc700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 0x7fdd700a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd78006010 tx=0x7fdd7800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.088+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdd7c0968f0 con 0x7fdd880fe9f0 2023-07-21T22:15:41.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.281+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd8819e3e0 con 0x7fdd880fe9f0 2023-07-21T22:15:41.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.282+0000 7fdd8c899700 1 -- 172.21.15.93:0/3245832035 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7fdd7c00de60 con 0x7fdd880fe9f0 2023-07-21T22:15:41.326 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:41.327 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:15:41.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.284+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 msgr2=0x7fdd700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.284+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 0x7fdd700a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd78006010 tx=0x7fdd7800b540 comp rx=0 tx=0).stop 2023-07-21T22:15:41.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.284+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 msgr2=0x7fdd881957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.284+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00d980 tx=0x7fdd7c00dd40 comp rx=0 tx=0).stop 2023-07-21T22:15:41.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 shutdown_connections 2023-07-21T22:15:41.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fdd700a12a0 0x7fdd700a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88100570 0x7fdd8819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd880fe9f0 0x7fdd881957c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 --2- 172.21.15.93:0/3245832035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd880ffbe0 0x7fdd88195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 >> 172.21.15.93:0/3245832035 conn(0x7fdd880f9ff0 msgr2=0x7fdd88068990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 shutdown_connections 2023-07-21T22:15:41.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.285+0000 7fdd8d89b700 1 -- 172.21.15.93:0/3245832035 wait complete. 2023-07-21T22:15:41.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:15:41.342 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:15:41.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 -- 172.21.15.93:0/3446656508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0668100cd0 msgr2=0x7f0668101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 --2- 172.21.15.93:0/3446656508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0668100cd0 0x7f0668101120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0658009990 tx=0x7f0658009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 -- 172.21.15.93:0/3446656508 shutdown_connections 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 --2- 172.21.15.93:0/3446656508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668101660 0x7f0668107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 --2- 172.21.15.93:0/3446656508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06680ffae0 0x7f06680ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 --2- 172.21.15.93:0/3446656508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0668100cd0 0x7f0668101120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 -- 172.21.15.93:0/3446656508 >> 172.21.15.93:0/3446656508 conn(0x7f06680fb0e0 msgr2=0x7f06680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 -- 172.21.15.93:0/3446656508 shutdown_connections 2023-07-21T22:15:41.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.422+0000 7f066cf64700 1 -- 172.21.15.93:0/3446656508 wait complete. 2023-07-21T22:15:41.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 Processor -- start 2023-07-21T22:15:41.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 -- start start 2023-07-21T22:15:41.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 0x7f0668195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 0x7f066819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066819d460 con 0x7f06680ffae0 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066819d5a0 con 0x7f0668101660 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066819d730 con 0x7f0668100cd0 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42804/0 (socket says 172.21.15.93:42804) 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 -- 172.21.15.93:0/4200761814 learned_addr learned my addr 172.21.15.93:0/4200761814 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:41.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f0665d9b700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 0x7f0668195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 -- 172.21.15.93:0/4200761814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 msgr2=0x7f0668195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 0x7f0668195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f0666d9d700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 0x7f066819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 -- 172.21.15.93:0/4200761814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 msgr2=0x7f066819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 0x7f066819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 -- 172.21.15.93:0/4200761814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580095f0 con 0x7f06680ffae0 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f0665d9b700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 0x7f0668195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:41.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f0666d9d700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 0x7f066819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:41.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066659c700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c970 tx=0x7f065c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c004220 con 0x7f06680ffae0 2023-07-21T22:15:41.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066819da10 con 0x7f06680ffae0 2023-07-21T22:15:41.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065c007d60 con 0x7f06680ffae0 2023-07-21T22:15:41.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.423+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c003d00 con 0x7f06680ffae0 2023-07-21T22:15:41.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.424+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066819df20 con 0x7f06680ffae0 2023-07-21T22:15:41.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.424+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0668107080 con 0x7f06680ffae0 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.426+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f065c004380 con 0x7f06680ffae0 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.427+0000 7f06577fe700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 0x7f06500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.427+0000 7f0665d9b700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 0x7f06500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.427+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f065c0cc8d0 con 0x7f06680ffae0 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.427+0000 7f0665d9b700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 0x7f06500a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.430+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f065c096070 con 0x7f06680ffae0 2023-07-21T22:15:41.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:41 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1902310465' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:15:41.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:41 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1902310465' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:15:41.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.616+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f066819dba0 con 0x7f06680ffae0 2023-07-21T22:15:41.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.617+0000 7f06577fe700 1 -- 172.21.15.93:0/4200761814 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f066819dba0 con 0x7f06680ffae0 2023-07-21T22:15:41.660 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:15:41.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 msgr2=0x7f06500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 0x7f06500a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 msgr2=0x7f06681958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c970 tx=0x7f065c00cd30 comp rx=0 tx=0).stop 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 shutdown_connections 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f06500a11e0 0x7f06500a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668101660 0x7f066819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f06680ffae0 0x7f06681958e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 --2- 172.21.15.93:0/4200761814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0668100cd0 0x7f0668195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 >> 172.21.15.93:0/4200761814 conn(0x7f06680fb0e0 msgr2=0x7f06681041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 shutdown_connections 2023-07-21T22:15:41.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.619+0000 7f066cf64700 1 -- 172.21.15.93:0/4200761814 wait complete. 2023-07-21T22:15:41.673 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:41 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1902310465' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:15:41.674 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:15:41.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 -- 172.21.15.93:0/2759033690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 msgr2=0x7f6d50073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2759033690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50073360 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009990 tx=0x7f6d38009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 -- 172.21.15.93:0/2759033690 shutdown_connections 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2759033690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 0x7f6d50109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2759033690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2759033690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50073360 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 -- 172.21.15.93:0/2759033690 >> 172.21.15.93:0/2759033690 conn(0x7f6d500fb160 msgr2=0x7f6d500fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 -- 172.21.15.93:0/2759033690 shutdown_connections 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.754+0000 7f6d566f2700 1 -- 172.21.15.93:0/2759033690 wait complete. 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 Processor -- start 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 -- start start 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 0x7f6d5019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019f580 con 0x7f6d50072ef0 2023-07-21T22:15:41.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019f6c0 con 0x7f6d501079f0 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d566f2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019f850 con 0x7f6d500744d0 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42818/0 (socket says 172.21.15.93:42818) 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 -- 172.21.15.93:0/2126361372 learned_addr learned my addr 172.21.15.93:0/2126361372 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d54c8f700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 0x7f6d5019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d47fff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 -- 172.21.15.93:0/2126361372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 msgr2=0x7f6d50197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 -- 172.21.15.93:0/2126361372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 msgr2=0x7f6d5019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 0x7f6d5019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 -- 172.21.15.93:0/2126361372 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380095f0 con 0x7f6d50072ef0 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d47fff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.755+0000 7f6d4ffff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000e9a0 tx=0x7f6d4000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.756+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40009790 con 0x7f6d50072ef0 2023-07-21T22:15:41.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.756+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d4000cbd0 con 0x7f6d50072ef0 2023-07-21T22:15:41.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.756+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40018680 con 0x7f6d50072ef0 2023-07-21T22:15:41.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.756+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5019fb30 con 0x7f6d50072ef0 2023-07-21T22:15:41.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.756+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5019fff0 con 0x7f6d50072ef0 2023-07-21T22:15:41.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.757+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50106130 con 0x7f6d50072ef0 2023-07-21T22:15:41.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.758+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6d4000cd40 con 0x7f6d50072ef0 2023-07-21T22:15:41.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.758+0000 7f6d4dffb700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 0x7f6d3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.758+0000 7f6d47fff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 0x7f6d3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.758+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d40014070 con 0x7f6d50072ef0 2023-07-21T22:15:41.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.759+0000 7f6d47fff700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 0x7f6d3c0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b2e0 tx=0x7f6d38005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.762+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6d40096880 con 0x7f6d50072ef0 2023-07-21T22:15:41.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.949+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d5002cce0 con 0x7f6d50072ef0 2023-07-21T22:15:41.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.949+0000 7f6d4dffb700 1 -- 172.21.15.93:0/2126361372 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d40099ea0 con 0x7f6d50072ef0 2023-07-21T22:15:41.993 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:15:41.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 msgr2=0x7f6d3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 0x7f6d3c0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b2e0 tx=0x7f6d38005e80 comp rx=0 tx=0).stop 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 msgr2=0x7f6d50197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000e9a0 tx=0x7f6d4000ed60 comp rx=0 tx=0).stop 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 shutdown_connections 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6d3c0a11e0 0x7f6d3c0a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d501079f0 0x7f6d5019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d50072ef0 0x7f6d50197a00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 --2- 172.21.15.93:0/2126361372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d500744d0 0x7f6d50197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 >> 172.21.15.93:0/2126361372 conn(0x7f6d500fb160 msgr2=0x7f6d500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 shutdown_connections 2023-07-21T22:15:41.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:41.951+0000 7f6d566f2700 1 -- 172.21.15.93:0/2126361372 wait complete. 2023-07-21T22:15:42.007 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:15:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 -- 172.21.15.93:0/3548398556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 msgr2=0x7f7004106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 --2- 172.21.15.93:0/3548398556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f7004106e20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b0a0 tx=0x7f6ffc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 -- 172.21.15.93:0/3548398556 shutdown_connections 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 --2- 172.21.15.93:0/3548398556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f7004106e20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 --2- 172.21.15.93:0/3548398556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70040fe9e0 0x7f70040fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 --2- 172.21.15.93:0/3548398556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040ffbd0 0x7f7004100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 -- 172.21.15.93:0/3548398556 >> 172.21.15.93:0/3548398556 conn(0x7f70040fa000 msgr2=0x7f70040fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 -- 172.21.15.93:0/3548398556 shutdown_connections 2023-07-21T22:15:42.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.087+0000 7f700be1e700 1 -- 172.21.15.93:0/3548398556 wait complete. 2023-07-21T22:15:42.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 Processor -- start 2023-07-21T22:15:42.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 -- start start 2023-07-21T22:15:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040fe9e0 0x7f7004102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70040ffbd0 0x7f7004100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004101c90 con 0x7f7004100560 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004101e00 con 0x7f70040fe9e0 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700be1e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700419f9d0 con 0x7f70040ffbd0 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700a3bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700a3bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42836/0 (socket says 172.21.15.93:42836) 2023-07-21T22:15:42.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f700a3bb700 1 -- 172.21.15.93:0/4018182660 learned_addr learned my addr 172.21.15.93:0/4018182660 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.088+0000 7f7009bba700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040fe9e0 0x7f7004102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 -- 172.21.15.93:0/4018182660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70040ffbd0 msgr2=0x7f7004100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70040ffbd0 0x7f7004100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 -- 172.21.15.93:0/4018182660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040fe9e0 msgr2=0x7f7004102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040fe9e0 0x7f7004102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 -- 172.21.15.93:0/4018182660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009d00 con 0x7f7004100560 2023-07-21T22:15:42.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700a3bb700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d010 tx=0x7f6ffc004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00e040 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700419fc50 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ffc009440 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004280 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.089+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70041a00d0 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.090+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700410a210 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.091+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6ffc01d720 con 0x7f7004100560 2023-07-21T22:15:42.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.091+0000 7f6ffaffd700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 0x7f6ff00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.091+0000 7f7009bba700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 0x7f6ff00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.091+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ffc01de40 con 0x7f7004100560 2023-07-21T22:15:42.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.092+0000 7f7009bba700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 0x7f6ff00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7000005fd0 tx=0x7f7000005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.095+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6ffc098170 con 0x7f7004100560 2023-07-21T22:15:42.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.281+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70041a0680 con 0x7f7004100560 2023-07-21T22:15:42.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.281+0000 7f6ffaffd700 1 -- 172.21.15.93:0/4018182660 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ffc021020 con 0x7f7004100560 2023-07-21T22:15:42.325 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:15:42.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 msgr2=0x7f6ff00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 0x7f6ff00a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7000005fd0 tx=0x7f7000005dd0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 msgr2=0x7f70041014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d010 tx=0x7f6ffc004d30 comp rx=0 tx=0).stop 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 shutdown_connections 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7004100560 0x7f70041014f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6ff00a11e0 0x7f6ff00a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70040fe9e0 0x7f7004102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 --2- 172.21.15.93:0/4018182660 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f70040ffbd0 0x7f7004100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 >> 172.21.15.93:0/4018182660 conn(0x7f70040fa000 msgr2=0x7f7004067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:42.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 shutdown_connections 2023-07-21T22:15:42.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.283+0000 7f700be1e700 1 -- 172.21.15.93:0/4018182660 wait complete. 2023-07-21T22:15:42.340 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:15:42.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- 172.21.15.93:0/4200997469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9140ff5c0 msgr2=0x7fe914106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 --2- 172.21.15.93:0/4200997469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9140ff5c0 0x7fe914106e70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00b0a0 tx=0x7fe90c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- 172.21.15.93:0/4200997469 shutdown_connections 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 --2- 172.21.15.93:0/4200997469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9140ff5c0 0x7fe914106e70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 --2- 172.21.15.93:0/4200997469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe914100050 0x7fe914100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 --2- 172.21.15.93:0/4200997469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140fec10 0x7fe9140ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- 172.21.15.93:0/4200997469 >> 172.21.15.93:0/4200997469 conn(0x7fe9140f9770 msgr2=0x7fe9140fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:42.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- 172.21.15.93:0/4200997469 shutdown_connections 2023-07-21T22:15:42.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- 172.21.15.93:0/4200997469 wait complete. 2023-07-21T22:15:42.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 Processor -- start 2023-07-21T22:15:42.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.417+0000 7fe919a05700 1 -- start start 2023-07-21T22:15:42.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9140fec10 0x7fe9141947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140ff5c0 0x7fe914194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419c320 con 0x7fe914100050 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419c460 con 0x7fe9140fec10 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419c5f0 con 0x7fe9140ff5c0 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42850/0 (socket says 172.21.15.93:42850) 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 -- 172.21.15.93:0/2323239777 learned_addr learned my addr 172.21.15.93:0/2323239777 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:42.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe912ffd700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9140fec10 0x7fe9141947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9127fc700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140ff5c0 0x7fe914194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 -- 172.21.15.93:0/2323239777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140ff5c0 msgr2=0x7fe914194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140ff5c0 0x7fe914194ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 -- 172.21.15.93:0/2323239777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9140fec10 msgr2=0x7fe9141947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9140fec10 0x7fe9141947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 -- 172.21.15.93:0/2323239777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c009d00 con 0x7fe914100050 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe9137fe700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00b780 tx=0x7fe90c00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00e040 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe90c004e60 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91419c870 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c005370 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.418+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91419cca0 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.419+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91410a1a0 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.420+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe90c01d470 con 0x7fe914100050 2023-07-21T22:15:42.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.421+0000 7fe918a03700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 0x7fe9000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.421+0000 7fe912ffd700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 0x7fe9000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.421+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe90c0ce450 con 0x7fe914100050 2023-07-21T22:15:42.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.421+0000 7fe912ffd700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 0x7fe9000a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009bf0 tx=0x7fe8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.424+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe90c097c80 con 0x7fe914100050 2023-07-21T22:15:42.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:42 smithi194 ceph-mon[98288]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:42.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:42 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3245832035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:42.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:42 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4200761814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:42 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2126361372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:42 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4018182660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:42 smithi134 ceph-mon[97873]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:42.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3245832035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4200761814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2126361372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:42 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4018182660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:42 smithi093 ceph-mon[96579]: pgmap v277: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:42.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3245832035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:42.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4200761814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2126361372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:42 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4018182660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:42.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.655+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe914105860 con 0x7fe914100050 2023-07-21T22:15:42.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.656+0000 7fe918a03700 1 -- 172.21.15.93:0/2323239777 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7fe90c09b2a0 con 0x7fe914100050 2023-07-21T22:15:42.699 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:42.700 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:15:42.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 msgr2=0x7fe9000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 0x7fe9000a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009bf0 tx=0x7fe8fc009260 comp rx=0 tx=0).stop 2023-07-21T22:15:42.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 msgr2=0x7fe91419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00b780 tx=0x7fe90c00bb60 comp rx=0 tx=0).stop 2023-07-21T22:15:42.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 shutdown_connections 2023-07-21T22:15:42.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe914100050 0x7fe91419bd50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe9000a1220 0x7fe9000a36d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9140fec10 0x7fe9141947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 --2- 172.21.15.93:0/2323239777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9140ff5c0 0x7fe914194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 >> 172.21.15.93:0/2323239777 conn(0x7fe9140f9770 msgr2=0x7fe9140fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:42.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 shutdown_connections 2023-07-21T22:15:42.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.658+0000 7fe919a05700 1 -- 172.21.15.93:0/2323239777 wait complete. 2023-07-21T22:15:42.705 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:15:42.714 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:15:42.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 -- 172.21.15.93:0/369895495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c100600 msgr2=0x7f396c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 --2- 172.21.15.93:0/369895495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c100600 0x7f396c106ea0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f396400b0a0 tx=0x7f396400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 -- 172.21.15.93:0/369895495 shutdown_connections 2023-07-21T22:15:42.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 --2- 172.21.15.93:0/369895495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c100600 0x7f396c106ea0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 --2- 172.21.15.93:0/369895495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 0x7f396c0ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 --2- 172.21.15.93:0/369895495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c0ffc50 0x7f396c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 -- 172.21.15.93:0/369895495 >> 172.21.15.93:0/369895495 conn(0x7f396c0f97a0 msgr2=0x7f396c0fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:42.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 -- 172.21.15.93:0/369895495 shutdown_connections 2023-07-21T22:15:42.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.791+0000 7f3971fa1700 1 -- 172.21.15.93:0/369895495 wait complete. 2023-07-21T22:15:42.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 Processor -- start 2023-07-21T22:15:42.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 -- start start 2023-07-21T22:15:42.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 0x7f396c195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c100600 0x7f396c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c19d400 con 0x7f396c0ffc50 2023-07-21T22:15:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c19d540 con 0x7f396c100600 2023-07-21T22:15:42.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f3971fa1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c19d6d0 con 0x7f396c0fef40 2023-07-21T22:15:42.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f396affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f396affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42862/0 (socket says 172.21.15.93:42862) 2023-07-21T22:15:42.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.792+0000 7f396affd700 1 -- 172.21.15.93:0/4080089421 learned_addr learned my addr 172.21.15.93:0/4080089421 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:42.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396b7fe700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 0x7f396c195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396bfff700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c100600 0x7f396c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 -- 172.21.15.93:0/4080089421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 msgr2=0x7f396c195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 0x7f396c195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 -- 172.21.15.93:0/4080089421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c100600 msgr2=0x7f396c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c100600 0x7f396c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 -- 172.21.15.93:0/4080089421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009d00 con 0x7f396c0ffc50 2023-07-21T22:15:42.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f396affd700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f396000e970 tx=0x7f396000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960009790 con 0x7f396c0ffc50 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39600044e0 con 0x7f396c0ffc50 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c19d9b0 con 0x7f396c0ffc50 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960003e30 con 0x7f396c0ffc50 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.793+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c19df10 con 0x7f396c0ffc50 2023-07-21T22:15:42.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.794+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c10a200 con 0x7f396c0ffc50 2023-07-21T22:15:42.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.795+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3960003f90 con 0x7f396c0ffc50 2023-07-21T22:15:42.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.795+0000 7f3968ff9700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 0x7f39540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:42.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.795+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3960014070 con 0x7f396c0ffc50 2023-07-21T22:15:42.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.795+0000 7f396b7fe700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 0x7f39540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:42.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.796+0000 7f396b7fe700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 0x7f39540a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f396c068440 tx=0x7f395c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:42.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.800+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3960096330 con 0x7f396c0ffc50 2023-07-21T22:15:43.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.992+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f396c10a3f0 con 0x7f396c0ffc50 2023-07-21T22:15:43.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.993+0000 7f3968ff9700 1 -- 172.21.15.93:0/4080089421 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f3960016330 con 0x7f396c0ffc50 2023-07-21T22:15:43.037 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:43.037 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 msgr2=0x7f39540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 0x7f39540a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f396c068440 tx=0x7f395c009260 comp rx=0 tx=0).stop 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 msgr2=0x7f396c195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f396000e970 tx=0x7f396000ed30 comp rx=0 tx=0).stop 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 shutdown_connections 2023-07-21T22:15:43.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f39540a1320 0x7f39540a37d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396c100600 0x7f396c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f396c0fef40 0x7f396c195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 --2- 172.21.15.93:0/4080089421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f396c0ffc50 0x7f396c195dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 >> 172.21.15.93:0/4080089421 conn(0x7f396c0f97a0 msgr2=0x7f396c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 shutdown_connections 2023-07-21T22:15:43.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:42.995+0000 7f3971fa1700 1 -- 172.21.15.93:0/4080089421 wait complete. 2023-07-21T22:15:43.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:15:43.053 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:15:43.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 -- 172.21.15.93:0/2148815468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 msgr2=0x7f41680fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 --2- 172.21.15.93:0/2148815468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f41680fedf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4158009990 tx=0x7f4158009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 -- 172.21.15.93:0/2148815468 shutdown_connections 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 --2- 172.21.15.93:0/2148815468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 0x7f4168106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 --2- 172.21.15.93:0/2148815468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f41680fedf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 --2- 172.21.15.93:0/2148815468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 0x7f4168100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 -- 172.21.15.93:0/2148815468 >> 172.21.15.93:0/2148815468 conn(0x7f41680fa000 msgr2=0x7f41680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 -- 172.21.15.93:0/2148815468 shutdown_connections 2023-07-21T22:15:43.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.133+0000 7f416db75700 1 -- 172.21.15.93:0/2148815468 wait complete. 2023-07-21T22:15:43.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 Processor -- start 2023-07-21T22:15:43.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 -- start start 2023-07-21T22:15:43.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 0x7f416819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 0x7f416819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416819d380 con 0x7f41680fe9e0 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416819d4f0 con 0x7f41680ffbd0 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f416db75700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416819d6b0 con 0x7f4168100560 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f41677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f41677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42874/0 (socket says 172.21.15.93:42874) 2023-07-21T22:15:43.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f41677fe700 1 -- 172.21.15.93:0/3177755880 learned_addr learned my addr 172.21.15.93:0/3177755880 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:43.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f4166ffd700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 0x7f416819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.134+0000 7f4167fff700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 0x7f416819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 -- 172.21.15.93:0/3177755880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 msgr2=0x7f416819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 0x7f416819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 -- 172.21.15.93:0/3177755880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 msgr2=0x7f416819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 0x7f416819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 -- 172.21.15.93:0/3177755880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580095f0 con 0x7f41680fe9e0 2023-07-21T22:15:43.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f41677fe700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415801d070 con 0x7f41680fe9e0 2023-07-21T22:15:43.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4158003f60 con 0x7f41680fe9e0 2023-07-21T22:15:43.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416819d930 con 0x7f41680fe9e0 2023-07-21T22:15:43.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158022b50 con 0x7f41680fe9e0 2023-07-21T22:15:43.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.135+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416819ddb0 con 0x7f41680fe9e0 2023-07-21T22:15:43.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.136+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416810a200 con 0x7f41680fe9e0 2023-07-21T22:15:43.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.137+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f41580040d0 con 0x7f41680fe9e0 2023-07-21T22:15:43.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.138+0000 7f4164ff9700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 0x7f41500a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.138+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f41580ce740 con 0x7f41680fe9e0 2023-07-21T22:15:43.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.138+0000 7f4166ffd700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 0x7f41500a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.138+0000 7f4166ffd700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 0x7f41500a3790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4168067f70 tx=0x7f415c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.141+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4158097e70 con 0x7f41680fe9e0 2023-07-21T22:15:43.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.331+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4168105890 con 0x7f41680fe9e0 2023-07-21T22:15:43.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.331+0000 7f4164ff9700 1 -- 172.21.15.93:0/3177755880 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f415809b490 con 0x7f41680fe9e0 2023-07-21T22:15:43.375 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:15:43.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 msgr2=0x7f41500a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 0x7f41500a3790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4168067f70 tx=0x7f415c0073d0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 msgr2=0x7f4168068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158005740 comp rx=0 tx=0).stop 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 shutdown_connections 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4168100560 0x7f416819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41680fe9e0 0x7f4168068200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.333+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41500a12e0 0x7f41500a3790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.334+0000 7f416db75700 1 --2- 172.21.15.93:0/3177755880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41680ffbd0 0x7f416819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.334+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 >> 172.21.15.93:0/3177755880 conn(0x7f41680fa000 msgr2=0x7f41680fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.334+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 shutdown_connections 2023-07-21T22:15:43.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.334+0000 7f416db75700 1 -- 172.21.15.93:0/3177755880 wait complete. 2023-07-21T22:15:43.389 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:15:43.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2750668662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 msgr2=0x7fb6f4107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2750668662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f4107fb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00b0a0 tx=0x7fb6ec00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.523 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2750668662 shutdown_connections 2023-07-21T22:15:43.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2750668662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f4107fb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2750668662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f40fdd90 0x7fb6f40fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2750668662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f4100d60 0x7fb6f41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2750668662 >> 172.21.15.93:0/2750668662 conn(0x7fb6f40f9770 msgr2=0x7fb6f40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2750668662 shutdown_connections 2023-07-21T22:15:43.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2750668662 wait complete. 2023-07-21T22:15:43.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 Processor -- start 2023-07-21T22:15:43.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- start start 2023-07-21T22:15:43.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f40fdd90 0x7fb6f41006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f4100d60 0x7fb6f40fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40ff9f0 con 0x7fb6f41016f0 2023-07-21T22:15:43.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40ffb60 con 0x7fb6f4100d60 2023-07-21T22:15:43.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.469+0000 7fb6fbb11700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419f900 con 0x7fb6f40fdd90 2023-07-21T22:15:43.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42892/0 (socket says 172.21.15.93:42892) 2023-07-21T22:15:43.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 -- 172.21.15.93:0/2723462649 learned_addr learned my addr 172.21.15.93:0/2723462649 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:43.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6f90ac700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f4100d60 0x7fb6f40fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6f98ad700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f40fdd90 0x7fb6f41006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 -- 172.21.15.93:0/2723462649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f40fdd90 msgr2=0x7fb6f41006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f40fdd90 0x7fb6f41006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 -- 172.21.15.93:0/2723462649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f4100d60 msgr2=0x7fb6f40fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f4100d60 0x7fb6f40fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 -- 172.21.15.93:0/2723462649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec009d00 con 0x7fb6f41016f0 2023-07-21T22:15:43.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fa0ae700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0062a0 tx=0x7fb6ec004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00e040 con 0x7fb6f41016f0 2023-07-21T22:15:43.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f419fb80 con 0x7fb6f41016f0 2023-07-21T22:15:43.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec00cec0 con 0x7fb6f41016f0 2023-07-21T22:15:43.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec004020 con 0x7fb6f41016f0 2023-07-21T22:15:43.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.470+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f41a0000 con 0x7fb6f41016f0 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.471+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f410b2b0 con 0x7fb6f41016f0 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.472+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb6ec01d690 con 0x7fb6f41016f0 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.473+0000 7fb6eaffd700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.473+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6ec0ce7a0 con 0x7fb6f41016f0 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.473+0000 7fb6f98ad700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.473+0000 7fb6f98ad700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 0x7fb6e00a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005950 tx=0x7fb6e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.476+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb6ec097fd0 con 0x7fb6f41016f0 2023-07-21T22:15:43.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:43 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2323239777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:43 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4080089421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:43 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2323239777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:43 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4080089421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:43 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2323239777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:43 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4080089421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:43.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.665+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6f41069b0 con 0x7fb6f41016f0 2023-07-21T22:15:43.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.666+0000 7fb6eaffd700 1 -- 172.21.15.93:0/2723462649 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6ec09b5f0 con 0x7fb6f41016f0 2023-07-21T22:15:43.710 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:15:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 msgr2=0x7fb6e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 0x7fb6e00a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005950 tx=0x7fb6e40058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 msgr2=0x7fb6f40ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0062a0 tx=0x7fb6ec004d00 comp rx=0 tx=0).stop 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 shutdown_connections 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6f41016f0 0x7fb6f40ff250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6f40fdd90 0x7fb6f41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb6e00a11e0 0x7fb6e00a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 --2- 172.21.15.93:0/2723462649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f4100d60 0x7fb6f40fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 >> 172.21.15.93:0/2723462649 conn(0x7fb6f40f9770 msgr2=0x7fb6f40fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 shutdown_connections 2023-07-21T22:15:43.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.668+0000 7fb6fbb11700 1 -- 172.21.15.93:0/2723462649 wait complete. 2023-07-21T22:15:43.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:15:43.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 -- 172.21.15.93:0/2661360346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f40fe9e0 msgr2=0x7f65f40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2661360346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f40fe9e0 0x7f65f40fedf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009990 tx=0x7f65e4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 -- 172.21.15.93:0/2661360346 shutdown_connections 2023-07-21T22:15:43.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2661360346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f4100560 0x7f65f4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2661360346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f40fe9e0 0x7f65f40fedf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2661360346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40ffbd0 0x7f65f4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 -- 172.21.15.93:0/2661360346 >> 172.21.15.93:0/2661360346 conn(0x7f65f40fa000 msgr2=0x7f65f40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:43.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 -- 172.21.15.93:0/2661360346 shutdown_connections 2023-07-21T22:15:43.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 -- 172.21.15.93:0/2661360346 wait complete. 2023-07-21T22:15:43.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.804+0000 7f65fa07d700 1 Processor -- start 2023-07-21T22:15:43.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 -- start start 2023-07-21T22:15:43.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 0x7f65f4068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f40ffbd0 0x7f65f419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f419d330 con 0x7f65f4100560 2023-07-21T22:15:43.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f419d4a0 con 0x7f65f40fe9e0 2023-07-21T22:15:43.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65fa07d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f419d660 con 0x7f65f40ffbd0 2023-07-21T22:15:43.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42904/0 (socket says 172.21.15.93:42904) 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 -- 172.21.15.93:0/2491858192 learned_addr learned my addr 172.21.15.93:0/2491858192 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f2ffd700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f40ffbd0 0x7f65f419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f37fe700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 0x7f65f4068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 -- 172.21.15.93:0/2491858192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f40ffbd0 msgr2=0x7f65f419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f40ffbd0 0x7f65f419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 -- 172.21.15.93:0/2491858192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 msgr2=0x7f65f4068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:43.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 0x7f65f4068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 -- 172.21.15.93:0/2491858192 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40095f0 con 0x7f65f4100560 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f37fe700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 0x7f65f4068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.805+0000 7f65f3fff700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65ec007ee0 tx=0x7f65ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x7f65f4100560 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f419d940 con 0x7f65f4100560 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65ec00f040 con 0x7f65f4100560 2023-07-21T22:15:43.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec005740 con 0x7f65f4100560 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f419de50 con 0x7f65f4100560 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.806+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f410a1b0 con 0x7f65f4100560 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.807+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f65ec00dc60 con 0x7f65f4100560 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.808+0000 7f65f0ff9700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 0x7f65dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.808+0000 7f65f37fe700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 0x7f65dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.808+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f65ec0cc100 con 0x7f65f4100560 2023-07-21T22:15:43.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.808+0000 7f65f37fe700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 0x7f65dc0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:43.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.812+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f65ec0958a0 con 0x7f65f4100560 2023-07-21T22:15:44.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.999+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65f4105890 con 0x7f65f4100560 2023-07-21T22:15:44.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:43.999+0000 7f65f0ff9700 1 -- 172.21.15.93:0/2491858192 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65ec095020 con 0x7f65f4100560 2023-07-21T22:15:44.043 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:15:44.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 msgr2=0x7f65dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 0x7f65dc0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 msgr2=0x7f65f419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f65ec007ee0 tx=0x7f65ec00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 shutdown_connections 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65f4100560 0x7f65f419cb60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f65dc0a11e0 0x7f65dc0a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65f40fe9e0 0x7f65f4068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 --2- 172.21.15.93:0/2491858192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65f40ffbd0 0x7f65f419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.001+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 >> 172.21.15.93:0/2491858192 conn(0x7f65f40fa000 msgr2=0x7f65f40fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.002+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 shutdown_connections 2023-07-21T22:15:44.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.002+0000 7f65fa07d700 1 -- 172.21.15.93:0/2491858192 wait complete. 2023-07-21T22:15:44.057 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:15:44.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 -- 172.21.15.93:0/1953591490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fec20 msgr2=0x7f9de40ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1953591490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fec20 0x7f9de40ff090 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009990 tx=0x7f9dd4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 -- 172.21.15.93:0/1953591490 shutdown_connections 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1953591490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de4106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1953591490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fffd0 0x7f9de41003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1953591490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fec20 0x7f9de40ff090 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 -- 172.21.15.93:0/1953591490 >> 172.21.15.93:0/1953591490 conn(0x7f9de40f97a0 msgr2=0x7f9de40fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 -- 172.21.15.93:0/1953591490 shutdown_connections 2023-07-21T22:15:44.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 -- 172.21.15.93:0/1953591490 wait complete. 2023-07-21T22:15:44.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.135+0000 7f9de92c2700 1 Processor -- start 2023-07-21T22:15:44.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 -- start start 2023-07-21T22:15:44.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 0x7f9de40fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de419d2c0 con 0x7f9de40fffd0 2023-07-21T22:15:44.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de419d430 con 0x7f9de40fec20 2023-07-21T22:15:44.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de419d5f0 con 0x7f9de40ff5d0 2023-07-21T22:15:44.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42918/0 (socket says 172.21.15.93:42918) 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 -- 172.21.15.93:0/1943140284 learned_addr learned my addr 172.21.15.93:0/1943140284 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de27fc700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de2ffd700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 0x7f9de40fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 -- 172.21.15.93:0/1943140284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 msgr2=0x7f9de419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 -- 172.21.15.93:0/1943140284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 msgr2=0x7f9de40fe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 0x7f9de40fe6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 -- 172.21.15.93:0/1943140284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd40095f0 con 0x7f9de40fffd0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de2ffd700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 0x7f9de40fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de27fc700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de37fe700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b970 tx=0x7f9ddc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00c9b0 con 0x7f9de40fffd0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de419d8d0 con 0x7f9de40fffd0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ddc014440 con 0x7f9de40fffd0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.136+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0075d0 con 0x7f9de40fffd0 2023-07-21T22:15:44.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.137+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de419dd90 con 0x7f9de40fffd0 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.137+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de410a250 con 0x7f9de40fffd0 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.139+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9ddc0145b0 con 0x7f9de40fffd0 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.139+0000 7f9dcbfff700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 0x7f9dcc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.139+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9ddc006e40 con 0x7f9de40fffd0 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.139+0000 7f9de2ffd700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 0x7f9dcc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.139+0000 7f9de2ffd700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 0x7f9dcc0a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.142+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ddc095ca0 con 0x7f9de40fffd0 2023-07-21T22:15:44.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.376+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de410a440 con 0x7f9de40fffd0 2023-07-21T22:15:44.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.377+0000 7f9dcbfff700 1 -- 172.21.15.93:0/1943140284 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f9ddc0992c0 con 0x7f9de40fffd0 2023-07-21T22:15:44.421 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:44.421 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:15:44.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 msgr2=0x7f9dcc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 0x7f9dcc0a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd8005e20 comp rx=0 tx=0).stop 2023-07-21T22:15:44.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 msgr2=0x7f9de419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b970 tx=0x7f9ddc00bd30 comp rx=0 tx=0).stop 2023-07-21T22:15:44.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 shutdown_connections 2023-07-21T22:15:44.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9de40fffd0 0x7f9de419caf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9dcc0a1220 0x7f9dcc0a36d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de40fec20 0x7f9de40fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 --2- 172.21.15.93:0/1943140284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9de40ff5d0 0x7f9de419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 >> 172.21.15.93:0/1943140284 conn(0x7f9de40f97a0 msgr2=0x7f9de40fb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 shutdown_connections 2023-07-21T22:15:44.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.379+0000 7f9de92c2700 1 -- 172.21.15.93:0/1943140284 wait complete. 2023-07-21T22:15:44.425 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:15:44.436 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:15:44.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.513+0000 7f8350d58700 1 -- 172.21.15.93:0/3038180877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 msgr2=0x7f834c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.513+0000 7f8350d58700 1 --2- 172.21.15.93:0/3038180877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c107fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f834400b0a0 tx=0x7f834400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- 172.21.15.93:0/3038180877 shutdown_connections 2023-07-21T22:15:44.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- 172.21.15.93:0/3038180877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c107fb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- 172.21.15.93:0/3038180877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 0x7f834c0fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- 172.21.15.93:0/3038180877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 0x7f834c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- 172.21.15.93:0/3038180877 >> 172.21.15.93:0/3038180877 conn(0x7f834c0f9770 msgr2=0x7f834c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- 172.21.15.93:0/3038180877 shutdown_connections 2023-07-21T22:15:44.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- 172.21.15.93:0/3038180877 wait complete. 2023-07-21T22:15:44.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 Processor -- start 2023-07-21T22:15:44.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- start start 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 0x7f834c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 0x7f834c0fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0ff9f0 con 0x7f834c1016f0 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0ffb60 con 0x7f834c100d60 2023-07-21T22:15:44.563 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.514+0000 7f8350d58700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c19f8f0 con 0x7f834c0fdd90 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42932/0 (socket says 172.21.15.93:42932) 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 -- 172.21.15.93:0/2401517621 learned_addr learned my addr 172.21.15.93:0/2401517621 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f8349d9b700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 0x7f834c0fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834a59c700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 0x7f834c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 -- 172.21.15.93:0/2401517621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 msgr2=0x7f834c1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.564 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 0x7f834c1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 -- 172.21.15.93:0/2401517621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 msgr2=0x7f834c0fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 0x7f834c0fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 -- 172.21.15.93:0/2401517621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344009d00 con 0x7f834c1016f0 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f834ad9d700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f83440062a0 tx=0x7f8344004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400e040 con 0x7f834c1016f0 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f834400cec0 con 0x7f834c1016f0 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c19fb70 con 0x7f834c1016f0 2023-07-21T22:15:44.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834401baf0 con 0x7f834c1016f0 2023-07-21T22:15:44.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.515+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c19ffa0 con 0x7f834c1016f0 2023-07-21T22:15:44.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.516+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c107040 con 0x7f834c1016f0 2023-07-21T22:15:44.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.517+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f834401bc50 con 0x7f834c1016f0 2023-07-21T22:15:44.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.518+0000 7f833b7fe700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 0x7f83340a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.518+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83440ce970 con 0x7f834c1016f0 2023-07-21T22:15:44.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.518+0000 7f834a59c700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 0x7f83340a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.518+0000 7f834a59c700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 0x7f83340a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f834c067470 tx=0x7f834000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.521+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f83440981a0 con 0x7f834c1016f0 2023-07-21T22:15:44.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:44 smithi134 ceph-mon[97873]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:15:44.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:44 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3177755880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:44 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2723462649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:44 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2491858192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:44 smithi093 ceph-mon[96579]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:15:44.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3177755880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2723462649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:44 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2491858192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.711+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f834c02ca20 con 0x7f834c1016f0 2023-07-21T22:15:44.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.713+0000 7f833b7fe700 1 -- 172.21.15.93:0/2401517621 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f834409b7c0 con 0x7f834c1016f0 2023-07-21T22:15:44.757 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:44.757 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":82,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:14:52.240249+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 msgr2=0x7f83340a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 0x7f83340a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f834c067470 tx=0x7f834000b410 comp rx=0 tx=0).stop 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 msgr2=0x7f834c0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f83440062a0 tx=0x7f8344004d00 comp rx=0 tx=0).stop 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 shutdown_connections 2023-07-21T22:15:44.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f834c1016f0 0x7f834c0ff250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f834c0fdd90 0x7f834c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83340a1220 0x7f83340a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 --2- 172.21.15.93:0/2401517621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f834c100d60 0x7f834c0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 >> 172.21.15.93:0/2401517621 conn(0x7f834c0f9770 msgr2=0x7f834c104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 shutdown_connections 2023-07-21T22:15:44.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.715+0000 7f8350d58700 1 -- 172.21.15.93:0/2401517621 wait complete. 2023-07-21T22:15:44.771 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:15:44.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:44 smithi194 ceph-mon[98288]: pgmap v278: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:15:44.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:44 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3177755880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:44 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2723462649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:44 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2491858192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.848+0000 7f9ee8b34700 1 -- 172.21.15.93:0/3014902612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee4100550 msgr2=0x7f9ee4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.848+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/3014902612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee4100550 0x7f9ee4106e10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00b0a0 tx=0x7f9edc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.848+0000 7f9ee8b34700 1 -- 172.21.15.93:0/3014902612 shutdown_connections 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/3014902612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee4100550 0x7f9ee4106e10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/3014902612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/3014902612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 0x7f9ee4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- 172.21.15.93:0/3014902612 >> 172.21.15.93:0/3014902612 conn(0x7f9ee40f9fd0 msgr2=0x7f9ee40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- 172.21.15.93:0/3014902612 shutdown_connections 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- 172.21.15.93:0/3014902612 wait complete. 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 Processor -- start 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- start start 2023-07-21T22:15:44.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 0x7f9ee419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee4100550 0x7f9ee419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419d380 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419d4f0 con 0x7f9ee4100550 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.849+0000 7f9ee8b34700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419d6b0 con 0x7f9ee40ffbc0 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42942/0 (socket says 172.21.15.93:42942) 2023-07-21T22:15:44.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 -- 172.21.15.93:0/33708206 learned_addr learned my addr 172.21.15.93:0/33708206 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee2d9d700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee4100550 0x7f9ee419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 -- 172.21.15.93:0/33708206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 msgr2=0x7f9ee419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee1d9b700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 0x7f9ee419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 0x7f9ee419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 -- 172.21.15.93:0/33708206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee4100550 msgr2=0x7f9ee419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee4100550 0x7f9ee419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:44.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 -- 172.21.15.93:0/33708206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc009d00 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee259c700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e970 tx=0x7f9ed400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4009790 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee419d990 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ed400eec0 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4003cf0 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.850+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee419def0 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.851+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4105e50 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.852+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9ed400ccd0 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.853+0000 7f9ed37fe700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 0x7f9ecc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:44.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.853+0000 7f9ee1d9b700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 0x7f9ecc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:44.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.853+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9ed4014070 con 0x7f9ee40fe9d0 2023-07-21T22:15:44.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.853+0000 7f9ee1d9b700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 0x7f9ecc0a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4067f70 tx=0x7f9ed8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:44.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:44.856+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ed4096420 con 0x7f9ee40fe9d0 2023-07-21T22:15:45.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.088+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ee402c850 con 0x7f9ee40fe9d0 2023-07-21T22:15:45.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.089+0000 7f9ed37fe700 1 -- 172.21.15.93:0/33708206 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f9ed4099a40 con 0x7f9ee40fe9d0 2023-07-21T22:15:45.133 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:15:45.133 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 msgr2=0x7f9ecc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 0x7f9ecc0a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4067f70 tx=0x7f9ed8005d50 comp rx=0 tx=0).stop 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 msgr2=0x7f9ee4068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e970 tx=0x7f9ed400ec80 comp rx=0 tx=0).stop 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 shutdown_connections 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f9ecc0a1220 0x7f9ecc0a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ee4100550 0x7f9ee419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ee40fe9d0 0x7f9ee4068200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 --2- 172.21.15.93:0/33708206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ee40ffbc0 0x7f9ee419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 >> 172.21.15.93:0/33708206 conn(0x7f9ee40f9fd0 msgr2=0x7f9ee4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 shutdown_connections 2023-07-21T22:15:45.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.091+0000 7f9ee8b34700 1 -- 172.21.15.93:0/33708206 wait complete. 2023-07-21T22:15:45.137 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:15:45.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:14:50.635520+0000', 'modified': '2023-07-21T22:14:58.012216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24482, 'mds_2': 24460, 'mds_3': 24448, 'mds_4': 14589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14577': {'gid': 14577, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.93:6839/3482994338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6838', 'nonce': 3482994338}, {'type': 'v1', 'addr': '172.21.15.93:6839', 'nonce': 3482994338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14589': {'gid': 14589, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.93:6841/4095455709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6840', 'nonce': 4095455709}, {'type': 'v1', 'addr': '172.21.15.93:6841', 'nonce': 4095455709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2616544286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2616544286}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2616544286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/3262717838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3262717838}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3262717838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/3279794938', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 3279794938}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 3279794938}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/117715569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 117715569}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 117715569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.194:6835/127707947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 127707947}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 127707947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.194:6837/714728190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 714728190}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 714728190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/3043386858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3043386858}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3043386858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.194:6839/1819166614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 1819166614}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 1819166614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:15:45.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T22:15:45.148 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-21T22:15:45.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.225+0000 7fa8d8caa700 1 -- 172.21.15.93:0/3784058735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d40fec10 msgr2=0x7fa8d40ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.269 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.225+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/3784058735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d40fec10 0x7fa8d40ff080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009990 tx=0x7fa8c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 -- 172.21.15.93:0/3784058735 shutdown_connections 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/3784058735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40ff5c0 0x7fa8d4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/3784058735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d4100050 0x7fa8d4100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/3784058735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d40fec10 0x7fa8d40ff080 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 -- 172.21.15.93:0/3784058735 >> 172.21.15.93:0/3784058735 conn(0x7fa8d40f9770 msgr2=0x7fa8d40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 -- 172.21.15.93:0/3784058735 shutdown_connections 2023-07-21T22:15:45.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 -- 172.21.15.93:0/3784058735 wait complete. 2023-07-21T22:15:45.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 Processor -- start 2023-07-21T22:15:45.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.226+0000 7fa8d8caa700 1 -- start start 2023-07-21T22:15:45.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 0x7fa8d4194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d40ff5c0 0x7fa8d4194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d419c290 con 0x7fa8d4100050 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d419c3d0 con 0x7fa8d40fec10 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d419c560 con 0x7fa8d40ff5c0 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.273 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42960/0 (socket says 172.21.15.93:42960) 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1990850358 learned_addr learned my addr 172.21.15.93:0/1990850358 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 0x7fa8d4194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d40ff5c0 0x7fa8d4194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1990850358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d40ff5c0 msgr2=0x7fa8d4194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d40ff5c0 0x7fa8d4194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1990850358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 msgr2=0x7fa8d4194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 0x7fa8d4194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1990850358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c80095f0 con 0x7fa8d4100050 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 0x7fa8d4194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc007d70 tx=0x7fa8cc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8c37fe700 1 -- 172.21.15.93:0/1990850358 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00ad40 con 0x7fa8d4100050 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8c37fe700 1 -- 172.21.15.93:0/1990850358 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8cc0128e0 con 0x7fa8d4100050 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d419c840 con 0x7fa8d4100050 2023-07-21T22:15:45.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8c37fe700 1 -- 172.21.15.93:0/1990850358 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011780 con 0x7fa8d4100050 2023-07-21T22:15:45.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.227+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d419cd50 con 0x7fa8d4100050 2023-07-21T22:15:45.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.229+0000 7fa8c37fe700 1 -- 172.21.15.93:0/1990850358 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa8cc012a50 con 0x7fa8d4100050 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.230+0000 7fa8c37fe700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 0x7fa8bc0a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.230+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 0x7fa8bc0a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.230+0000 7fa8c37fe700 1 -- 172.21.15.93:0/1990850358 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8cc0ccbc0 con 0x7fa8d4100050 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.230+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 0x7fa8bc0a3790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40099d0 tx=0x7fa8c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 Processor -- start 2023-07-21T22:15:45.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 -- start start 2023-07-21T22:15:45.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 0x55dc1636f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc1636f9c0 0x55dc1636fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16274c60 con 0x55dc163707e0 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc162c79f0 con 0x55dc16371150 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc162c0e80 con 0x55dc1636f9c0 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42964/0 (socket says 172.21.15.93:42964) 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.231+0000 7fa8d259c700 1 -- 172.21.15.93:0/2084949427 learned_addr learned my addr 172.21.15.93:0/2084949427 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 0x55dc1636f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 -- 172.21.15.93:0/2084949427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc1636f9c0 msgr2=0x55dc1636fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc1636f9c0 0x55dc1636fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc1636f9c0 0x55dc1636fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 -- 172.21.15.93:0/2084949427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 msgr2=0x55dc1636f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 0x55dc1636f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 -- 172.21.15.93:0/2084949427 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc160eed90 con 0x55dc163707e0 2023-07-21T22:15:45.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 0x55dc1636f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8d259c700 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400d850 tx=0x7fa8c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b249b90b6dc44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8c0ff9700 1 -- 172.21.15.93:0/2084949427 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40146c0 con 0x55dc163707e0 2023-07-21T22:15:45.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8c0ff9700 1 -- 172.21.15.93:0/2084949427 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c4014ce0 con 0x55dc163707e0 2023-07-21T22:15:45.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.232+0000 7fa8c0ff9700 1 -- 172.21.15.93:0/2084949427 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40123e0 con 0x55dc163707e0 2023-07-21T22:15:45.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/2084949427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 msgr2=0x55dc16370bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400d850 tx=0x7fa8c4010000 comp rx=0 tx=0).stop 2023-07-21T22:15:45.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/2084949427 shutdown_connections 2023-07-21T22:15:45.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc1636f9c0 0x55dc1636fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc16370bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/2084949427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc16371150 0x55dc1636f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/2084949427 >> 172.21.15.93:0/2084949427 conn(0x55dc16364ea0 msgr2=0x55dc1636a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/2084949427 shutdown_connections 2023-07-21T22:15:45.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/2084949427 wait complete. 2023-07-21T22:15:45.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- start start 2023-07-21T22:15:45.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 Processor -- start 2023-07-21T22:15:45.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc163795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 0x55dc16380610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16260a70 con 0x55dc1636f9c0 2023-07-21T22:15:45.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc16260770 con 0x55dc163707e0 2023-07-21T22:15:45.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.233+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc161448d0 con 0x55dc16371150 2023-07-21T22:15:45.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8e4e2fb80 10 client.0 _ll_register_callbacks cb 0x55dc1625e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42970/0 (socket says 172.21.15.93:42970) 2023-07-21T22:15:45.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 -- 172.21.15.93:0/509010248 learned_addr learned my addr 172.21.15.93:0/509010248 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc163795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 0x55dc16380610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 -- 172.21.15.93:0/509010248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 msgr2=0x55dc16380610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 0x55dc16380610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 -- 172.21.15.93:0/509010248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 msgr2=0x55dc163795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc163795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 -- 172.21.15.93:0/509010248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c400e040 con 0x55dc1636f9c0 2023-07-21T22:15:45.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 0x55dc16380610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:45.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8d259c700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009ea0 tx=0x7fa8c401e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a699ff5af637bdb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4014ee0 con 0x55dc1636f9c0 2023-07-21T22:15:45.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8c40148d0 con 0x55dc1636f9c0 2023-07-21T22:15:45.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc1621a4a0 con 0x55dc1636f9c0 2023-07-21T22:15:45.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8c4014a90 con 0x55dc1636f9c0 2023-07-21T22:15:45.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8e4e2fb80 10 client.15003 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.234+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc160d7c10 con 0x55dc1636f9c0 2023-07-21T22:15:45.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc1621a4a0 con 0x55dc1636f9c0 2023-07-21T22:15:45.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8c4019070 con 0x55dc1636f9c0 2023-07-21T22:15:45.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 10 client.15003 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 10 client.15003 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T22:15:45.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 20 client.15003 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc163847f0 0x55dc16383650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 4 client.15003 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:15:45.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc162b9e60 con 0x55dc163847f0 2023-07-21T22:15:45.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc163847f0 0x55dc16383650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8b9ffb700 10 client.15003 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:15:45.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.235+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc163847f0 0x55dc16383650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9315af4c1cfaf262 server_cookie=5bbd59c569ad684a in_seq=0 out_seq=0 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/509010248 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc162b9e60 con 0x55dc163847f0 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8b9ffb700 10 client.15003 handle_command_reply: tid=0 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 client.15003 shutdown 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 msgr2=0x55dc16379090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009ea0 tx=0x7fa8c401e6f0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 shutdown_connections 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc163847f0 0x55dc16383650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16371150 0x55dc16380610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8b9ffb700 0 client.15003 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc16379090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/509010248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc163795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 >> 172.21.15.93:0/509010248 conn(0x55dc16364ea0 msgr2=0x55dc163699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 shutdown_connections 2023-07-21T22:15:45.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/509010248 wait complete. 2023-07-21T22:15:45.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.236+0000 7fa8e4e2fb80 20 client.15003 trim_cache size 0 max 16384 2023-07-21T22:15:45.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.253+0000 7fa8e4e2fb80 1 Processor -- start 2023-07-21T22:15:45.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.253+0000 7fa8e4e2fb80 1 -- start start 2023-07-21T22:15:45.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc16383650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc16376f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1621a4a0 con 0x55dc1636f9c0 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1626a470 con 0x55dc163707e0 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1628f790 con 0x55dc16364ea0 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42980/0 (socket says 172.21.15.93:42980) 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 -- 172.21.15.93:0/3570378741 learned_addr learned my addr 172.21.15.93:0/3570378741 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d259c700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc16383650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 -- 172.21.15.93:0/3570378741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 msgr2=0x55dc16383650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc16383650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 -- 172.21.15.93:0/3570378741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 msgr2=0x55dc16376f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc16376f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc16376f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 -- 172.21.15.93:0/3570378741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc16115200 con 0x55dc1636f9c0 2023-07-21T22:15:45.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc16376f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:45.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009990 tx=0x7fa8c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda4fe6b7fe6016a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8c17fa700 1 -- 172.21.15.93:0/3570378741 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800bb30 con 0x55dc1636f9c0 2023-07-21T22:15:45.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8c17fa700 1 -- 172.21.15.93:0/3570378741 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c801e040 con 0x55dc1636f9c0 2023-07-21T22:15:45.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.254+0000 7fa8c17fa700 1 -- 172.21.15.93:0/3570378741 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c80056a0 con 0x55dc1636f9c0 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/3570378741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 msgr2=0x55dc163805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009990 tx=0x7fa8c8009310 comp rx=0 tx=0).stop 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/3570378741 shutdown_connections 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc163707e0 0x55dc16376f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc16383650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/3570378741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc1636f9c0 0x55dc163805d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/3570378741 >> 172.21.15.93:0/3570378741 conn(0x55dc1636f5f0 msgr2=0x55dc16377d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/3570378741 shutdown_connections 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/3570378741 wait complete. 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- start start 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 Processor -- start 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc1637bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 0x55dc1637c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1637f630 con 0x55dc163707e0 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1637f7a0 con 0x55dc1636f9c0 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.255+0000 7fa8e4e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc1637f960 con 0x55dc16364ea0 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8e4e2fb80 10 client.0 _ll_register_callbacks cb 0x7fa8c400d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42988/0 (socket says 172.21.15.93:42988) 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1374268301 learned_addr learned my addr 172.21.15.93:0/1374268301 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc1637bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 0x55dc1637c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1374268301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 msgr2=0x55dc1637bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc1637bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1374268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 msgr2=0x55dc1637c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 0x55dc1637c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 -- 172.21.15.93:0/1374268301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d4107820 con 0x55dc163707e0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d1d9b700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 0x55dc1637c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8d2d9d700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc096530 tx=0x7fa8cc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86463248bc07f7bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc09e060 con 0x55dc163707e0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa8cc0a2480 con 0x55dc163707e0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8c4014ce0 con 0x55dc163707e0 2023-07-21T22:15:45.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8cc0a1ff0 con 0x55dc163707e0 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8e4e2fb80 10 client.15009 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc160ed260 con 0x55dc163707e0 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c4014ce0 con 0x55dc163707e0 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.256+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa8cc0b8300 con 0x55dc163707e0 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 10 client.15009 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 10 client.15009 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T22:15:45.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 20 client.15009 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc16371150 0x55dc163da610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 4 client.15009 mds_command: new command op to 14589 tid=0[{"prefix": "status"}] 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dc1627c000 con 0x55dc16371150 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc16371150 0x55dc163da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8b9ffb700 10 client.15009 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8d259c700 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc16371150 0x55dc163da610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f82cd4912dbfb743 server_cookie=7ec14c060dc917a5 in_seq=0 out_seq=0 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8b9ffb700 1 -- 172.21.15.93:0/1374268301 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55dc1627c000 con 0x55dc16371150 2023-07-21T22:15:45.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8b9ffb700 10 client.15009 handle_command_reply: tid=0 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 client.15009 shutdown 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 msgr2=0x55dc1637f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc096530 tx=0x7fa8cc007bd0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 shutdown_connections 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55dc163707e0 0x55dc1637f0f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55dc16371150 0x55dc163da610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dc16364ea0 0x55dc1637bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8b9ffb700 0 client.15009 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 --2- 172.21.15.93:0/1374268301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc1636f9c0 0x55dc1637c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 >> 172.21.15.93:0/1374268301 conn(0x55dc1636f5f0 msgr2=0x55dc16376f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 shutdown_connections 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 1 -- 172.21.15.93:0/1374268301 wait complete. 2023-07-21T22:15:45.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.257+0000 7fa8e4e2fb80 20 client.15009 trim_cache size 0 max 16384 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 4, 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 14589, 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:15:45.318 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 48.262013076999999, 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 82, 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 62.491651144999999 2023-07-21T22:15:45.319 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:15:45.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 msgr2=0x7fa8bc0a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 0x7fa8bc0a3790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40099d0 tx=0x7fa8c4008040 comp rx=0 tx=0).stop 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 msgr2=0x7fa8d419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc007d70 tx=0x7fa8cc00a340 comp rx=0 tx=0).stop 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 shutdown_connections 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8d4100050 0x7fa8d419bcc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fa8bc0a12e0 0x7fa8bc0a3790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8d40fec10 0x7fa8d4194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 --2- 172.21.15.93:0/1990850358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8d40ff5c0 0x7fa8d4194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 >> 172.21.15.93:0/1990850358 conn(0x7fa8d40f9770 msgr2=0x7fa8d41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 shutdown_connections 2023-07-21T22:15:45.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.258+0000 7fa8d8caa700 1 -- 172.21.15.93:0/1990850358 wait complete. 2023-07-21T22:15:45.322 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-21T22:15:45.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:45 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1943140284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:45 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2401517621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:45 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/33708206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 -- 172.21.15.93:0/4134368687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fffa0 msgr2=0x7f23c41003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 --2- 172.21.15.93:0/4134368687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fffa0 0x7f23c41003b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009990 tx=0x7f23ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 -- 172.21.15.93:0/4134368687 shutdown_connections 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 --2- 172.21.15.93:0/4134368687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 --2- 172.21.15.93:0/4134368687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fffa0 0x7f23c41003b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 --2- 172.21.15.93:0/4134368687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fec90 0x7f23c40ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 -- 172.21.15.93:0/4134368687 >> 172.21.15.93:0/4134368687 conn(0x7f23c40f9770 msgr2=0x7f23c40fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 -- 172.21.15.93:0/4134368687 shutdown_connections 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.402+0000 7f23ca271700 1 -- 172.21.15.93:0/4134368687 wait complete. 2023-07-21T22:15:45.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 Processor -- start 2023-07-21T22:15:45.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 -- start start 2023-07-21T22:15:45.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fffa0 0x7f23c419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419d380 con 0x7f23c40fec90 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419d4f0 con 0x7f23c40fffa0 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23ca271700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419d6b0 con 0x7f23c40ff640 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42996/0 (socket says 172.21.15.93:42996) 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 -- 172.21.15.93:0/3225591173 learned_addr learned my addr 172.21.15.93:0/3225591173 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c880e700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fffa0 0x7f23c419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 -- 172.21.15.93:0/3225591173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 msgr2=0x7f23c419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 -- 172.21.15.93:0/3225591173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fffa0 msgr2=0x7f23c419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fffa0 0x7f23c419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c3fff700 1 -- 172.21.15.93:0/3225591173 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0095f0 con 0x7f23c40fec90 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.403+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23c17fa700 1 -- 172.21.15.93:0/3225591173 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01d070 con 0x7f23c40fec90 2023-07-21T22:15:45.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23c17fa700 1 -- 172.21.15.93:0/3225591173 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23ac004060 con 0x7f23c40fec90 2023-07-21T22:15:45.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c419d930 con 0x7f23c40fec90 2023-07-21T22:15:45.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23c17fa700 1 -- 172.21.15.93:0/3225591173 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac022d30 con 0x7f23c40fec90 2023-07-21T22:15:45.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.404+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c419dde0 con 0x7f23c40fec90 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.406+0000 7f23c17fa700 1 -- 172.21.15.93:0/3225591173 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f23ac003ba0 con 0x7f23c40fec90 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.407+0000 7f23c17fa700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.407+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.407+0000 7f23c17fa700 1 -- 172.21.15.93:0/3225591173 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f23ac0cdcc0 con 0x7f23c40fec90 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.407+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 0x7f23b00a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005950 tx=0x7f23b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 Processor -- start 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 -- start start 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a2d0c60 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a3239f0 con 0x55ff9a3cd150 2023-07-21T22:15:45.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a31ce80 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43008/0 (socket says 172.21.15.93:43008) 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 -- 172.21.15.93:0/3512252758 learned_addr learned my addr 172.21.15.93:0/3512252758 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 -- 172.21.15.93:0/3512252758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 msgr2=0x55ff9a3ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 -- 172.21.15.93:0/3512252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 msgr2=0x55ff9a3cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 -- 172.21.15.93:0/3512252758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff9a14ad90 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.408+0000 7f23c880e700 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27a6bb57c846b5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23ba7fc700 1 -- 172.21.15.93:0/3512252758 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc019070 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23ba7fc700 1 -- 172.21.15.93:0/3512252758 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23bc00f040 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23ba7fc700 1 -- 172.21.15.93:0/3512252758 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc007bc0 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3512252758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 msgr2=0x55ff9a3cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc009c00 comp rx=0 tx=0).stop 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3512252758 shutdown_connections 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cbe40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3512252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3512252758 >> 172.21.15.93:0/3512252758 conn(0x55ff9a3c0ea0 msgr2=0x55ff9a3c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3512252758 shutdown_connections 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.409+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3512252758 wait complete. 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 -- start start 2023-07-21T22:15:45.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 Processor -- start 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a2bca70 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a2bc770 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a1a08d0 con 0x55ff9a3cd150 2023-07-21T22:15:45.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23d63f6b80 10 client.0 _ll_register_callbacks cb 0x55ff9a2ba710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23c37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23c37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43012/0 (socket says 172.21.15.93:43012) 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23c37fe700 1 -- 172.21.15.93:0/1609107165 learned_addr learned my addr 172.21.15.93:0/1609107165 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23c3fff700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.410+0000 7f23c880e700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 -- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cd150 msgr2=0x55ff9a3dc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 -- 172.21.15.93:0/1609107165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 msgr2=0x55ff9a3d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 -- 172.21.15.93:0/1609107165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc003c50 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c3fff700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23c37fe700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23b400ff60 tx=0x7f23b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307444f680e6c7c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4017040 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f23b4003a90 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff9a2764a0 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b4004580 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 10 client.15021 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff9a133c10 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff9a2764a0 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23b4011c90 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 10 client.15021 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 10 client.15021 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 20 client.15021 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a3e02e0 0x55ff9a3df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 4 client.15021 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.411+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff9a315e60 con 0x55ff9a3e02e0 2023-07-21T22:15:45.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.412+0000 7f23c3fff700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a3e02e0 0x55ff9a3df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.412+0000 7f23c3fff700 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a3e02e0 0x55ff9a3df6c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1971b6170b0cc366 server_cookie=2760455aa5520759 in_seq=0 out_seq=0 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.412+0000 7f23a37fe700 10 client.15021 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23a37fe700 1 -- 172.21.15.93:0/1609107165 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff9a315e60 con 0x55ff9a3e02e0 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23a37fe700 10 client.15021 handle_command_reply: tid=0 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 client.15021 shutdown 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 msgr2=0x55ff9a3d5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23b400ff60 tx=0x7f23b4004750 comp rx=0 tx=0).stop 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 shutdown_connections 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a3e02e0 0x55ff9a3df6c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/1609107165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d5560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23a37fe700 0 client.15021 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:15:45.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 >> 172.21.15.93:0/1609107165 conn(0x55ff9a3c0ea0 msgr2=0x55ff9a3c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 shutdown_connections 2023-07-21T22:15:45.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 1 -- 172.21.15.93:0/1609107165 wait complete. 2023-07-21T22:15:45.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.413+0000 7f23d63f6b80 20 client.15021 trim_cache size 0 max 16384 2023-07-21T22:15:45.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.430+0000 7f23d63f6b80 1 Processor -- start 2023-07-21T22:15:45.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 -- start start 2023-07-21T22:15:45.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a2eb790 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a1acde0 con 0x55ff9a3cd150 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a2db480 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43020/0 (socket says 172.21.15.93:43020) 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 -- 172.21.15.93:0/3364944188 learned_addr learned my addr 172.21.15.93:0/3364944188 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c3fff700 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 -- 172.21.15.93:0/3364944188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 msgr2=0x55ff9a3cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 -- 172.21.15.93:0/3364944188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 msgr2=0x55ff9a3dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 -- 172.21.15.93:0/3364944188 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff9a171200 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23c37fe700 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f23b401f430 tx=0x7f23b401f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f9abb6310a6080 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23baffd700 1 -- 172.21.15.93:0/3364944188 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401fbe0 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23baffd700 1 -- 172.21.15.93:0/3364944188 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b4004310 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.431+0000 7f23baffd700 1 -- 172.21.15.93:0/3364944188 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b402c6e0 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3364944188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 msgr2=0x55ff9a3dc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f23b401f430 tx=0x7f23b401f650 comp rx=0 tx=0).stop 2023-07-21T22:15:45.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3364944188 shutdown_connections 2023-07-21T22:15:45.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/3364944188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3dc4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3364944188 >> 172.21.15.93:0/3364944188 conn(0x55ff9a3c0ea0 msgr2=0x55ff9a3cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3364944188 shutdown_connections 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- 172.21.15.93:0/3364944188 wait complete. 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 -- start start 2023-07-21T22:15:45.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.432+0000 7f23d63f6b80 1 Processor -- start 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a3db090 con 0x55ff9a3cd150 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a3db200 con 0x55ff9a3cc7e0 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9a3db3c0 con 0x55ff9a3cb9c0 2023-07-21T22:15:45.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 10 client.0 _ll_register_callbacks cb 0x7f23b400bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43028/0 (socket says 172.21.15.93:43028) 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 -- 172.21.15.93:0/65253272 learned_addr learned my addr 172.21.15.93:0/65253272 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c37fe700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 -- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 msgr2=0x55ff9a3d7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 -- 172.21.15.93:0/65253272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cc7e0 msgr2=0x55ff9a3d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 -- 172.21.15.93:0/65253272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff9a133c10 con 0x55ff9a3cd150 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23c880e700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede5b7e85186e8de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc019070 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f23bc005240 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.433+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23bc00f040 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23bc005400 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 10 client.15027 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ff9a149260 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23bc02f020 con 0x55ff9a3cd150 2023-07-21T22:15:45.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23bc007b80 con 0x55ff9a3cd150 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 10 client.15027 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 20 client.15027 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a4398d0 0x55ff9a436280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 4 client.15027 mds_command: new command op to 24448 tid=0[{"prefix": "status"}] 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff9a2d8000 con 0x55ff9a4398d0 2023-07-21T22:15:45.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.434+0000 7f23c3fff700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a4398d0 0x55ff9a436280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23a37fe700 10 client.15027 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23c3fff700 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a4398d0 0x55ff9a436280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=920581fe871f3064 server_cookie=515d7960da231319 in_seq=0 out_seq=0 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23a37fe700 1 -- 172.21.15.93:0/65253272 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ff9a2d8000 con 0x55ff9a4398d0 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23a37fe700 10 client.15027 handle_command_reply: tid=0 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 client.15027 shutdown 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 msgr2=0x55ff9a3dab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc004d40 comp rx=0 tx=0).stop 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 shutdown_connections 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55ff9a4398d0 0x55ff9a436280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff9a3cd150 0x55ff9a3dab50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ff9a3cb9c0 0x55ff9a3d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 --2- 172.21.15.93:0/65253272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ff9a3cc7e0 0x55ff9a3d7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23a37fe700 0 client.15027 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:15:45.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 >> 172.21.15.93:0/65253272 conn(0x55ff9a3c0ea0 msgr2=0x55ff9a3dcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 shutdown_connections 2023-07-21T22:15:45.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 1 -- 172.21.15.93:0/65253272 wait complete. 2023-07-21T22:15:45.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23d63f6b80 20 client.15027 trim_cache size 0 max 16384 2023-07-21T22:15:45.490 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:15:45.490 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 3, 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24448, 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 49.159219493000002, 2023-07-21T22:15:45.491 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:15:45.492 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 82, 2023-07-21T22:15:45.492 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:15:45.492 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 73.669150908999995 2023-07-21T22:15:45.492 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:15:45.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 msgr2=0x7f23b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.435+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 0x7f23b00a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005950 tx=0x7f23b40058e0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 msgr2=0x7f23c40fe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac0047b0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 shutdown_connections 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23c40fffa0 0x7f23c419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23c40fec90 0x7f23c40fe710 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 --2- 172.21.15.93:0/3225591173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23c40ff640 0x7f23c419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 >> 172.21.15.93:0/3225591173 conn(0x7f23c40f9770 msgr2=0x7f23c41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 shutdown_connections 2023-07-21T22:15:45.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.436+0000 7f23ca271700 1 -- 172.21.15.93:0/3225591173 wait complete. 2023-07-21T22:15:45.495 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-21T22:15:45.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 -- 172.21.15.93:0/2767581291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 msgr2=0x7f190c100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 --2- 172.21.15.93:0/2767581291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c100100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1900009990 tx=0x7f1900009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 -- 172.21.15.93:0/2767581291 shutdown_connections 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 --2- 172.21.15.93:0/2767581291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c100640 0x7f190c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 --2- 172.21.15.93:0/2767581291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c0fea00 0x7f190c0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 --2- 172.21.15.93:0/2767581291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c100100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 -- 172.21.15.93:0/2767581291 >> 172.21.15.93:0/2767581291 conn(0x7f190c0fa020 msgr2=0x7f190c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 -- 172.21.15.93:0/2767581291 shutdown_connections 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.574+0000 7f19140b6700 1 -- 172.21.15.93:0/2767581291 wait complete. 2023-07-21T22:15:45.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 Processor -- start 2023-07-21T22:15:45.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 -- start start 2023-07-21T22:15:45.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c0fea00 0x7f190c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c100640 0x7f190c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19d330 con 0x7f190c0ffc90 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19d4a0 con 0x7f190c100640 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f19140b6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19d660 con 0x7f190c0fea00 2023-07-21T22:15:45.622 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43040/0 (socket says 172.21.15.93:43040) 2023-07-21T22:15:45.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 -- 172.21.15.93:0/3239536001 learned_addr learned my addr 172.21.15.93:0/3239536001 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911e52700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c0fea00 0x7f190c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1912653700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c100640 0x7f190c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 -- 172.21.15.93:0/3239536001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c0fea00 msgr2=0x7f190c0693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c0fea00 0x7f190c0693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 -- 172.21.15.93:0/3239536001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c100640 msgr2=0x7f190c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c100640 0x7f190c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.575+0000 7f1911651700 1 -- 172.21.15.93:0/3239536001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19000095f0 con 0x7f190c0ffc90 2023-07-21T22:15:45.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f1911651700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1900000c00 tx=0x7f190000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f18feffd700 1 -- 172.21.15.93:0/3239536001 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190001d070 con 0x7f190c0ffc90 2023-07-21T22:15:45.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c19d8e0 con 0x7f190c0ffc90 2023-07-21T22:15:45.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f18feffd700 1 -- 172.21.15.93:0/3239536001 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190000fb50 con 0x7f190c0ffc90 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f18feffd700 1 -- 172.21.15.93:0/3239536001 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000173f0 con 0x7f190c0ffc90 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.576+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c19dd90 con 0x7f190c0ffc90 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.578+0000 7f18feffd700 1 -- 172.21.15.93:0/3239536001 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f190000fcc0 con 0x7f190c0ffc90 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.578+0000 7f18feffd700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 0x7f18f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.578+0000 7f1911e52700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 0x7f18f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.578+0000 7f18feffd700 1 -- 172.21.15.93:0/3239536001 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19000cdad0 con 0x7f190c0ffc90 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f1911e52700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 0x7f18f80a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1908007c60 tx=0x7f19080073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 Processor -- start 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 -- start start 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763414dd0 0x55b763415250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763315c60 con 0x55b763415bf0 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7633689f0 con 0x55b763416560 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f192023bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763361e80 con 0x55b763414dd0 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43046/0 (socket says 172.21.15.93:43046) 2023-07-21T22:15:45.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.579+0000 7f1911e52700 1 -- 172.21.15.93:0/693484076 learned_addr learned my addr 172.21.15.93:0/693484076 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911651700 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1912653700 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763414dd0 0x55b763415250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 -- 172.21.15.93:0/693484076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763414dd0 msgr2=0x55b763415250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763414dd0 0x55b763415250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 -- 172.21.15.93:0/693484076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 msgr2=0x55b7634147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 -- 172.21.15.93:0/693484076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b76318fd90 con 0x55b763415bf0 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f1911e52700 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1908010f70 tx=0x7f1908013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8393856305e88a20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f18f7fff700 1 -- 172.21.15.93:0/693484076 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908011db0 con 0x55b763415bf0 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f18f7fff700 1 -- 172.21.15.93:0/693484076 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1908015070 con 0x55b763415bf0 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.580+0000 7f18f7fff700 1 -- 172.21.15.93:0/693484076 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908003d20 con 0x55b763415bf0 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- 172.21.15.93:0/693484076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 msgr2=0x55b763416000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1908010f70 tx=0x7f1908013000 comp rx=0 tx=0).stop 2023-07-21T22:15:45.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- 172.21.15.93:0/693484076 shutdown_connections 2023-07-21T22:15:45.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763414dd0 0x55b763415250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763415bf0 0x55b763416000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 --2- 172.21.15.93:0/693484076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- 172.21.15.93:0/693484076 >> 172.21.15.93:0/693484076 conn(0x55b763405ea0 msgr2=0x55b76340b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- 172.21.15.93:0/693484076 shutdown_connections 2023-07-21T22:15:45.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- 172.21.15.93:0/693484076 wait complete. 2023-07-21T22:15:45.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 -- start start 2023-07-21T22:15:45.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.581+0000 7f192023bb80 1 Processor -- start 2023-07-21T22:15:45.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b76341e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763301a70 con 0x55b763414dd0 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763301770 con 0x55b763416560 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7631e58d0 con 0x55b763415bf0 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 10 client.0 _ll_register_callbacks cb 0x55b7632ff710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43052/0 (socket says 172.21.15.93:43052) 2023-07-21T22:15:45.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 -- 172.21.15.93:0/1847342154 learned_addr learned my addr 172.21.15.93:0/1847342154 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1912653700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911651700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b76341e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 -- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 msgr2=0x55b76341e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b76341e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 -- 172.21.15.93:0/1847342154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 msgr2=0x55b7634259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 -- 172.21.15.93:0/1847342154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1908010c50 con 0x55b763414dd0 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f1911e52700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fe0 tx=0x7f1908011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0fd20ec911cc6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f18f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908005460 con 0x55b763414dd0 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1908017070 con 0x55b763414dd0 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7632bb4a0 con 0x55b763414dd0 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.582+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19080055c0 con 0x55b763414dd0 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 10 client.15039 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b763178c10 con 0x55b763414dd0 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7632bb4a0 con 0x55b763414dd0 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f190803d070 con 0x55b763414dd0 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 10 client.15039 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 20 client.15039 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b7634296b0 0x55b763428a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 4 client.15039 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b76335ae60 con 0x55b7634296b0 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f1911651700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b7634296b0 0x55b763428a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f18f57fa700 10 client.15039 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.583+0000 7f1911651700 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b7634296b0 0x55b763428a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9cd0103fccb191d server_cookie=243f28564be65356 in_seq=0 out_seq=0 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f18f57fa700 1 -- 172.21.15.93:0/1847342154 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b76335ae60 con 0x55b7634296b0 2023-07-21T22:15:45.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f18f57fa700 10 client.15039 handle_command_reply: tid=0 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 client.15039 shutdown 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 msgr2=0x55b76341e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fe0 tx=0x7f1908011e90 comp rx=0 tx=0).stop 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 shutdown_connections 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b7634259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341e470 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b7634296b0 0x55b763428a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 --2- 172.21.15.93:0/1847342154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b76341e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 >> 172.21.15.93:0/1847342154 conn(0x55b763405ea0 msgr2=0x55b76340a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.584+0000 7f18f57fa700 0 client.15039 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:15:45.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.585+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 shutdown_connections 2023-07-21T22:15:45.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.585+0000 7f192023bb80 1 -- 172.21.15.93:0/1847342154 wait complete. 2023-07-21T22:15:45.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.585+0000 7f192023bb80 20 client.15039 trim_cache size 0 max 16384 2023-07-21T22:15:45.646 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:45 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1943140284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.647 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:45 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2401517621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.647 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:45 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/33708206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 Processor -- start 2023-07-21T22:15:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 -- start start 2023-07-21T22:15:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b7634259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b763426220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7632bb4d0 con 0x55b763414dd0 2023-07-21T22:15:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763403070 con 0x55b763416560 2023-07-21T22:15:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f192023bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76330b470 con 0x55b763415bf0 2023-07-21T22:15:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43060/0 (socket says 172.21.15.93:43060) 2023-07-21T22:15:45.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f1911e52700 1 -- 172.21.15.93:0/2503919012 learned_addr learned my addr 172.21.15.93:0/2503919012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.602+0000 7f1912653700 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b763426220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 -- 172.21.15.93:0/2503919012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 msgr2=0x55b7634259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b7634259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 -- 172.21.15.93:0/2503919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 msgr2=0x55b763426220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b763426220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 -- 172.21.15.93:0/2503919012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7631b6200 con 0x55b763414dd0 2023-07-21T22:15:45.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1912653700 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b763426220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f1911e52700 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f19080267b0 tx=0x7f1908010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c124f6a13ee70425 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f18fcff9700 1 -- 172.21.15.93:0/2503919012 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908026440 con 0x55b763414dd0 2023-07-21T22:15:45.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f18fcff9700 1 -- 172.21.15.93:0/2503919012 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190803a070 con 0x55b763414dd0 2023-07-21T22:15:45.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f18fcff9700 1 -- 172.21.15.93:0/2503919012 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908003c40 con 0x55b763414dd0 2023-07-21T22:15:45.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 -- 172.21.15.93:0/2503919012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 msgr2=0x55b76341d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f19080267b0 tx=0x7f1908010a40 comp rx=0 tx=0).stop 2023-07-21T22:15:45.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 -- 172.21.15.93:0/2503919012 shutdown_connections 2023-07-21T22:15:45.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763416560 0x55b763426220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b76341d1a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 --2- 172.21.15.93:0/2503919012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763415bf0 0x55b7634259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 -- 172.21.15.93:0/2503919012 >> 172.21.15.93:0/2503919012 conn(0x55b763405ea0 msgr2=0x55b7634151a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.603+0000 7f192023bb80 1 -- 172.21.15.93:0/2503919012 shutdown_connections 2023-07-21T22:15:45.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 -- 172.21.15.93:0/2503919012 wait complete. 2023-07-21T22:15:45.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 -- start start 2023-07-21T22:15:45.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 Processor -- start 2023-07-21T22:15:45.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763415bf0 0x55b763421160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763416560 0x55b76347d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763425b30 con 0x55b763414dd0 2023-07-21T22:15:45.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763425ca0 con 0x55b763415bf0 2023-07-21T22:15:45.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76347da70 con 0x55b763416560 2023-07-21T22:15:45.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f192023bb80 10 client.0 _ll_register_callbacks cb 0x7f1900005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f1911e52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43066/0 (socket says 172.21.15.93:43066) 2023-07-21T22:15:45.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f1911e52700 1 -- 172.21.15.93:0/4008511409 learned_addr learned my addr 172.21.15.93:0/4008511409 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f1911651700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763415bf0 0x55b763421160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.604+0000 7f1912653700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763416560 0x55b76347d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 -- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763416560 msgr2=0x55b76347d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763416560 0x55b76347d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 -- 172.21.15.93:0/4008511409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763415bf0 msgr2=0x55b763421160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763415bf0 0x55b763421160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 -- 172.21.15.93:0/4008511409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b763178c10 con 0x55b763414dd0 2023-07-21T22:15:45.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f1911e52700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fe0 tx=0x7f1908026550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf531f9179eb3c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908005560 con 0x55b763414dd0 2023-07-21T22:15:45.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1908017040 con 0x55b763414dd0 2023-07-21T22:15:45.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1908015070 con 0x55b763414dd0 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19080039c0 con 0x55b763414dd0 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 10 client.15045 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b76318e260 con 0x55b763414dd0 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190804f020 con 0x55b763414dd0 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f190803d070 con 0x55b763414dd0 2023-07-21T22:15:45.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 20 client.15045 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b763482240 0x55b76347ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 4 client.15045 mds_command: new command op to 24460 tid=0[{"prefix": "status"}] 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.605+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b76331d000 con 0x55b763482240 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f1911651700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b763482240 0x55b76347ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f1911651700 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b763482240 0x55b76347ec30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ba7f4a390bd11d7 server_cookie=ba380738ffe70dae in_seq=0 out_seq=0 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f18f57fa700 10 client.15045 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f18f57fa700 1 -- 172.21.15.93:0/4008511409 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b76331d000 con 0x55b763482240 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f18f57fa700 10 client.15045 handle_command_reply: tid=0 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 client.15045 shutdown 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 msgr2=0x55b763420c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1908010fe0 tx=0x7f1908026550 comp rx=0 tx=0).stop 2023-07-21T22:15:45.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 shutdown_connections 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55b763482240 0x55b76347ec30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b763416560 0x55b76347d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b763414dd0 0x55b763420c20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f18f57fa700 0 client.15045 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 --2- 172.21.15.93:0/4008511409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b763415bf0 0x55b763421160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.606+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 >> 172.21.15.93:0/4008511409 conn(0x55b763405ea0 msgr2=0x55b76341d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 shutdown_connections 2023-07-21T22:15:45.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f192023bb80 1 -- 172.21.15.93:0/4008511409 wait complete. 2023-07-21T22:15:45.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f192023bb80 20 client.15045 trim_cache size 0 max 16384 2023-07-21T22:15:45.670 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:15:45.670 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:15:45.670 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24460, 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 50.336610329999999, 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 82, 2023-07-21T22:15:45.671 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:15:45.672 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 71.560234999000002 2023-07-21T22:15:45.672 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 msgr2=0x7f18f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 0x7f18f80a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1908007c60 tx=0x7f19080073d0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 msgr2=0x7f190c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1900000c00 tx=0x7f190000f710 comp rx=0 tx=0).stop 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 shutdown_connections 2023-07-21T22:15:45.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18f80a11e0 0x7f18f80a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f190c100640 0x7f190c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f190c0fea00 0x7f190c0693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 --2- 172.21.15.93:0/3239536001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f190c0ffc90 0x7f190c19a510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 >> 172.21.15.93:0/3239536001 conn(0x7f190c0fa020 msgr2=0x7f190c0fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 shutdown_connections 2023-07-21T22:15:45.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.607+0000 7f19140b6700 1 -- 172.21.15.93:0/3239536001 wait complete. 2023-07-21T22:15:45.675 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:15:45.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 -- 172.21.15.93:0/1205856514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 msgr2=0x7f51641003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 --2- 172.21.15.93:0/1205856514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f51641003b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5158009990 tx=0x7f5158009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 -- 172.21.15.93:0/1205856514 shutdown_connections 2023-07-21T22:15:45.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 --2- 172.21.15.93:0/1205856514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640ff660 0x7f5164106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 --2- 172.21.15.93:0/1205856514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f51641003b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 --2- 172.21.15.93:0/1205856514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640fecd0 0x7f51640ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 -- 172.21.15.93:0/1205856514 >> 172.21.15.93:0/1205856514 conn(0x7f51640f97b0 msgr2=0x7f51640fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 -- 172.21.15.93:0/1205856514 shutdown_connections 2023-07-21T22:15:45.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 -- 172.21.15.93:0/1205856514 wait complete. 2023-07-21T22:15:45.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.753+0000 7f51694f1700 1 Processor -- start 2023-07-21T22:15:45.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 -- start start 2023-07-21T22:15:45.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 0x7f51640fe750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640ff660 0x7f516419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419d350 con 0x7f51640fffa0 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419d4c0 con 0x7f51640fecd0 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516419d680 con 0x7f51640ff660 2023-07-21T22:15:45.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43084/0 (socket says 172.21.15.93:43084) 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 -- 172.21.15.93:0/2107758856 learned_addr learned my addr 172.21.15.93:0/2107758856 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f5162ffd700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 0x7f51640fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51627fc700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640ff660 0x7f516419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 -- 172.21.15.93:0/2107758856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640ff660 msgr2=0x7f516419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640ff660 0x7f516419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 -- 172.21.15.93:0/2107758856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 msgr2=0x7f51640fe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 0x7f51640fe750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 -- 172.21.15.93:0/2107758856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580095f0 con 0x7f51640fffa0 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f5162ffd700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 0x7f51640fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51637fe700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f515c007ee0 tx=0x7f515c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f514bfff700 1 -- 172.21.15.93:0/2107758856 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c017070 con 0x7f51640fffa0 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516419d960 con 0x7f51640fffa0 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f514bfff700 1 -- 172.21.15.93:0/2107758856 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515c00f040 con 0x7f51640fffa0 2023-07-21T22:15:45.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.754+0000 7f514bfff700 1 -- 172.21.15.93:0/2107758856 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c005740 con 0x7f51640fffa0 2023-07-21T22:15:45.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.755+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516419de20 con 0x7f51640fffa0 2023-07-21T22:15:45.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.757+0000 7f514bfff700 1 -- 172.21.15.93:0/2107758856 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f515c0058a0 con 0x7f51640fffa0 2023-07-21T22:15:45.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.757+0000 7f514bfff700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 0x7f514c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.808 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.757+0000 7f5162ffd700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 0x7f514c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.757+0000 7f514bfff700 1 -- 172.21.15.93:0/2107758856 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f515c0cc2b0 con 0x7f51640fffa0 2023-07-21T22:15:45.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5162ffd700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 0x7f514c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5158006010 tx=0x7f515800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 Processor -- start 2023-07-21T22:15:45.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 -- start start 2023-07-21T22:15:45.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2febfe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2febfc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feafdc60 con 0x55f2febfcdd0 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feb509f0 con 0x55f2febfe560 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f5175676b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feb49e80 con 0x55f2febfdbf0 2023-07-21T22:15:45.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43092/0 (socket says 172.21.15.93:43092) 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f51637fe700 1 -- 172.21.15.93:0/3330545581 learned_addr learned my addr 172.21.15.93:0/3330545581 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f51627fc700 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2febfc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.758+0000 7f51637fe700 1 -- 172.21.15.93:0/3330545581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 msgr2=0x55f2febfe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51637fe700 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2febfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51637fe700 1 -- 172.21.15.93:0/3330545581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 msgr2=0x55f2febfc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51637fe700 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2febfc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51637fe700 1 -- 172.21.15.93:0/3330545581 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2fe977d90 con 0x55f2febfcdd0 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51627fc700 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2febfc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51637fe700 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f515c096080 tx=0x7f515c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75db3c06c118b029 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51497fa700 1 -- 172.21.15.93:0/3330545581 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0b90d0 con 0x55f2febfcdd0 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51497fa700 1 -- 172.21.15.93:0/3330545581 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515c0a1a60 con 0x55f2febfcdd0 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f51497fa700 1 -- 172.21.15.93:0/3330545581 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c099870 con 0x55f2febfcdd0 2023-07-21T22:15:45.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f5175676b80 1 -- 172.21.15.93:0/3330545581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 msgr2=0x55f2febfd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.759+0000 7f5175676b80 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f515c096080 tx=0x7f515c004b50 comp rx=0 tx=0).stop 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- 172.21.15.93:0/3330545581 shutdown_connections 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2febfd250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2febfe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- 172.21.15.93:0/3330545581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2febfc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- 172.21.15.93:0/3330545581 >> 172.21.15.93:0/3330545581 conn(0x55f2febedea0 msgr2=0x55f2febf3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- 172.21.15.93:0/3330545581 shutdown_connections 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- 172.21.15.93:0/3330545581 wait complete. 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- start start 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 Processor -- start 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec06a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfe560 0x55f2fec0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feae9a70 con 0x55f2febfcdd0 2023-07-21T22:15:45.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feae9770 con 0x55f2febfdbf0 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe9cd8d0 con 0x55f2febfe560 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5175676b80 10 client.0 _ll_register_callbacks cb 0x55f2feae7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5162ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5162ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43094/0 (socket says 172.21.15.93:43094) 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f5162ffd700 1 -- 172.21.15.93:0/1180972500 learned_addr learned my addr 172.21.15.93:0/1180972500 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.760+0000 7f51627fc700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec06a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 -- 172.21.15.93:0/1180972500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfe560 msgr2=0x55f2fec0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfe560 0x55f2fec0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 -- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 msgr2=0x55f2fec06a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec06a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 -- 172.21.15.93:0/1180972500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516419de20 con 0x55f2febfcdd0 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f51627fc700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec06a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5158005f50 tx=0x7f5158009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f9af2f0ad413ea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515802b070 con 0x55f2febfcdd0 2023-07-21T22:15:45.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5158003890 con 0x55f2febfcdd0 2023-07-21T22:15:45.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2feaa34a0 con 0x55f2febfcdd0 2023-07-21T22:15:45.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158003a50 con 0x55f2febfcdd0 2023-07-21T22:15:45.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 10 client.15057 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f2fe960c10 con 0x55f2febfcdd0 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2feaa34a0 con 0x55f2febfcdd0 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f51580244a0 con 0x55f2febfcdd0 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 10 client.15057 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 20 client.15057 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec11820 0x55f2fec10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 4 client.15057 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.761+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2feb42e60 con 0x55f2fec11820 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.762+0000 7f51627fc700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec11820 0x55f2fec10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.762+0000 7f5146ff5700 10 client.15057 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.762+0000 7f51627fc700 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec11820 0x55f2fec10bf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12d976813a80daec server_cookie=b5b5f012207362c8 in_seq=0 out_seq=0 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5146ff5700 1 -- 172.21.15.93:0/1180972500 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f2feb42e60 con 0x55f2fec11820 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5146ff5700 10 client.15057 handle_command_reply: tid=0 2023-07-21T22:15:45.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 client.15057 shutdown 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 msgr2=0x55f2fec06520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5158005f50 tx=0x7f5158009310 comp rx=0 tx=0).stop 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 shutdown_connections 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec11820 0x55f2fec10bf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfe560 0x55f2fec0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5146ff5700 0 client.15057 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:15:45.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec06520 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 --2- 172.21.15.93:0/1180972500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec06a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 >> 172.21.15.93:0/1180972500 conn(0x55f2febedea0 msgr2=0x55f2febf29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 shutdown_connections 2023-07-21T22:15:45.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 1 -- 172.21.15.93:0/1180972500 wait complete. 2023-07-21T22:15:45.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.763+0000 7f5175676b80 20 client.15057 trim_cache size 0 max 16384 2023-07-21T22:15:45.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 Processor -- start 2023-07-21T22:15:45.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 -- start start 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfcdd0 0x55f2fec1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec0da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe9d9e20 con 0x55f2febfe560 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2feb084b0 con 0x55f2febfdbf0 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5175676b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fec059d0 con 0x55f2febfcdd0 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43106/0 (socket says 172.21.15.93:43106) 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 -- 172.21.15.93:0/1691142027 learned_addr learned my addr 172.21.15.93:0/1691142027 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 -- 172.21.15.93:0/1691142027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfcdd0 msgr2=0x55f2fec1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfcdd0 0x55f2fec1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfcdd0 0x55f2fec1a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 -- 172.21.15.93:0/1691142027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 msgr2=0x55f2fec0da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51627fc700 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec0da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec0da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 -- 172.21.15.93:0/1691142027 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2febfc9e0 con 0x55f2febfe560 2023-07-21T22:15:45.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f51637fe700 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f515c0997d0 tx=0x7f515c0994b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea00b846029db85 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5149ffb700 1 -- 172.21.15.93:0/1691142027 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0b90d0 con 0x55f2febfe560 2023-07-21T22:15:45.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5149ffb700 1 -- 172.21.15.93:0/1691142027 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515c0b6030 con 0x55f2febfe560 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.781+0000 7f5149ffb700 1 -- 172.21.15.93:0/1691142027 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c098d60 con 0x55f2febfe560 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- 172.21.15.93:0/1691142027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 msgr2=0x55f2fec051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f515c0997d0 tx=0x7f515c0994b0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- 172.21.15.93:0/1691142027 shutdown_connections 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfe560 0x55f2fec051a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfcdd0 0x55f2fec1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 --2- 172.21.15.93:0/1691142027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfdbf0 0x55f2fec0da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- 172.21.15.93:0/1691142027 >> 172.21.15.93:0/1691142027 conn(0x55f2febedea0 msgr2=0x55f2febf32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- 172.21.15.93:0/1691142027 shutdown_connections 2023-07-21T22:15:45.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- 172.21.15.93:0/1691142027 wait complete. 2023-07-21T22:15:45.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 -- start start 2023-07-21T22:15:45.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.782+0000 7f5175676b80 1 Processor -- start 2023-07-21T22:15:45.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2fec0d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2fec656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fec65d20 con 0x55f2febfcdd0 2023-07-21T22:15:45.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fec65e90 con 0x55f2febfe560 2023-07-21T22:15:45.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fec66050 con 0x55f2febfdbf0 2023-07-21T22:15:45.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 10 client.0 _ll_register_callbacks cb 0x7f515800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43108/0 (socket says 172.21.15.93:43108) 2023-07-21T22:15:45.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 -- 172.21.15.93:0/1310871428 learned_addr learned my addr 172.21.15.93:0/1310871428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f51627fc700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2fec0d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f51637fe700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2fec656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 -- 172.21.15.93:0/1310871428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 msgr2=0x55f2fec0d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2fec0d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 -- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 msgr2=0x55f2fec656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2fec656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 -- 172.21.15.93:0/1310871428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2fe977d90 con 0x55f2febfcdd0 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f51637fe700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2fec656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5162ffd700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5158024800 tx=0x7f5158024a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd76b62883b3fd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5146ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515802b040 con 0x55f2febfcdd0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5158003a90 con 0x55f2febfcdd0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.783+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f515c0a1a60 con 0x55f2febfcdd0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5158024620 con 0x55f2febfcdd0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 10 client.15063 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f2fe99e200 con 0x55f2febfcdd0 2023-07-21T22:15:45.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f515c0a1a60 con 0x55f2febfcdd0 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f515804a680 con 0x55f2febfcdd0 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 10 client.15063 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 20 client.15063 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec6a9d0 0x55f2fec67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 4 client.15063 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-07-21T22:15:45.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f2feb05000 con 0x55f2fec6a9d0 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.784+0000 7f51627fc700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec6a9d0 0x55f2fec67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5146ff5700 10 client.15063 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f51627fc700 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec6a9d0 0x55f2fec67320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9403fd1d53afa666 server_cookie=ed8e7e2e15c5c91b in_seq=0 out_seq=0 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5146ff5700 1 -- 172.21.15.93:0/1310871428 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f2feb05000 con 0x55f2fec6a9d0 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5146ff5700 10 client.15063 handle_command_reply: tid=0 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 client.15063 shutdown 2023-07-21T22:15:45.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 msgr2=0x55f2fec03860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5158024800 tx=0x7f5158024a30 comp rx=0 tx=0).stop 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 shutdown_connections 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f2fec6a9d0 0x55f2fec67320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2febfe560 0x55f2fec656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f2febfcdd0 0x55f2fec03860 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 --2- 172.21.15.93:0/1310871428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f2febfdbf0 0x55f2fec0d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5146ff5700 0 client.15063 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:15:45.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 >> 172.21.15.93:0/1310871428 conn(0x55f2febedea0 msgr2=0x55f2fec051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 shutdown_connections 2023-07-21T22:15:45.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 1 -- 172.21.15.93:0/1310871428 wait complete. 2023-07-21T22:15:45.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.785+0000 7f5175676b80 20 client.15063 trim_cache size 0 max 16384 2023-07-21T22:15:45.844 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24482, 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:15:45.845 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 51.527135229999999, 2023-07-21T22:15:45.846 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:15:45.846 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 82, 2023-07-21T22:15:45.846 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:15:45.846 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 60.867956448999998 2023-07-21T22:15:45.846 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:15:45.847 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:45 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1943140284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.847 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:45 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2401517621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.848 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:45 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/33708206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:15:45.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 msgr2=0x7f514c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 0x7f514c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5158006010 tx=0x7f515800b540 comp rx=0 tx=0).stop 2023-07-21T22:15:45.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 msgr2=0x7f516419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f515c007ee0 tx=0x7f515c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 shutdown_connections 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51640fffa0 0x7f516419cb80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f514c0a11e0 0x7f514c0a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51640fecd0 0x7f51640fe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 --2- 172.21.15.93:0/2107758856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51640ff660 0x7f516419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 >> 172.21.15.93:0/2107758856 conn(0x7f51640f97b0 msgr2=0x7f51640fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 shutdown_connections 2023-07-21T22:15:45.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.786+0000 7f51694f1700 1 -- 172.21.15.93:0/2107758856 wait complete. 2023-07-21T22:15:45.850 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:15:45.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 -- 172.21.15.93:0/3719659817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 msgr2=0x7f84040ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 --2- 172.21.15.93:0/3719659817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f84040ff120 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009990 tx=0x7f83f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 -- 172.21.15.93:0/3719659817 shutdown_connections 2023-07-21T22:15:45.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 --2- 172.21.15.93:0/3719659817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84040ff660 0x7f8404106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 --2- 172.21.15.93:0/3719659817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84040fffa0 0x7f84041003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 --2- 172.21.15.93:0/3719659817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f84040ff120 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 -- 172.21.15.93:0/3719659817 >> 172.21.15.93:0/3719659817 conn(0x7f84040f97b0 msgr2=0x7f84040fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 -- 172.21.15.93:0/3719659817 shutdown_connections 2023-07-21T22:15:45.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.927+0000 7f840976f700 1 -- 172.21.15.93:0/3719659817 wait complete. 2023-07-21T22:15:45.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 Processor -- start 2023-07-21T22:15:45.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 -- start start 2023-07-21T22:15:45.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84040ff660 0x7f8404197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84040fffa0 0x7f840419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840419f580 con 0x7f84040fecd0 2023-07-21T22:15:45.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840419f6c0 con 0x7f84040ff660 2023-07-21T22:15:45.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f840976f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840419f850 con 0x7f84040fffa0 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43122/0 (socket says 172.21.15.93:43122) 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 -- 172.21.15.93:0/2333829951 learned_addr learned my addr 172.21.15.93:0/2333829951 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 -- 172.21.15.93:0/2333829951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84040fffa0 msgr2=0x7f840419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84040fffa0 0x7f840419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 -- 172.21.15.93:0/2333829951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84040ff660 msgr2=0x7f8404197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84040ff660 0x7f8404197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.928+0000 7f8402ffd700 1 -- 172.21.15.93:0/2333829951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40095f0 con 0x7f84040fecd0 2023-07-21T22:15:45.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f8402ffd700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d900 tx=0x7f83f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f83ebfff700 1 -- 172.21.15.93:0/2333829951 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8004020 con 0x7f84040fecd0 2023-07-21T22:15:45.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840419fb30 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f83ebfff700 1 -- 172.21.15.93:0/2333829951 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83f8009730 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f83ebfff700 1 -- 172.21.15.93:0/2333829951 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8003a30 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.929+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84041a0090 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.931+0000 7f83ebfff700 1 -- 172.21.15.93:0/2333829951 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f83f80098a0 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.931+0000 7f83ebfff700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 0x7f83ec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.931+0000 7f84027fc700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 0x7f83ec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f83ebfff700 1 -- 172.21.15.93:0/2333829951 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83f80cc960 con 0x7f84040fecd0 2023-07-21T22:15:45.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84027fc700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 0x7f83ec0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83f400b680 tx=0x7f83f4009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84158f4b80 1 Processor -- start 2023-07-21T22:15:45.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84158f4b80 1 -- start start 2023-07-21T22:15:45.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c53f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.932+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c53f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f84158f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c52fec60 con 0x5571c53fa7e0 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f84158f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c53519f0 con 0x5571c53fb150 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f84158f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c534ae80 con 0x5571c53f99c0 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43130/0 (socket says 172.21.15.93:43130) 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 -- 172.21.15.93:0/3016063203 learned_addr learned my addr 172.21.15.93:0/3016063203 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 -- 172.21.15.93:0/3016063203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 msgr2=0x5571c53f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c53f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 -- 172.21.15.93:0/3016063203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 msgr2=0x5571c53f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c53f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 -- 172.21.15.93:0/3016063203 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c5178d90 con 0x5571c53fa7e0 2023-07-21T22:15:45.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f8402ffd700 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83f80a1be0 tx=0x7f83f8099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e65165403a6d7b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f83e97fa700 1 -- 172.21.15.93:0/3016063203 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f80a2110 con 0x5571c53fa7e0 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f83e97fa700 1 -- 172.21.15.93:0/3016063203 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83f8003da0 con 0x5571c53fa7e0 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.933+0000 7f83e97fa700 1 -- 172.21.15.93:0/3016063203 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f80b2210 con 0x5571c53fa7e0 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- 172.21.15.93:0/3016063203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 msgr2=0x5571c53fabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83f80a1be0 tx=0x7f83f8099d40 comp rx=0 tx=0).stop 2023-07-21T22:15:45.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- 172.21.15.93:0/3016063203 shutdown_connections 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c53f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c53fabf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 --2- 172.21.15.93:0/3016063203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c53f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- 172.21.15.93:0/3016063203 >> 172.21.15.93:0/3016063203 conn(0x5571c53eeea0 msgr2=0x5571c53f4310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- 172.21.15.93:0/3016063203 shutdown_connections 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- 172.21.15.93:0/3016063203 wait complete. 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 -- start start 2023-07-21T22:15:45.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 Processor -- start 2023-07-21T22:15:45.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.934+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5403060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c540a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c52eaa70 con 0x5571c53fa7e0 2023-07-21T22:15:45.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c52ea770 con 0x5571c53fb150 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c51ce8d0 con 0x5571c53f99c0 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 10 client.0 _ll_register_callbacks cb 0x5571c52e8710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43138/0 (socket says 172.21.15.93:43138) 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 -- 172.21.15.93:0/4132846928 learned_addr learned my addr 172.21.15.93:0/4132846928 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84037fe700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c540a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 -- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 msgr2=0x5571c5403060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f8402ffd700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5403060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5403060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 -- 172.21.15.93:0/4132846928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 msgr2=0x5571c540a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c540a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 -- 172.21.15.93:0/4132846928 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84041a0090 con 0x5571c53fa7e0 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84037fe700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c540a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84027fc700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005780 tx=0x7f83f4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224031693fd7a9c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:45.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f83e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4029070 con 0x5571c53fa7e0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83f4004ce0 con 0x5571c53fa7e0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571c52a44a0 con 0x5571c53fa7e0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83f401d070 con 0x5571c53fa7e0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 10 client.15075 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5571c5161c10 con 0x5571c53fa7e0 2023-07-21T22:15:45.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.935+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571c52a44a0 con 0x5571c53fa7e0 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f83f400f4f0 con 0x5571c53fa7e0 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 20 client.15075 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c540e390 0x5571c540d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 4 client.15075 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:15:45.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571c5343e60 con 0x5571c540e390 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f8402ffd700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c540e390 0x5571c540d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f83e6ff5700 10 client.15075 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.936+0000 7f8402ffd700 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c540e390 0x5571c540d730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7efb9fa17a19c2 server_cookie=e39f9ac3f858c447 in_seq=0 out_seq=0 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f83e6ff5700 1 -- 172.21.15.93:0/4132846928 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571c5343e60 con 0x5571c540e390 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f83e6ff5700 10 client.15075 handle_command_reply: tid=0 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 client.15075 shutdown 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 msgr2=0x5571c54035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:45.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005780 tx=0x7f83f4004620 comp rx=0 tx=0).stop 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 shutdown_connections 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c540e390 0x5571c540d730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fb150 0x5571c540a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f83e6ff5700 0 client.15075 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5403060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 --2- 172.21.15.93:0/4132846928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fa7e0 0x5571c54035a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 >> 172.21.15.93:0/4132846928 conn(0x5571c53eeea0 msgr2=0x5571c53f39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:45.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 shutdown_connections 2023-07-21T22:15:45.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 1 -- 172.21.15.93:0/4132846928 wait complete. 2023-07-21T22:15:45.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.937+0000 7f84158f4b80 20 client.15075 trim_cache size 0 max 16384 2023-07-21T22:15:46.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 Processor -- start 2023-07-21T22:15:46.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 -- start start 2023-07-21T22:15:46.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53f99c0 0x5571c540ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53fa7e0 0x5571c540a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5319790 con 0x5571c53fb150 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c51dade0 con 0x5571c53f99c0 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84158f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5309480 con 0x5571c53fa7e0 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43148/0 (socket says 172.21.15.93:43148) 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 -- 172.21.15.93:0/697837773 learned_addr learned my addr 172.21.15.93:0/697837773 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f8402ffd700 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53f99c0 0x5571c540ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 -- 172.21.15.93:0/697837773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53fa7e0 msgr2=0x5571c540a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53fa7e0 0x5571c540a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 -- 172.21.15.93:0/697837773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53f99c0 msgr2=0x5571c540ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53f99c0 0x5571c540ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 -- 172.21.15.93:0/697837773 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c519f200 con 0x5571c53fb150 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.955+0000 7f84037fe700 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83fc007790 tx=0x7f83fc007aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7696cc8ef2461039 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f83e9ffb700 1 -- 172.21.15.93:0/697837773 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc003a20 con 0x5571c53fb150 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f83e9ffb700 1 -- 172.21.15.93:0/697837773 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83fc00f070 con 0x5571c53fb150 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f83e9ffb700 1 -- 172.21.15.93:0/697837773 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0156d0 con 0x5571c53fb150 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- 172.21.15.93:0/697837773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 msgr2=0x5571c5400ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83fc007790 tx=0x7f83fc007aa0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- 172.21.15.93:0/697837773 shutdown_connections 2023-07-21T22:15:46.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5400ed0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53f99c0 0x5571c540ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 --2- 172.21.15.93:0/697837773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53fa7e0 0x5571c540a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- 172.21.15.93:0/697837773 >> 172.21.15.93:0/697837773 conn(0x5571c53eeea0 msgr2=0x5571c53f39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- 172.21.15.93:0/697837773 shutdown_connections 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- 172.21.15.93:0/697837773 wait complete. 2023-07-21T22:15:46.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.956+0000 7f84158f4b80 1 -- start start 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 Processor -- start 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5405c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 0x5571c5406180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5409650 con 0x5571c53fb150 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c54097c0 con 0x5571c53fa7e0 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c5409980 con 0x5571c53f99c0 2023-07-21T22:15:46.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84158f4b80 10 client.0 _ll_register_callbacks cb 0x7f83f4005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43154/0 (socket says 172.21.15.93:43154) 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 -- 172.21.15.93:0/87431561 learned_addr learned my addr 172.21.15.93:0/87431561 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f8402ffd700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5405c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84027fc700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 0x5571c5406180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 -- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 msgr2=0x5571c5405c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:46.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5405c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 -- 172.21.15.93:0/87431561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 msgr2=0x5571c5406180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 0x5571c5406180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 -- 172.21.15.93:0/87431561 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc0073e0 con 0x5571c53fb150 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f8402ffd700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5405c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84027fc700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 0x5571c5406180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:46.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f84037fe700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83fc000f80 tx=0x7f83fc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6791a56177e56cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.957+0000 7f83e6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004340 con 0x5571c53fb150 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83fc015470 con 0x5571c53fb150 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83f8003da0 con 0x5571c53fb150 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83fc015630 con 0x5571c53fb150 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 10 client.15081 fetch_fsmap learned FSMap version 26 2023-07-21T22:15:46.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5571c5177260 con 0x5571c53fb150 2023-07-21T22:15:46.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83f8003da0 con 0x5571c53fb150 2023-07-21T22:15:46.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f83fc027020 con 0x5571c53fb150 2023-07-21T22:15:46.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:15:46.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 10 client.15081 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:15:46.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 20 client.15081 populate_metadata read hostname 'smithi093' 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c5467b80 0x5571c5464620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 4 client.15081 mds_command: new command op to 24484 tid=0[{"prefix": "status"}] 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5571c5306000 con 0x5571c5467b80 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.958+0000 7f8402ffd700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c5467b80 0x5571c5464620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f83e6ff5700 10 client.15081 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f8402ffd700 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c5467b80 0x5571c5464620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16adda2704e35d30 server_cookie=10526c74e5538b37 in_seq=0 out_seq=0 2023-07-21T22:15:46.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f83e6ff5700 1 -- 172.21.15.93:0/87431561 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5571c5306000 con 0x5571c5467b80 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f83e6ff5700 10 client.15081 handle_command_reply: tid=0 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 client.15081 shutdown 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 msgr2=0x5571c5409110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83fc000f80 tx=0x7f83fc003ce0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 shutdown_connections 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5571c5467b80 0x5571c5464620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5571c53fb150 0x5571c5409110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f83e6ff5700 0 client.15081 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571c53f99c0 0x5571c5405c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 --2- 172.21.15.93:0/87431561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571c53fa7e0 0x5571c5406180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 >> 172.21.15.93:0/87431561 conn(0x5571c53eeea0 msgr2=0x5571c540ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 shutdown_connections 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 1 -- 172.21.15.93:0/87431561 wait complete. 2023-07-21T22:15:46.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.959+0000 7f84158f4b80 20 client.15081 trim_cache size 0 max 16384 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24484, 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:15:46.017 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 54.716135821999998, 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 82, 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 67.414051401999998 2023-07-21T22:15:46.018 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:15:46.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 msgr2=0x7f83ec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 0x7f83ec0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83f400b680 tx=0x7f83f4009510 comp rx=0 tx=0).stop 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 msgr2=0x7f8404197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d900 tx=0x7f83f800dcc0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 shutdown_connections 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84040fffa0 0x7f840419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84040fecd0 0x7f8404197a00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f83ec0a11e0 0x7f83ec0a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 --2- 172.21.15.93:0/2333829951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84040ff660 0x7f8404197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:46.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 >> 172.21.15.93:0/2333829951 conn(0x7f84040f97b0 msgr2=0x7f84040fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:46.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 shutdown_connections 2023-07-21T22:15:46.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:45.960+0000 7f840976f700 1 -- 172.21.15.93:0/2333829951 wait complete. 2023-07-21T22:15:46.022 INFO:tasks.cephadm:Setup complete, yielding 2023-07-21T22:15:46.022 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:15:46.032 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:15:46.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-21T22:15:46.583 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:46.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:46 smithi134 ceph-mon[97873]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:46.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:46 smithi093 ceph-mon[96579]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:46.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:46 smithi194 ceph-mon[98288]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:47.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 -- 172.21.15.93:0/720217616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34480ffb90 msgr2=0x7f34480fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 --2- 172.21.15.93:0/720217616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34480ffb90 0x7f34480fffa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f343c009990 tx=0x7f343c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 -- 172.21.15.93:0/720217616 shutdown_connections 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 --2- 172.21.15.93:0/720217616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3448101720 0x7f3448107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 --2- 172.21.15.93:0/720217616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34480ffb90 0x7f34480fffa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 --2- 172.21.15.93:0/720217616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448100d90 0x7f34481011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 -- 172.21.15.93:0/720217616 >> 172.21.15.93:0/720217616 conn(0x7f34480fb1a0 msgr2=0x7f34480fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 -- 172.21.15.93:0/720217616 shutdown_connections 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.717+0000 7f344f13d700 1 -- 172.21.15.93:0/720217616 wait complete. 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 Processor -- start 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 -- start start 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 0x7f34480742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:47.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448101720 0x7f3448072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480735c0 con 0x7f3448100d90 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3448073730 con 0x7f34480ffb90 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344f13d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344819f980 con 0x7f3448101720 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43166/0 (socket says 172.21.15.93:43166) 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 -- 172.21.15.93:0/1071554533 learned_addr learned my addr 172.21.15.93:0/1071554533 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:47.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344d6da700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448101720 0x7f3448072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344ced9700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 0x7f34480742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 -- 172.21.15.93:0/1071554533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448101720 msgr2=0x7f3448072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448101720 0x7f3448072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 -- 172.21.15.93:0/1071554533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 msgr2=0x7f34480742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 0x7f34480742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f3447fff700 1 -- 172.21.15.93:0/1071554533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0095f0 con 0x7f3448100d90 2023-07-21T22:15:47.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.718+0000 7f344ced9700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 0x7f34480742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f3447fff700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f343400b530 tx=0x7f343400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434011830 con 0x7f3448100d90 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3434011e50 con 0x7f3448100d90 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344819fc60 con 0x7f3448100d90 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400f2e0 con 0x7f3448100d90 2023-07-21T22:15:47.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.719+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34481a01b0 con 0x7f3448100d90 2023-07-21T22:15:47.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.720+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34481070e0 con 0x7f3448100d90 2023-07-21T22:15:47.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.721+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3434010580 con 0x7f3448100d90 2023-07-21T22:15:47.768 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.721+0000 7f3445ffb700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 0x7f34380a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:47.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.721+0000 7f344ced9700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 0x7f34380a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:47.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.721+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f34340cc280 con 0x7f3448100d90 2023-07-21T22:15:47.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.722+0000 7f344ced9700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 0x7f34380a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f343c00b5c0 tx=0x7f343c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:47.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.726+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3434095b30 con 0x7f3448100d90 2023-07-21T22:15:47.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.925+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3448060950 con 0x7f34380a1260 2023-07-21T22:15:47.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.926+0000 7f3445ffb700 1 -- 172.21.15.93:0/1071554533 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3448060950 con 0x7f34380a1260 2023-07-21T22:15:47.969 INFO:teuthology.orchestra.run.smithi093.stdout:Backend: cephadm 2023-07-21T22:15:47.970 INFO:teuthology.orchestra.run.smithi093.stdout:Available: Yes 2023-07-21T22:15:47.970 INFO:teuthology.orchestra.run.smithi093.stdout:Paused: No 2023-07-21T22:15:47.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 msgr2=0x7f34380a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:47.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 0x7f34380a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f343c00b5c0 tx=0x7f343c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 msgr2=0x7f3448072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f343400b530 tx=0x7f343400b8f0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 shutdown_connections 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3448101720 0x7f3448072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f34380a1260 0x7f34380a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34480ffb90 0x7f34480742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 --2- 172.21.15.93:0/1071554533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3448100d90 0x7f3448072900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:47.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 >> 172.21.15.93:0/1071554533 conn(0x7f34480fb1a0 msgr2=0x7f3448104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:47.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 shutdown_connections 2023-07-21T22:15:47.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:47.928+0000 7f344f13d700 1 -- 172.21.15.93:0/1071554533 wait complete. 2023-07-21T22:15:48.327 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:48 smithi093 ceph-mon[96579]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:15:48.328 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-21T22:15:48.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:48 smithi194 ceph-mon[98288]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:15:48.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:48 smithi134 ceph-mon[97873]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:15:48.879 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:49.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:49 smithi194 ceph-mon[98288]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:49.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:49 smithi134 ceph-mon[97873]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:49.382 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:49 smithi093 ceph-mon[96579]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:50.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 -- 172.21.15.93:0/1415228317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80ffc80 msgr2=0x7f27a81000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:50.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 --2- 172.21.15.93:0/1415228317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80ffc80 0x7f27a81000d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2794009990 tx=0x7f2794009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 -- 172.21.15.93:0/1415228317 shutdown_connections 2023-07-21T22:15:50.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 --2- 172.21.15.93:0/1415228317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a8100610 0x7f27a8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 --2- 172.21.15.93:0/1415228317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a80fea80 0x7f27a80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 --2- 172.21.15.93:0/1415228317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80ffc80 0x7f27a81000d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 -- 172.21.15.93:0/1415228317 >> 172.21.15.93:0/1415228317 conn(0x7f27a80fa0b0 msgr2=0x7f27a80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 -- 172.21.15.93:0/1415228317 shutdown_connections 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 -- 172.21.15.93:0/1415228317 wait complete. 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.043+0000 7f27aec87700 1 Processor -- start 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 -- start start 2023-07-21T22:15:50.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a80ffc80 0x7f27a8195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 0x7f27a819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a819d550 con 0x7f27a80fea80 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a819d690 con 0x7f27a8100610 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a819d820 con 0x7f27a80ffc80 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:50.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43178/0 (socket says 172.21.15.93:43178) 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 -- 172.21.15.93:0/2436666115 learned_addr learned my addr 172.21.15.93:0/2436666115 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27a7fff700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a80ffc80 0x7f27a8195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27ad224700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 0x7f27a819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 -- 172.21.15.93:0/2436666115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a80ffc80 msgr2=0x7f27a8195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a80ffc80 0x7f27a8195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 -- 172.21.15.93:0/2436666115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 msgr2=0x7f27a819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:50.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 0x7f27a819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 -- 172.21.15.93:0/2436666115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940095f0 con 0x7f27a80fea80 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27ad224700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 0x7f27a819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aca23700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279c00b530 tx=0x7f279c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c011830 con 0x7f27a80fea80 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f279c011e50 con 0x7f27a80fea80 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a819db00 con 0x7f27a80fea80 2023-07-21T22:15:50.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.044+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00f2e0 con 0x7f27a80fea80 2023-07-21T22:15:50.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.045+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a819e000 con 0x7f27a80fea80 2023-07-21T22:15:50.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.046+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a810a3b0 con 0x7f27a80fea80 2023-07-21T22:15:50.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.046+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f279c010580 con 0x7f27a80fea80 2023-07-21T22:15:50.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.047+0000 7f27a5ffb700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 0x7f27980a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:50.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.047+0000 7f27a7fff700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 0x7f27980a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:50.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.047+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f279c0cc320 con 0x7f27a80fea80 2023-07-21T22:15:50.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.047+0000 7f27a7fff700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 0x7f27980a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f279400b5c0 tx=0x7f2794005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:50.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.051+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f279c095ad0 con 0x7f27a80fea80 2023-07-21T22:15:50.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.246+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f27a80687a0 con 0x7f27980a1260 2023-07-21T22:15:50.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.258+0000 7f27a5ffb700 1 -- 172.21.15.93:0/2436666115 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f27a80687a0 con 0x7f27980a1260 2023-07-21T22:15:50.302 INFO:teuthology.orchestra.run.smithi093.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-21T22:15:50.302 INFO:teuthology.orchestra.run.smithi093.stdout:mds.a smithi093 running (67s) 58s ago 67s 13.3M - 18.0.0-4735-gbffe049d fa7da5dee971 eec88a370793 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.b smithi134 running (65s) 58s ago 65s 15.6M - 18.0.0-4735-gbffe049d fa7da5dee971 707b57e11fc9 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.c smithi194 running (76s) 58s ago 76s 15.7M - 18.0.0-4735-gbffe049d fa7da5dee971 701e8086c856 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.d smithi093 running (83s) 58s ago 82s 14.0M - 18.0.0-4735-gbffe049d fa7da5dee971 3c500534fd8b 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.e smithi134 running (81s) 58s ago 80s 15.7M - 18.0.0-4735-gbffe049d fa7da5dee971 1982af2ee81e 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.f smithi194 running (78s) 58s ago 78s 14.3M - 18.0.0-4735-gbffe049d fa7da5dee971 dc59b8726a11 2023-07-21T22:15:50.303 INFO:teuthology.orchestra.run.smithi093.stdout:mds.g smithi093 running (69s) 58s ago 69s 16.8M - 18.0.0-4735-gbffe049d fa7da5dee971 4567bb4295c9 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mds.h smithi134 running (74s) 58s ago 73s 16.4M - 18.0.0-4735-gbffe049d fa7da5dee971 4d4417878999 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mds.i smithi194 running (72s) 58s ago 71s 13.4M - 18.0.0-4735-gbffe049d fa7da5dee971 7e9037a91fa8 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mds.j smithi093 running (85s) 58s ago 85s 15.6M - 18.0.0-4735-gbffe049d fa7da5dee971 0dcf36615344 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mds.k smithi134 running (87s) 58s ago 87s 16.5M - 18.0.0-4735-gbffe049d fa7da5dee971 fbb34323aecf 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mds.l smithi194 running (89s) 58s ago 89s 16.5M - 18.0.0-4735-gbffe049d fa7da5dee971 04effaa8794f 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.x smithi093 *:9283,8765 running (7m) 58s ago 7m 510M - 18.0.0-4735-gbffe049d fa7da5dee971 9bfff3ca84f6 2023-07-21T22:15:50.304 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.y smithi134 *:8443,8765 running (5m) 58s ago 5m 448M - 18.0.0-4735-gbffe049d fa7da5dee971 430624262cb7 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.z smithi194 *:8443,8765 running (6m) 58s ago 6m 448M - 18.0.0-4735-gbffe049d fa7da5dee971 59d66bc02ea6 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:mon.a smithi093 running (7m) 58s ago 7m 83.0M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 5facc5ac0552 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:mon.b smithi134 running (6m) 58s ago 6m 57.7M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 6410d121a71e 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:mon.c smithi194 running (6m) 58s ago 6m 58.1M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 a441c3d0057d 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 smithi093 running (5m) 58s ago 5m 55.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 bcca1f15de51 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 smithi134 running (5m) 58s ago 5m 58.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 6530d3af4b91 2023-07-21T22:15:50.305 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 smithi194 running (5m) 58s ago 4m 53.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 260b7e1e1cd5 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 smithi093 running (4m) 58s ago 4m 50.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 e9848dee7328 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 smithi134 running (4m) 58s ago 4m 51.8M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 a1193fad62b6 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 smithi194 running (3m) 58s ago 3m 55.1M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 995a0bc3a76d 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 smithi093 running (3m) 58s ago 3m 47.2M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 8d309f022a0a 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 smithi134 running (3m) 58s ago 3m 51.5M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 49db18eebd94 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 smithi194 running (2m) 58s ago 2m 48.7M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 2bcda5b5a521 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 smithi093 running (2m) 58s ago 2m 48.4M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 b6add59c3950 2023-07-21T22:15:50.306 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 smithi134 running (2m) 58s ago 2m 47.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 162517210bbe 2023-07-21T22:15:50.307 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 smithi194 running (107s) 58s ago 107s 45.1M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 d992c429fb22 2023-07-21T22:15:50.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 msgr2=0x7f27980a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:50.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 0x7f27980a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f279400b5c0 tx=0x7f2794005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 msgr2=0x7f27a81959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279c00b530 tx=0x7f279c00b840 comp rx=0 tx=0).stop 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 shutdown_connections 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f27980a1260 0x7f27980a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27a8100610 0x7f27a819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27a80fea80 0x7f27a81959c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 --2- 172.21.15.93:0/2436666115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a80ffc80 0x7f27a8195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:50.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 >> 172.21.15.93:0/2436666115 conn(0x7f27a80fa0b0 msgr2=0x7f27a8068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:50.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 shutdown_connections 2023-07-21T22:15:50.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:50.261+0000 7f27aec87700 1 -- 172.21.15.93:0/2436666115 wait complete. 2023-07-21T22:15:50.324 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:50 smithi093 ceph-mon[96579]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T22:15:50.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:50 smithi194 ceph-mon[98288]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T22:15:50.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:50 smithi134 ceph-mon[97873]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-21T22:15:50.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-21T22:15:51.273 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:51.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:51 smithi194 ceph-mon[98288]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:51.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:51 smithi134 ceph-mon[97873]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:51.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:51 smithi093 ceph-mon[96579]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:52.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T22:15:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:15:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-07-21T22:15:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-21T22:15:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-07-21T22:15:52.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-21T22:15:52.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:15:52.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:52.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:52 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T22:15:52.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:15:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-07-21T22:15:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-21T22:15:52.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-07-21T22:15:52.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-21T22:15:52.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:15:52.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:52.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:52 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:52.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-07-21T22:15:52.410 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:15:52.411 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:52 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:52.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 -- 172.21.15.93:0/650271245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c0ffb90 msgr2=0x7f578c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:52.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 --2- 172.21.15.93:0/650271245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c0ffb90 0x7f578c0fffa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5788009990 tx=0x7f5788009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 -- 172.21.15.93:0/650271245 shutdown_connections 2023-07-21T22:15:52.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 --2- 172.21.15.93:0/650271245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c101720 0x7f578c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 --2- 172.21.15.93:0/650271245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c0ffb90 0x7f578c0fffa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 --2- 172.21.15.93:0/650271245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c100d90 0x7f578c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 -- 172.21.15.93:0/650271245 >> 172.21.15.93:0/650271245 conn(0x7f578c0fb1a0 msgr2=0x7f578c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:52.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 -- 172.21.15.93:0/650271245 shutdown_connections 2023-07-21T22:15:52.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.672+0000 7f579429d700 1 -- 172.21.15.93:0/650271245 wait complete. 2023-07-21T22:15:52.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 Processor -- start 2023-07-21T22:15:52.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 -- start start 2023-07-21T22:15:52.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c0ffb90 0x7f578c0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:52.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c100d90 0x7f578c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:52.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:52.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0735c0 con 0x7f578c101720 2023-07-21T22:15:52.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c073730 con 0x7f578c100d90 2023-07-21T22:15:52.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579429d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c19f900 con 0x7f578c0ffb90 2023-07-21T22:15:52.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:52.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43200/0 (socket says 172.21.15.93:43200) 2023-07-21T22:15:52.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 -- 172.21.15.93:0/2321815824 learned_addr learned my addr 172.21.15.93:0/2321815824 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f5792039700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c0ffb90 0x7f578c0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f5791838700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c100d90 0x7f578c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 -- 172.21.15.93:0/2321815824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c0ffb90 msgr2=0x7f578c0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c0ffb90 0x7f578c0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 -- 172.21.15.93:0/2321815824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c100d90 msgr2=0x7f578c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c100d90 0x7f578c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 -- 172.21.15.93:0/2321815824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880095f0 con 0x7f578c101720 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f579283a700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f578400ef30 tx=0x7f578400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.673+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784018070 con 0x7f578c101720 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.674+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f578400f040 con 0x7f578c101720 2023-07-21T22:15:52.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.674+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c19fb80 con 0x7f578c101720 2023-07-21T22:15:52.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.674+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400cd00 con 0x7f578c101720 2023-07-21T22:15:52.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.674+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c1a0080 con 0x7f578c101720 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.675+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c1071b0 con 0x7f578c101720 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.676+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5784003a00 con 0x7f578c101720 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.676+0000 7f57837fe700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 0x7f57780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.677+0000 7f5792039700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 0x7f57780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.677+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f57840ccf20 con 0x7f578c101720 2023-07-21T22:15:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.677+0000 7f5792039700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 0x7f57780a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f578800b2e0 tx=0x7f5788005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:52.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.680+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5784096570 con 0x7f578c101720 2023-07-21T22:15:52.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.893+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f578c060950 con 0x7f57780a1260 2023-07-21T22:15:52.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.898+0000 7f57837fe700 1 -- 172.21.15.93:0/2321815824 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f578c060950 con 0x7f57780a1260 2023-07-21T22:15:52.943 INFO:teuthology.orchestra.run.smithi093.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-21T22:15:52.943 INFO:teuthology.orchestra.run.smithi093.stdout:mds.all 12/12 61s ago 94s smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi134=b;smithi134=e;smithi134=h;smithi134=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2023-07-21T22:15:52.943 INFO:teuthology.orchestra.run.smithi093.stdout:mgr 3/3 61s ago 6m smithi093=x;smithi134=y;smithi194=z;count:3 2023-07-21T22:15:52.943 INFO:teuthology.orchestra.run.smithi093.stdout:mon 3/3 61s ago 6m smithi093:172.21.15.93=a;smithi134:172.21.15.134=b;smithi194:172.21.15.194=c;count:3 2023-07-21T22:15:52.943 INFO:teuthology.orchestra.run.smithi093.stdout:osd 12 61s ago - 2023-07-21T22:15:52.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 msgr2=0x7f57780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:52.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 0x7f57780a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f578800b2e0 tx=0x7f5788005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 msgr2=0x7f578c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f578400ef30 tx=0x7f578400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 shutdown_connections 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f578c101720 0x7f578c072e40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f578c0ffb90 0x7f578c0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57780a1260 0x7f57780a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 --2- 172.21.15.93:0/2321815824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f578c100d90 0x7f578c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:52.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 >> 172.21.15.93:0/2321815824 conn(0x7f578c0fb1a0 msgr2=0x7f578c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:52.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 shutdown_connections 2023-07-21T22:15:52.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:52.902+0000 7f579429d700 1 -- 172.21.15.93:0/2321815824 wait complete. 2023-07-21T22:15:53.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-21T22:15:53.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:15:53.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:53 smithi194 ceph-mon[98288]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:15:53.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T22:15:53.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:15:53.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-07-21T22:15:53.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-21T22:15:53.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-07-21T22:15:53.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-21T22:15:53.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:15:53.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:53 smithi134 ceph-mon[97873]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:15:53.432 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-07-21T22:15:53.432 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:15:53.432 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-07-21T22:15:53.432 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-07-21T22:15:53.433 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-07-21T22:15:53.433 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-21T22:15:53.433 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:15:53.433 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:53 smithi093 ceph-mon[96579]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:15:53.434 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-21T22:15:53.992 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:54.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:54 smithi194 ceph-mon[98288]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:54.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:54 smithi194 ceph-mon[98288]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:15:54.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:54 smithi194 ceph-mon[98288]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:54.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:54 smithi134 ceph-mon[97873]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:54.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:54 smithi134 ceph-mon[97873]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:15:54.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:54 smithi134 ceph-mon[97873]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:54.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:54 smithi093 ceph-mon[96579]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:54.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:54 smithi093 ceph-mon[96579]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:15:54.419 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:54 smithi093 ceph-mon[96579]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:55.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 -- 172.21.15.93:0/1991243942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974102f90 msgr2=0x7f2974105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 --2- 172.21.15.93:0/1991243942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974102f90 0x7f2974105370 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2968009990 tx=0x7f2968009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 -- 172.21.15.93:0/1991243942 shutdown_connections 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 --2- 172.21.15.93:0/1991243942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 0x7f2974107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 --2- 172.21.15.93:0/1991243942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974100670 0x7f2974102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 --2- 172.21.15.93:0/1991243942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974102f90 0x7f2974105370 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 -- 172.21.15.93:0/1991243942 >> 172.21.15.93:0/1991243942 conn(0x7f29740fa0b0 msgr2=0x7f29740fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:55.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 -- 172.21.15.93:0/1991243942 shutdown_connections 2023-07-21T22:15:55.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 -- 172.21.15.93:0/1991243942 wait complete. 2023-07-21T22:15:55.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.194+0000 7f297a8f6700 1 Processor -- start 2023-07-21T22:15:55.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 -- start start 2023-07-21T22:15:55.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:55.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 0x7f2974195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 0x7f297419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419d460 con 0x7f2974100670 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419d5a0 con 0x7f2974102f90 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f297a8f6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297419d730 con 0x7f29741058b0 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43208/0 (socket says 172.21.15.93:43208) 2023-07-21T22:15:55.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 -- 172.21.15.93:0/2211346009 learned_addr learned my addr 172.21.15.93:0/2211346009 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f29737fe700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 0x7f2974195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2978e93700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 0x7f297419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 -- 172.21.15.93:0/2211346009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 msgr2=0x7f297419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 0x7f297419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 -- 172.21.15.93:0/2211346009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 msgr2=0x7f2974195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 0x7f2974195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f2973fff700 1 -- 172.21.15.93:0/2211346009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680095f0 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.195+0000 7f29737fe700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 0x7f2974195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f2973fff700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9d0 tx=0x7f296400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964009790 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297419da10 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f296400cbd0 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964018680 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.196+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297419df10 con 0x7f2974100670 2023-07-21T22:15:55.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.197+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297410b370 con 0x7f2974100670 2023-07-21T22:15:55.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.198+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2964010430 con 0x7f2974100670 2023-07-21T22:15:55.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.199+0000 7f29717fa700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 0x7f295c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:55.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.199+0000 7f29737fe700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 0x7f295c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:55.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.199+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2964014070 con 0x7f2974100670 2023-07-21T22:15:55.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.199+0000 7f29737fe700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 0x7f295c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f296800b2e0 tx=0x7f2968005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:55.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.203+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2964051770 con 0x7f2974100670 2023-07-21T22:15:55.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:55 smithi194 ceph-mon[98288]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-21T22:15:55.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:55 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:55 smithi134 ceph-mon[97873]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-21T22:15:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:55 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:55 smithi093 ceph-mon[96579]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-21T22:15:55.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:55 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:55.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.421+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2974102240 con 0x7f295c0a11e0 2023-07-21T22:15:55.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.422+0000 7f29717fa700 1 -- 172.21.15.93:0/2211346009 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f2974102240 con 0x7f295c0a11e0 2023-07-21T22:15:55.466 INFO:teuthology.orchestra.run.smithi093.stdout:HOST ADDR LABELS STATUS 2023-07-21T22:15:55.466 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 172.21.15.93 2023-07-21T22:15:55.466 INFO:teuthology.orchestra.run.smithi093.stdout:smithi134 172.21.15.134 2023-07-21T22:15:55.466 INFO:teuthology.orchestra.run.smithi093.stdout:smithi194 172.21.15.194 2023-07-21T22:15:55.467 INFO:teuthology.orchestra.run.smithi093.stdout:3 hosts in cluster 2023-07-21T22:15:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 msgr2=0x7f295c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 0x7f295c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f296800b2e0 tx=0x7f2968005fb0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 msgr2=0x7f29741958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f296400e9d0 tx=0x7f296400ed90 comp rx=0 tx=0).stop 2023-07-21T22:15:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 shutdown_connections 2023-07-21T22:15:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29741058b0 0x7f297419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2974100670 0x7f29741958d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f295c0a11e0 0x7f295c0a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 --2- 172.21.15.93:0/2211346009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974102f90 0x7f2974195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.424+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 >> 172.21.15.93:0/2211346009 conn(0x7f29740fa0b0 msgr2=0x7f2974101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.425+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 shutdown_connections 2023-07-21T22:15:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:55.425+0000 7f297a8f6700 1 -- 172.21.15.93:0/2211346009 wait complete. 2023-07-21T22:15:55.845 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-21T22:15:56.395 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:56.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:56.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:56 smithi194 ceph-mon[98288]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:56.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:56.588 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:56.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:56.589 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:56 smithi093 ceph-mon[96579]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:56.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:56 smithi134 ceph-mon[97873]: from='mgr.14152 172.21.15.93:0/3667045618' entity='mgr.x' 2023-07-21T22:15:57.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1823922905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c741000a0 msgr2=0x7f3c741004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1823922905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c741000a0 0x7f3c741004b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009990 tx=0x7f3c64009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1823922905 shutdown_connections 2023-07-21T22:15:57.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1823922905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c740ff6b0 0x7f3c74106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1823922905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c741000a0 0x7f3c741004b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1823922905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740fed00 0x7f3c740ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1823922905 >> 172.21.15.93:0/1823922905 conn(0x7f3c740f9860 msgr2=0x7f3c740fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:57.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1823922905 shutdown_connections 2023-07-21T22:15:57.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1823922905 wait complete. 2023-07-21T22:15:57.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 Processor -- start 2023-07-21T22:15:57.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.436+0000 7f3c7a3c1700 1 -- start start 2023-07-21T22:15:57.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:57.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740ff6b0 0x7f3c7419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:57.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 0x7f3c7419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:57.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7419d320 con 0x7f3c740fed00 2023-07-21T22:15:57.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7419d490 con 0x7f3c741000a0 2023-07-21T22:15:57.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7a3c1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7419d650 con 0x7f3c740ff6b0 2023-07-21T22:15:57.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:57.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43220/0 (socket says 172.21.15.93:43220) 2023-07-21T22:15:57.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 -- 172.21.15.93:0/1531051628 learned_addr learned my addr 172.21.15.93:0/1531051628 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:57.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c737fe700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740ff6b0 0x7f3c7419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:57.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7895e700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 0x7f3c7419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:57.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 -- 172.21.15.93:0/1531051628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740ff6b0 msgr2=0x7f3c7419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740ff6b0 0x7f3c7419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 -- 172.21.15.93:0/1531051628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 msgr2=0x7f3c7419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 0x7f3c7419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 -- 172.21.15.93:0/1531051628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640095f0 con 0x7f3c740fed00 2023-07-21T22:15:57.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c7895e700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 0x7f3c7419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:15:57.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.437+0000 7f3c73fff700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c640095c0 tx=0x7f3c6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:57.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.438+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6401d070 con 0x7f3c740fed00 2023-07-21T22:15:57.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.438+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c64022470 con 0x7f3c740fed00 2023-07-21T22:15:57.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.438+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7419d8d0 con 0x7f3c740fed00 2023-07-21T22:15:57.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.438+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64017650 con 0x7f3c740fed00 2023-07-21T22:15:57.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.438+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7419dce0 con 0x7f3c740fed00 2023-07-21T22:15:57.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.439+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c74106010 con 0x7f3c740fed00 2023-07-21T22:15:57.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.440+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3c6400fad0 con 0x7f3c740fed00 2023-07-21T22:15:57.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.440+0000 7f3c717fa700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 0x7f3c600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:57.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.440+0000 7f3c737fe700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 0x7f3c600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:57.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.440+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3c640cde70 con 0x7f3c740fed00 2023-07-21T22:15:57.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.440+0000 7f3c737fe700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 0x7f3c600a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c740fecd0 tx=0x7f3c5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:57.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.444+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3c64097640 con 0x7f3c740fed00 2023-07-21T22:15:57.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.639+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c74060950 con 0x7f3c600a11e0 2023-07-21T22:15:57.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.641+0000 7f3c717fa700 1 -- 172.21.15.93:0/1531051628 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f3c74060950 con 0x7f3c600a11e0 2023-07-21T22:15:57.684 INFO:teuthology.orchestra.run.smithi093.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-21T22:15:57.684 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040150400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:15:57.685 INFO:teuthology.orchestra.run.smithi093.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:15:57.685 INFO:teuthology.orchestra.run.smithi093.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 109s ago LVM detected, locked 2023-07-21T22:15:57.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 msgr2=0x7f3c600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 0x7f3c600a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c740fecd0 tx=0x7f3c5c00b410 comp rx=0 tx=0).stop 2023-07-21T22:15:57.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 msgr2=0x7f3c740fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c640095c0 tx=0x7f3c6400f690 comp rx=0 tx=0).stop 2023-07-21T22:15:57.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 shutdown_connections 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3c600a11e0 0x7f3c600a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c741000a0 0x7f3c7419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c740fed00 0x7f3c740fe770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 --2- 172.21.15.93:0/1531051628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c740ff6b0 0x7f3c7419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 >> 172.21.15.93:0/1531051628 conn(0x7f3c740f9860 msgr2=0x7f3c74103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 shutdown_connections 2023-07-21T22:15:57.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:57.643+0000 7f3c7a3c1700 1 -- 172.21.15.93:0/1531051628 wait complete. 2023-07-21T22:15:58.037 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:15:58.046 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:15:58.047 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-21T22:15:58.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:58 smithi194 ceph-mon[98288]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:58.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:58 smithi194 ceph-mon[98288]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:58.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:58 smithi134 ceph-mon[97873]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:58.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:58 smithi134 ceph-mon[97873]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:58.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:58 smithi093 ceph-mon[96579]: pgmap v287: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:15:58.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:58 smithi093 ceph-mon[96579]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:58.600 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:59 smithi093 ceph-mon[96579]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-21T22:15:59.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:15:59 smithi093 ceph-mon[96579]: Cluster is now healthy 2023-07-21T22:15:59.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:59 smithi194 ceph-mon[98288]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-21T22:15:59.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:15:59 smithi194 ceph-mon[98288]: Cluster is now healthy 2023-07-21T22:15:59.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:59 smithi134 ceph-mon[97873]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-21T22:15:59.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:15:59 smithi134 ceph-mon[97873]: Cluster is now healthy 2023-07-21T22:15:59.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 -- 172.21.15.93:0/1934709125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc100610 msgr2=0x7f79dc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:59.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 --2- 172.21.15.93:0/1934709125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc100610 0x7f79dc106ed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b0a0 tx=0x7f79d400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 -- 172.21.15.93:0/1934709125 shutdown_connections 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 --2- 172.21.15.93:0/1934709125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc100610 0x7f79dc106ed0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 --2- 172.21.15.93:0/1934709125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 0x7f79dc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 --2- 172.21.15.93:0/1934709125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc0ffc80 0x7f79dc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 -- 172.21.15.93:0/1934709125 >> 172.21.15.93:0/1934709125 conn(0x7f79dc0fa0b0 msgr2=0x7f79dc0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 -- 172.21.15.93:0/1934709125 shutdown_connections 2023-07-21T22:15:59.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.731+0000 7f79e30ba700 1 -- 172.21.15.93:0/1934709125 wait complete. 2023-07-21T22:15:59.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 Processor -- start 2023-07-21T22:15:59.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 -- start start 2023-07-21T22:15:59.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 0x7f79dc1948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:59.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc100610 0x7f79dc19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19c450 con 0x7f79dc0ffc80 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19c590 con 0x7f79dc100610 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79e30ba700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19c720 con 0x7f79dc0fea80 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43234/0 (socket says 172.21.15.93:43234) 2023-07-21T22:15:59.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.732+0000 7f79dbfff700 1 -- 172.21.15.93:0/2658767662 learned_addr learned my addr 172.21.15.93:0/2658767662 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:15:59.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79e1657700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc100610 0x7f79dc19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:59.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79e0e56700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 0x7f79dc1948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:59.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 -- 172.21.15.93:0/2658767662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 msgr2=0x7f79dc1948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:59.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 0x7f79dc1948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 -- 172.21.15.93:0/2658767662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc100610 msgr2=0x7f79dc19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:59.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc100610 0x7f79dc19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:59.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 -- 172.21.15.93:0/2658767662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009d00 con 0x7f79dc0ffc80 2023-07-21T22:15:59.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79dbfff700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f79c800d9b0 tx=0x7f79c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8004a30 con 0x7f79dc0ffc80 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c80054e0 con 0x7f79dc0ffc80 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc19ca00 con 0x7f79dc0ffc80 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8009c90 con 0x7f79dc0ffc80 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.733+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc19ceb0 con 0x7f79dc0ffc80 2023-07-21T22:15:59.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.734+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc106020 con 0x7f79dc0ffc80 2023-07-21T22:15:59.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.735+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f79c8004c20 con 0x7f79dc0ffc80 2023-07-21T22:15:59.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.736+0000 7f79d9ffb700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:59.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.736+0000 7f79e0e56700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:59.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.736+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79c80cc980 con 0x7f79dc0ffc80 2023-07-21T22:15:59.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.736+0000 7f79e0e56700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 0x7f79cc0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005950 tx=0x7f79d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:59.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.740+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f79c8095fa0 con 0x7f79dc0ffc80 2023-07-21T22:16:00.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.970+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79dc02ca70 con 0x7f79dc0ffc80 2023-07-21T22:16:00.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.970+0000 7f79d9ffb700 1 -- 172.21.15.93:0/2658767662 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2927 (secure 0 0 0) 0x7f79c8017070 con 0x7f79dc0ffc80 2023-07-21T22:16:00.014 INFO:teuthology.orchestra.run.smithi093.stdout:e26 2023-07-21T22:16:00.015 INFO:teuthology.orchestra.run.smithi093.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:16:00.015 INFO:teuthology.orchestra.run.smithi093.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:16:00.015 INFO:teuthology.orchestra.run.smithi093.stdout:legacy client fscid: 1 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:Filesystem 'cephfs' (1) 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:fs_name cephfs 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 26 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-07-21T22:14:50.635520+0000 2023-07-21T22:16:00.016 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-07-21T22:14:58.012216+0000 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:tableserver 0 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:root 0 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:session_timeout 300 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:session_autoclose 300 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:max_file_size 1099511627776 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:max_xattr_size 65536 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:required_client_features {} 2023-07-21T22:16:00.017 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure 0 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure_osd_epoch 0 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:max_mds 5 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:in 0,1,2,3,4 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:up {0=24484,1=24482,2=24460,3=24448,4=14589} 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:failed 2023-07-21T22:16:00.018 INFO:teuthology.orchestra.run.smithi093.stdout:damaged 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:stopped 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:data_pools [3] 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:metadata_pool 2 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:inline_data disabled 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:balancer 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:bal_rank_mask -1 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:standby_count_wanted 0 2023-07-21T22:16:00.019 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.i{0:24484} state up:active seq 5 addr [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.020 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.h{0:24446} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/117715569,v1:172.21.15.134:6837/117715569] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.020 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b{1:24482} state up:active seq 4 addr [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.020 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.l{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.194:6832/3279794938,v1:172.21.15.194:6833/3279794938] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.020 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c{2:24460} state up:active seq 7 addr [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.021 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.e{2:24416} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/3262717838,v1:172.21.15.134:6835/3262717838] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.021 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.f{3:24448} state up:active seq 8 addr [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.021 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.k{3:24398} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/2616544286,v1:172.21.15.134:6833/2616544286] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.021 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a{4:14589} state up:active seq 5 addr [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.021 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.g{4:14577} state up:standby-replay seq 1 addr [v2:172.21.15.93:6838/3482994338,v1:172.21.15.93:6839/3482994338] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout:Standby daemons: 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.93:6834/3390846004,v1:172.21.15.93:6835/3390846004] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.022 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.93:6836/3934873649,v1:172.21.15.93:6837/3934873649] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:16:00.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 msgr2=0x7f79cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:00.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 0x7f79cc0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005950 tx=0x7f79d00058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 msgr2=0x7f79dc194e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f79c800d9b0 tx=0x7f79c800dcc0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 shutdown_connections 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79dc100610 0x7f79dc19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79dc0fea80 0x7f79dc1948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 --2- 172.21.15.93:0/2658767662 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79dc0ffc80 0x7f79dc194e00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:00.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.972+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 >> 172.21.15.93:0/2658767662 conn(0x7f79dc0fa0b0 msgr2=0x7f79dc103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:00.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.973+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 shutdown_connections 2023-07-21T22:16:00.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:15:59.973+0000 7f79e30ba700 1 -- 172.21.15.93:0/2658767662 wait complete. 2023-07-21T22:16:00.025 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:00.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:00 smithi194 ceph-mon[98288]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:00.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:00 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2658767662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:16:00.378 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:00 smithi134 ceph-mon[97873]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:00.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:00 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2658767662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:16:00.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:00 smithi093 ceph-mon[96579]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:00.386 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:00 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2658767662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:16:00.439 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-21T22:16:00.992 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:16:02.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- 172.21.15.93:0/3116384380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300ffc80 msgr2=0x7fea301000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 --2- 172.21.15.93:0/3116384380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300ffc80 0x7fea301000d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fea20009990 tx=0x7fea20009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- 172.21.15.93:0/3116384380 shutdown_connections 2023-07-21T22:16:02.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 --2- 172.21.15.93:0/3116384380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea30100610 0x7fea30106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 --2- 172.21.15.93:0/3116384380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea300fea80 0x7fea300fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 --2- 172.21.15.93:0/3116384380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300ffc80 0x7fea301000d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- 172.21.15.93:0/3116384380 >> 172.21.15.93:0/3116384380 conn(0x7fea300fa0b0 msgr2=0x7fea300fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- 172.21.15.93:0/3116384380 shutdown_connections 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- 172.21.15.93:0/3116384380 wait complete. 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 Processor -- start 2023-07-21T22:16:02.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.115+0000 7fea373fb700 1 -- start start 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea300ffc80 0x7fea30195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30100610 0x7fea3019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3019d4f0 con 0x7fea300fea80 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3019d630 con 0x7fea300ffc80 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3019d7c0 con 0x7fea30100610 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43246/0 (socket says 172.21.15.93:43246) 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 -- 172.21.15.93:0/995531727 learned_addr learned my addr 172.21.15.93:0/995531727 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35998700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30100610 0x7fea3019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea34996700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea300ffc80 0x7fea30195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 -- 172.21.15.93:0/995531727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30100610 msgr2=0x7fea3019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30100610 0x7fea3019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 -- 172.21.15.93:0/995531727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea300ffc80 msgr2=0x7fea30195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea300ffc80 0x7fea30195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 -- 172.21.15.93:0/995531727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200095f0 con 0x7fea300fea80 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea35197700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00b530 tx=0x7fea2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c011830 con 0x7fea300fea80 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3019daa0 con 0x7fea300fea80 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea2c011e50 con 0x7fea300fea80 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c010de0 con 0x7fea300fea80 2023-07-21T22:16:02.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.116+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3019dff0 con 0x7fea300fea80 2023-07-21T22:16:02.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.117+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3010a2e0 con 0x7fea300fea80 2023-07-21T22:16:02.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.118+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fea2c010480 con 0x7fea300fea80 2023-07-21T22:16:02.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.119+0000 7fea267fc700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 0x7fea1c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.119+0000 7fea34996700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 0x7fea1c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.119+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fea2c0cc2c0 con 0x7fea300fea80 2023-07-21T22:16:02.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.119+0000 7fea34996700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 0x7fea1c0a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea2000b5c0 tx=0x7fea20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:02.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.123+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fea2c0959f0 con 0x7fea300fea80 2023-07-21T22:16:02.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.317+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fea30106b50 con 0x7fea300fea80 2023-07-21T22:16:02.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.318+0000 7fea267fc700 1 -- 172.21.15.93:0/995531727 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5246 (secure 0 0 0) 0x7fea2c099010 con 0x7fea300fea80 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 84 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:fsid de1cec7a-2812-11ee-9b34-001a4aab830c 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-07-21T22:08:02.004166+0000 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-07-21T22:15:53.025400+0000 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-21T22:16:02.362 INFO:teuthology.orchestra.run.smithi093.stdout:crush_version 26 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:full_ratio 0.95 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:backfillfull_ratio 0.9 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:nearfull_ratio 0.85 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:require_min_compat_client luminous 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:min_compat_client luminous 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:require_osd_release reef 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:stretch_mode_enabled false 2023-07-21T22:16:02.363 INFO:teuthology.orchestra.run.smithi093.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:max_osd 12 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6802/2127775933,v1:172.21.15.93:6803/2127775933] [v2:172.21.15.93:6804/2127775933,v1:172.21.15.93:6805/2127775933] exists,up 1a67ed18-5188-4868-8069-de6b0b8165cd 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/2449999795,v1:172.21.15.134:6801/2449999795] [v2:172.21.15.134:6802/2449999795,v1:172.21.15.134:6803/2449999795] exists,up 6567d033-de01-4296-ab88-b6781b7519ea 2023-07-21T22:16:02.364 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/866955250,v1:172.21.15.194:6801/866955250] [v2:172.21.15.194:6802/866955250,v1:172.21.15.194:6803/866955250] exists,up 4704710d-70ac-4f07-8dbd-509266a7d1c6 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6810/921753281,v1:172.21.15.93:6811/921753281] [v2:172.21.15.93:6812/921753281,v1:172.21.15.93:6813/921753281] exists,up 5622ee06-de4f-4d56-9829-efced7a3fbcc 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 up in weight 1 up_from 32 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/2960934129,v1:172.21.15.134:6809/2960934129] [v2:172.21.15.134:6810/2960934129,v1:172.21.15.134:6811/2960934129] exists,up 8bfd6964-22c3-4847-b0a2-095f09b257e7 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/1989091547,v1:172.21.15.194:6809/1989091547] [v2:172.21.15.194:6810/1989091547,v1:172.21.15.194:6811/1989091547] exists,up ece1a413-eba9-4d35-bc27-352be61010b1 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 up in weight 1 up_from 45 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6818/4017577032,v1:172.21.15.93:6819/4017577032] [v2:172.21.15.93:6820/4017577032,v1:172.21.15.93:6821/4017577032] exists,up f05d761d-7844-4e6d-99e7-94acab20daad 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 up in weight 1 up_from 50 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/4117548000,v1:172.21.15.134:6817/4117548000] [v2:172.21.15.134:6818/4117548000,v1:172.21.15.134:6819/4117548000] exists,up 1147542a-1a5b-4935-99ae-2c196c6bb096 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 up in weight 1 up_from 57 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/773360431,v1:172.21.15.194:6817/773360431] [v2:172.21.15.194:6818/773360431,v1:172.21.15.194:6819/773360431] exists,up f34cab3d-d40c-47b9-a691-b95f36965de5 2023-07-21T22:16:02.365 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 up in weight 1 up_from 64 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6826/692701270,v1:172.21.15.93:6827/692701270] [v2:172.21.15.93:6828/692701270,v1:172.21.15.93:6829/692701270] exists,up 7aca17f8-875e-421f-a4b8-9b85656aa761 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 up in weight 1 up_from 69 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/51617631,v1:172.21.15.134:6825/51617631] [v2:172.21.15.134:6826/51617631,v1:172.21.15.134:6827/51617631] exists,up ef648196-6eb5-4d15-bb0f-b6dedf1c03b7 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/4036125050,v1:172.21.15.194:6825/4036125050] [v2:172.21.15.194:6826/4036125050,v1:172.21.15.194:6827/4036125050] exists,up 68ec4cd3-029b-45b3-af2a-16ef9ea3be47 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.6 [8,11] 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.a [2,11] 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.11 [4,8] 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.13 [4,9] 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.14 [4,6] 2023-07-21T22:16:02.366 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.18 [3,9] 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.33 [5,9] 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/3590233301 expires 2023-07-22T22:08:51.131041+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/3590233301 expires 2023-07-22T22:08:51.131041+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/854330056 expires 2023-07-22T22:08:51.131041+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/2048241181 expires 2023-07-22T22:08:29.027813+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/2048241181 expires 2023-07-22T22:08:29.027813+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/3733549091 expires 2023-07-22T22:08:29.027813+0000 2023-07-21T22:16:02.367 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/2873292277 expires 2023-07-22T22:08:51.131041+0000 2023-07-21T22:16:02.368 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/1829577156 expires 2023-07-22T22:08:51.131041+0000 2023-07-21T22:16:02.368 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/2824313704 expires 2023-07-22T22:08:29.027813+0000 2023-07-21T22:16:02.368 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/3858928377 expires 2023-07-22T22:08:29.027813+0000 2023-07-21T22:16:02.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 msgr2=0x7fea1c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 0x7fea1c0a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea2000b5c0 tx=0x7fea20005fd0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 msgr2=0x7fea30195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00b530 tx=0x7fea2c00b840 comp rx=0 tx=0).stop 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 shutdown_connections 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30100610 0x7fea3019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea300fea80 0x7fea30195960 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea1c0a1260 0x7fea1c0a3710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 --2- 172.21.15.93:0/995531727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea300ffc80 0x7fea30195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 >> 172.21.15.93:0/995531727 conn(0x7fea300fa0b0 msgr2=0x7fea300fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:02.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 shutdown_connections 2023-07-21T22:16:02.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.320+0000 7fea373fb700 1 -- 172.21.15.93:0/995531727 wait complete. 2023-07-21T22:16:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:02 smithi194 ceph-mon[98288]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:02.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:02 smithi134 ceph-mon[97873]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:02.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:02 smithi093 ceph-mon[96579]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:02.738 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-21T22:16:02.751 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:02.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/3905580109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db80ffb00 msgr2=0x7f8db80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/3905580109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db80ffb00 0x7f8db80fff10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009990 tx=0x7f8da8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/3905580109 shutdown_connections 2023-07-21T22:16:02.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/3905580109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/3905580109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db80ffb00 0x7f8db80fff10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/3905580109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db8100cf0 0x7f8db8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/3905580109 >> 172.21.15.93:0/3905580109 conn(0x7f8db80fb100 msgr2=0x7f8db80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:02.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/3905580109 shutdown_connections 2023-07-21T22:16:02.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.833+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/3905580109 wait complete. 2023-07-21T22:16:02.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 Processor -- start 2023-07-21T22:16:02.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 -- start start 2023-07-21T22:16:02.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db80ffb00 0x7f8db81959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819d530 con 0x7f8db8100cf0 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819d670 con 0x7f8db8101680 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8dbd0e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819d800 con 0x7f8db80ffb00 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43262/0 (socket says 172.21.15.93:43262) 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db659c700 1 -- 172.21.15.93:0/651566148 learned_addr learned my addr 172.21.15.93:0/651566148 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db759e700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db6d9d700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db80ffb00 0x7f8db81959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db659c700 1 -- 172.21.15.93:0/651566148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db80ffb00 msgr2=0x7f8db81959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.834+0000 7f8db659c700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db80ffb00 0x7f8db81959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8db659c700 1 -- 172.21.15.93:0/651566148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 msgr2=0x7f8db819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:02.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8db659c700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8db659c700 1 -- 172.21.15.93:0/651566148 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80095f0 con 0x7f8db8100cf0 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8db759e700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8db659c700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8da000b530 tx=0x7f8da000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0011830 con 0x7f8db8100cf0 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da0011e50 con 0x7f8db8100cf0 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db819dae0 con 0x7f8db8100cf0 2023-07-21T22:16:02.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da000f2e0 con 0x7f8db8100cf0 2023-07-21T22:16:02.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.835+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db819e040 con 0x7f8db8100cf0 2023-07-21T22:16:02.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.836+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db8106a80 con 0x7f8db8100cf0 2023-07-21T22:16:02.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.837+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8da0010580 con 0x7f8db8100cf0 2023-07-21T22:16:02.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.838+0000 7f8daffff700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 0x7f8da40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:02.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.838+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8da00cc960 con 0x7f8db8100cf0 2023-07-21T22:16:02.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.838+0000 7f8db6d9d700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 0x7f8da40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:02.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.839+0000 7f8db6d9d700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 0x7f8da40a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8da800b5c0 tx=0x7f8da8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:02.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:02.842+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8da0096090 con 0x7f8db8100cf0 2023-07-21T22:16:03.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.035+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db802caa0 con 0x7f8db8100cf0 2023-07-21T22:16:03.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.037+0000 7f8daffff700 1 -- 172.21.15.93:0/651566148 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7f8da0015020 con 0x7f8db8100cf0 2023-07-21T22:16:03.082 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:03.082 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:03.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 msgr2=0x7f8da40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 0x7f8da40a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8da800b5c0 tx=0x7f8da8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 msgr2=0x7f8db8195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8da000b530 tx=0x7f8da000b8f0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 shutdown_connections 2023-07-21T22:16:03.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8da40a12a0 0x7f8da40a3750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8db8101680 0x7f8db819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8db80ffb00 0x7f8db81959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 --2- 172.21.15.93:0/651566148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8db8100cf0 0x7f8db8195ef0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 >> 172.21.15.93:0/651566148 conn(0x7f8db80fb100 msgr2=0x7f8db80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 shutdown_connections 2023-07-21T22:16:03.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.039+0000 7f8dbd0e8700 1 -- 172.21.15.93:0/651566148 wait complete. 2023-07-21T22:16:03.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:16:03.097 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:16:03.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 -- 172.21.15.93:0/1001877626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900ffc00 msgr2=0x7faa90100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 --2- 172.21.15.93:0/1001877626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900ffc00 0x7faa90100070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faa84009990 tx=0x7faa84009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 -- 172.21.15.93:0/1001877626 shutdown_connections 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 --2- 172.21.15.93:0/1001877626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa901005b0 0x7faa90106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 --2- 172.21.15.93:0/1001877626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa900feef0 0x7faa900ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 --2- 172.21.15.93:0/1001877626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900ffc00 0x7faa90100070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 -- 172.21.15.93:0/1001877626 >> 172.21.15.93:0/1001877626 conn(0x7faa900f9770 msgr2=0x7faa900fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 -- 172.21.15.93:0/1001877626 shutdown_connections 2023-07-21T22:16:03.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.177+0000 7faa95205700 1 -- 172.21.15.93:0/1001877626 wait complete. 2023-07-21T22:16:03.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.178+0000 7faa95205700 1 Processor -- start 2023-07-21T22:16:03.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.178+0000 7faa95205700 1 -- start start 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa900ffc00 0x7faa90194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 0x7faa9019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9019c320 con 0x7faa900feef0 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9019c460 con 0x7faa901005b0 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9019c5f0 con 0x7faa900ffc00 2023-07-21T22:16:03.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43272/0 (socket says 172.21.15.93:43272) 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 -- 172.21.15.93:0/4042668795 learned_addr learned my addr 172.21.15.93:0/4042668795 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8f59e700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 0x7faa9019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8e59c700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa900ffc00 0x7faa90194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 -- 172.21.15.93:0/4042668795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa900ffc00 msgr2=0x7faa90194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa900ffc00 0x7faa90194ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 -- 172.21.15.93:0/4042668795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 msgr2=0x7faa9019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 0x7faa9019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 -- 172.21.15.93:0/4042668795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840095f0 con 0x7faa900feef0 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8f59e700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 0x7faa9019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa8ed9d700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faa8000d980 tx=0x7faa8000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000de60 con 0x7faa900feef0 2023-07-21T22:16:03.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa800044e0 con 0x7faa900feef0 2023-07-21T22:16:03.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9019c8d0 con 0x7faa900feef0 2023-07-21T22:16:03.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80003be0 con 0x7faa900feef0 2023-07-21T22:16:03.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.179+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9019cde0 con 0x7faa900feef0 2023-07-21T22:16:03.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.180+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9010a1b0 con 0x7faa900feef0 2023-07-21T22:16:03.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.181+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faa80009bb0 con 0x7faa900feef0 2023-07-21T22:16:03.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.182+0000 7faa77fff700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 0x7faa780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.182+0000 7faa8e59c700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 0x7faa780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.182+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa800ccc10 con 0x7faa900feef0 2023-07-21T22:16:03.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.182+0000 7faa8e59c700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 0x7faa780a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa84006010 tx=0x7faa8400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.185+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faa800961d0 con 0x7faa900feef0 2023-07-21T22:16:03.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.381+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa901059c0 con 0x7faa900feef0 2023-07-21T22:16:03.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.381+0000 7faa77fff700 1 -- 172.21.15.93:0/4042668795 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa80009940 con 0x7faa900feef0 2023-07-21T22:16:03.426 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:16:03.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 msgr2=0x7faa780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 0x7faa780a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa84006010 tx=0x7faa8400b540 comp rx=0 tx=0).stop 2023-07-21T22:16:03.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 msgr2=0x7faa901947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faa8000d980 tx=0x7faa8000dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 shutdown_connections 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa780a1260 0x7faa780a3710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa901005b0 0x7faa9019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa900feef0 0x7faa901947a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 --2- 172.21.15.93:0/4042668795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa900ffc00 0x7faa90194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.383+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 >> 172.21.15.93:0/4042668795 conn(0x7faa900f9770 msgr2=0x7faa900fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.384+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 shutdown_connections 2023-07-21T22:16:03.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.384+0000 7faa95205700 1 -- 172.21.15.93:0/4042668795 wait complete. 2023-07-21T22:16:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:03 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/995531727' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:16:03.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:03 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/651566148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:03.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:16:03.565 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 -- 172.21.15.93:0/3479365289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe201005c0 msgr2=0x7fbe20106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 --2- 172.21.15.93:0/3479365289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe201005c0 0x7fbe20106e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800b0a0 tx=0x7fbe1800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 -- 172.21.15.93:0/3479365289 shutdown_connections 2023-07-21T22:16:03.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 --2- 172.21.15.93:0/3479365289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe201005c0 0x7fbe20106e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 --2- 172.21.15.93:0/3479365289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe200fea40 0x7fbe200fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.520+0000 7fbe2553e700 1 --2- 172.21.15.93:0/3479365289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 0x7fbe20100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- 172.21.15.93:0/3479365289 >> 172.21.15.93:0/3479365289 conn(0x7fbe200fa060 msgr2=0x7fbe200fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- 172.21.15.93:0/3479365289 shutdown_connections 2023-07-21T22:16:03.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- 172.21.15.93:0/3479365289 wait complete. 2023-07-21T22:16:03.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 Processor -- start 2023-07-21T22:16:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- start start 2023-07-21T22:16:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 0x7fbe2019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe201005c0 0x7fbe2019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019d4a0 con 0x7fbe200fea40 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019d610 con 0x7fbe200ffc30 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.521+0000 7fbe2553e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019d7d0 con 0x7fbe201005c0 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43286/0 (socket says 172.21.15.93:43286) 2023-07-21T22:16:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 -- 172.21.15.93:0/1921955071 learned_addr learned my addr 172.21.15.93:0/1921955071 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:03.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1e7fc700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 0x7fbe2019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1f7fe700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe201005c0 0x7fbe2019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 -- 172.21.15.93:0/1921955071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe201005c0 msgr2=0x7fbe2019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe201005c0 0x7fbe2019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 -- 172.21.15.93:0/1921955071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 msgr2=0x7fbe2019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 0x7fbe2019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 -- 172.21.15.93:0/1921955071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18009d00 con 0x7fbe200fea40 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe1effd700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400d980 tx=0x7fbe1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004020 con 0x7fbe200fea40 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe1400f690 con 0x7fbe200fea40 2023-07-21T22:16:03.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2019dab0 con 0x7fbe200fea40 2023-07-21T22:16:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14003a60 con 0x7fbe200fea40 2023-07-21T22:16:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.522+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2019dfc0 con 0x7fbe200fea40 2023-07-21T22:16:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.523+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2010a200 con 0x7fbe200fea40 2023-07-21T22:16:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.524+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbe1400fb00 con 0x7fbe200fea40 2023-07-21T22:16:03.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.525+0000 7fbe07fff700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 0x7fbe080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.525+0000 7fbe1e7fc700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 0x7fbe080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.525+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbe140cc660 con 0x7fbe200fea40 2023-07-21T22:16:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.525+0000 7fbe1e7fc700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 0x7fbe080a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe20069180 tx=0x7fbe1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.529+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbe14095c80 con 0x7fbe200fea40 2023-07-21T22:16:03.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:03 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/995531727' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:16:03.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:03 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/651566148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:03 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/995531727' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:16:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:03 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/651566148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:03.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.717+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe201058a0 con 0x7fbe200fea40 2023-07-21T22:16:03.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.717+0000 7fbe07fff700 1 -- 172.21.15.93:0/1921955071 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe140992a0 con 0x7fbe200fea40 2023-07-21T22:16:03.763 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:16:03.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 msgr2=0x7fbe080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 0x7fbe080a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe20069180 tx=0x7fbe1000b410 comp rx=0 tx=0).stop 2023-07-21T22:16:03.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 msgr2=0x7fbe20069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400d980 tx=0x7fbe1400dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:03.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 shutdown_connections 2023-07-21T22:16:03.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe201005c0 0x7fbe2019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe200fea40 0x7fbe20069410 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fbe080a11e0 0x7fbe080a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.766 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 --2- 172.21.15.93:0/1921955071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe200ffc30 0x7fbe2019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 >> 172.21.15.93:0/1921955071 conn(0x7fbe200fa060 msgr2=0x7fbe200fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 shutdown_connections 2023-07-21T22:16:03.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.720+0000 7fbe2553e700 1 -- 172.21.15.93:0/1921955071 wait complete. 2023-07-21T22:16:03.776 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:16:03.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 -- 172.21.15.93:0/416972834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 msgr2=0x7ff0b4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 --2- 172.21.15.93:0/416972834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4101120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009990 tx=0x7ff0a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 -- 172.21.15.93:0/416972834 shutdown_connections 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 --2- 172.21.15.93:0/416972834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 --2- 172.21.15.93:0/416972834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 --2- 172.21.15.93:0/416972834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4101120 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 -- 172.21.15.93:0/416972834 >> 172.21.15.93:0/416972834 conn(0x7ff0b40fb120 msgr2=0x7ff0b40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 -- 172.21.15.93:0/416972834 shutdown_connections 2023-07-21T22:16:03.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 -- 172.21.15.93:0/416972834 wait complete. 2023-07-21T22:16:03.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.857+0000 7ff0b88db700 1 Processor -- start 2023-07-21T22:16:03.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 -- start start 2023-07-21T22:16:03.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b4195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d3f0 con 0x7ff0b4100cd0 2023-07-21T22:16:03.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d530 con 0x7ff0b40ffae0 2023-07-21T22:16:03.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d6c0 con 0x7ff0b4101660 2023-07-21T22:16:03.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43302/0 (socket says 172.21.15.93:43302) 2023-07-21T22:16:03.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 -- 172.21.15.93:0/1628387282 learned_addr learned my addr 172.21.15.93:0/1628387282 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:03.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b259c700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b4195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b2d9d700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 -- 172.21.15.93:0/1628387282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 msgr2=0x7ff0b419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 -- 172.21.15.93:0/1628387282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 msgr2=0x7ff0b4195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b4195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 -- 172.21.15.93:0/1628387282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80095f0 con 0x7ff0b4100cd0 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b2d9d700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b259c700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b4195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b1d9b700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c010 tx=0x7ff0a8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a801d070 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b419d940 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a8004ca0 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.858+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80055e0 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.859+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b419dd00 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.859+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b4106a10 con 0x7ff0b4100cd0 2023-07-21T22:16:03.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.860+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff0a800fc70 con 0x7ff0b4100cd0 2023-07-21T22:16:03.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.861+0000 7ff0a37fe700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 0x7ff09c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.861+0000 7ff0b259c700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 0x7ff09c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.861+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff0a80ce2f0 con 0x7ff0b4100cd0 2023-07-21T22:16:03.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.861+0000 7ff0b259c700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 0x7ff09c0a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4100b30 tx=0x7ff0a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:03.864+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff0a8097940 con 0x7ff0b4100cd0 2023-07-21T22:16:04.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.054+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0b402caa0 con 0x7ff0b4100cd0 2023-07-21T22:16:04.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.054+0000 7ff0a37fe700 1 -- 172.21.15.93:0/1628387282 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0a8005740 con 0x7ff0b4100cd0 2023-07-21T22:16:04.099 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:16:04.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 msgr2=0x7ff09c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 0x7ff09c0a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4100b30 tx=0x7ff0a4009260 comp rx=0 tx=0).stop 2023-07-21T22:16:04.101 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 msgr2=0x7ff0b4195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c010 tx=0x7ff0a8005e80 comp rx=0 tx=0).stop 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 shutdown_connections 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b4101660 0x7ff0b419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ff09c0a11e0 0x7ff09c0a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0b40ffae0 0x7ff0b4195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 --2- 172.21.15.93:0/1628387282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0b4100cd0 0x7ff0b4195db0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 >> 172.21.15.93:0/1628387282 conn(0x7ff0b40fb120 msgr2=0x7ff0b40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 shutdown_connections 2023-07-21T22:16:04.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.057+0000 7ff0b88db700 1 -- 172.21.15.93:0/1628387282 wait complete. 2023-07-21T22:16:04.113 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:16:04.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- 172.21.15.93:0/2036889586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 msgr2=0x7f8cc00ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/2036889586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc00ff0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009990 tx=0x7f8ca8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- 172.21.15.93:0/2036889586 shutdown_connections 2023-07-21T22:16:04.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/2036889586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 0x7f8cc0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/2036889586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc01003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/2036889586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc00ff0a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- 172.21.15.93:0/2036889586 >> 172.21.15.93:0/2036889586 conn(0x7f8cc00f9770 msgr2=0x7f8cc00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- 172.21.15.93:0/2036889586 shutdown_connections 2023-07-21T22:16:04.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- 172.21.15.93:0/2036889586 wait complete. 2023-07-21T22:16:04.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 Processor -- start 2023-07-21T22:16:04.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.189+0000 7f8cc68e2700 1 -- start start 2023-07-21T22:16:04.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 0x7f8cc0194c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc019c250 con 0x7f8cc00fec50 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc019c390 con 0x7f8cc00fffb0 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc019c520 con 0x7f8cc00ff5e0 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43314/0 (socket says 172.21.15.93:43314) 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 -- 172.21.15.93:0/846166964 learned_addr learned my addr 172.21.15.93:0/846166964 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbf7fe700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 0x7f8cc0194c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc4e7f700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 -- 172.21.15.93:0/846166964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 msgr2=0x7f8cc0194c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 0x7f8cc0194c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 -- 172.21.15.93:0/846166964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 msgr2=0x7f8cc019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 -- 172.21.15.93:0/846166964 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80095f0 con 0x7f8cc00fec50 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc4e7f700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbffff700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d980 tx=0x7f8cb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000b950 con 0x7f8cc00fec50 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb0007620 con 0x7f8cc00fec50 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc019c800 con 0x7f8cc00fec50 2023-07-21T22:16:04.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000f3c0 con 0x7f8cc00fec50 2023-07-21T22:16:04.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.190+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc019cd60 con 0x7f8cc00fec50 2023-07-21T22:16:04.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.191+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc010a1f0 con 0x7f8cc00fec50 2023-07-21T22:16:04.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.192+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8cb0010640 con 0x7f8cc00fec50 2023-07-21T22:16:04.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.193+0000 7f8cbd7fa700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 0x7f8cac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.193+0000 7f8cbf7fe700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 0x7f8cac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.193+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8cb00cd150 con 0x7f8cc00fec50 2023-07-21T22:16:04.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.193+0000 7f8cbf7fe700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 0x7f8cac0a3750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800b2e0 tx=0x7f8ca8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.196+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8cb0096710 con 0x7f8cc00fec50 2023-07-21T22:16:04.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:04 smithi093 ceph-mon[96579]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:04.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:04 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4042668795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:04 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1921955071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:04 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1628387282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.431+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc0105920 con 0x7f8cc00fec50 2023-07-21T22:16:04.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.431+0000 7f8cbd7fa700 1 -- 172.21.15.93:0/846166964 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f8cb0016090 con 0x7f8cc00fec50 2023-07-21T22:16:04.476 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:04.476 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:16:04.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 msgr2=0x7f8cac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 0x7f8cac0a3750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800b2e0 tx=0x7f8ca8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 msgr2=0x7f8cc01946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d980 tx=0x7f8cb000dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:04.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 shutdown_connections 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f8cac0a12a0 0x7f8cac0a3750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cc00fffb0 0x7f8cc019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cc00fec50 0x7f8cc01946d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 --2- 172.21.15.93:0/846166964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cc00ff5e0 0x7f8cc0194c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.433+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 >> 172.21.15.93:0/846166964 conn(0x7f8cc00f9770 msgr2=0x7f8cc00fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.434+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 shutdown_connections 2023-07-21T22:16:04.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.434+0000 7f8cc68e2700 1 -- 172.21.15.93:0/846166964 wait complete. 2023-07-21T22:16:04.480 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:04.490 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:04.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:04 smithi194 ceph-mon[98288]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:04 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4042668795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:04 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1921955071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:04 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1628387282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 -- 172.21.15.93:0/2505222019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc0fe9d0 msgr2=0x7f6bfc0fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 --2- 172.21.15.93:0/2505222019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc0fede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009990 tx=0x7f6bec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 -- 172.21.15.93:0/2505222019 shutdown_connections 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 --2- 172.21.15.93:0/2505222019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc100550 0x7f6bfc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 --2- 172.21.15.93:0/2505222019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc0fede0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 --2- 172.21.15.93:0/2505222019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 0x7f6bfc100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 -- 172.21.15.93:0/2505222019 >> 172.21.15.93:0/2505222019 conn(0x7f6bfc0f9fd0 msgr2=0x7f6bfc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 -- 172.21.15.93:0/2505222019 shutdown_connections 2023-07-21T22:16:04.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.567+0000 7f6c02156700 1 -- 172.21.15.93:0/2505222019 wait complete. 2023-07-21T22:16:04.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 Processor -- start 2023-07-21T22:16:04.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 -- start start 2023-07-21T22:16:04.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 0x7f6bfc100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc101c80 con 0x7f6bfc100550 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc101df0 con 0x7f6bfc0ffbc0 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6c02156700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19f870 con 0x7f6bfc0fe9d0 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6bfbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6bfbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43332/0 (socket says 172.21.15.93:43332) 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.568+0000 7f6bfbfff700 1 -- 172.21.15.93:0/1175423883 learned_addr learned my addr 172.21.15.93:0/1175423883 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfb7fe700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfaffd700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 0x7f6bfc100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 -- 172.21.15.93:0/1175423883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 msgr2=0x7f6bfc102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 -- 172.21.15.93:0/1175423883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 msgr2=0x7f6bfc100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 0x7f6bfc100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 -- 172.21.15.93:0/1175423883 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0095f0 con 0x7f6bfc100550 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfb7fe700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bfbfff700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007ee0 tx=0x7f6bf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4017070 con 0x7f6bfc100550 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc19fb50 con 0x7f6bfc100550 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf400f040 con 0x7f6bfc100550 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005740 con 0x7f6bfc100550 2023-07-21T22:16:04.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.569+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc1a0010 con 0x7f6bfc100550 2023-07-21T22:16:04.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.570+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc10a1a0 con 0x7f6bfc100550 2023-07-21T22:16:04.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.571+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6bf40058a0 con 0x7f6bfc100550 2023-07-21T22:16:04.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.571+0000 7f6bf8ff9700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 0x7f6be40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.572+0000 7f6bfb7fe700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 0x7f6be40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.572+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6bf40cc300 con 0x7f6bfc100550 2023-07-21T22:16:04.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.572+0000 7f6bfb7fe700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 0x7f6be40a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.576+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6bf4095920 con 0x7f6bfc100550 2023-07-21T22:16:04.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:04 smithi134 ceph-mon[97873]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-21T22:16:04.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:04 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4042668795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:04 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1921955071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:04 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1628387282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:04.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.767+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bfc1a05c0 con 0x7f6bfc100550 2023-07-21T22:16:04.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.768+0000 7f6bf8ff9700 1 -- 172.21.15.93:0/1175423883 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7f6bf4098f40 con 0x7f6bfc100550 2023-07-21T22:16:04.813 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:04.813 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:04.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 msgr2=0x7f6be40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 0x7f6be40a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000c00 tx=0x7f6bec005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 msgr2=0x7f6bfc1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4007ee0 tx=0x7f6bf400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 shutdown_connections 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bfc100550 0x7f6bfc1014e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc0fe9d0 0x7f6bfc102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f6be40a1220 0x7f6be40a36d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 --2- 172.21.15.93:0/1175423883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6bfc0ffbc0 0x7f6bfc100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 >> 172.21.15.93:0/1175423883 conn(0x7f6bfc0f9fd0 msgr2=0x7f6bfc067710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 shutdown_connections 2023-07-21T22:16:04.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.770+0000 7f6c02156700 1 -- 172.21.15.93:0/1175423883 wait complete. 2023-07-21T22:16:04.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:16:04.828 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:16:04.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2420992051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec40ffc40 msgr2=0x7f0ec4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2420992051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec40ffc40 0x7f0ec4100090 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009990 tx=0x7f0ec0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2420992051 shutdown_connections 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2420992051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec41005d0 0x7f0ec4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2420992051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40fea50 0x7f0ec40fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2420992051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec40ffc40 0x7f0ec4100090 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2420992051 >> 172.21.15.93:0/2420992051 conn(0x7f0ec40fa070 msgr2=0x7f0ec40fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2420992051 shutdown_connections 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.907+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2420992051 wait complete. 2023-07-21T22:16:04.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 Processor -- start 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 -- start start 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec40fea50 0x7f0ec4102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40ffc40 0x7f0ec41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4101fc0 con 0x7f0ec41005d0 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4102130 con 0x7f0ec40fea50 2023-07-21T22:16:04.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec419f830 con 0x7f0ec40ffc40 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43346/0 (socket says 172.21.15.93:43346) 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 -- 172.21.15.93:0/2992281132 learned_addr learned my addr 172.21.15.93:0/2992281132 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec8d50700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40ffc40 0x7f0ec41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 -- 172.21.15.93:0/2992281132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40ffc40 msgr2=0x7f0ec41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40ffc40 0x7f0ec41012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 -- 172.21.15.93:0/2992281132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec40fea50 msgr2=0x7f0ec4102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9551700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec40fea50 0x7f0ec4102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec40fea50 0x7f0ec4102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 -- 172.21.15.93:0/2992281132 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00095f0 con 0x7f0ec41005d0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ec9d52700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc007ee0 tx=0x7f0ebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc017070 con 0x7f0ec41005d0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ebc00f040 con 0x7f0ec41005d0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec419fb10 con 0x7f0ec41005d0 2023-07-21T22:16:04.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.908+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc005740 con 0x7f0ec41005d0 2023-07-21T22:16:04.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.909+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec41a0070 con 0x7f0ec41005d0 2023-07-21T22:16:04.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.909+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec4106ae0 con 0x7f0ec41005d0 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.911+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0ebc0058a0 con 0x7f0ec41005d0 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.911+0000 7f0eba7fc700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 0x7f0eb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.911+0000 7f0ec9551700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 0x7f0eb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.911+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0ebc0cc870 con 0x7f0ec41005d0 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.911+0000 7f0ec9551700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 0x7f0eb00a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40ffaa0 tx=0x7f0eb4005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:04.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:04.914+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0ebc095e90 con 0x7f0ec41005d0 2023-07-21T22:16:05.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.105+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ec402c9e0 con 0x7f0ec41005d0 2023-07-21T22:16:05.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.106+0000 7f0eba7fc700 1 -- 172.21.15.93:0/2992281132 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ebc0994b0 con 0x7f0ec41005d0 2023-07-21T22:16:05.150 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:16:05.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 msgr2=0x7f0eb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 0x7f0eb00a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40ffaa0 tx=0x7f0eb4005cf0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 msgr2=0x7f0ec4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc007ee0 tx=0x7f0ebc00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 shutdown_connections 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ec41005d0 0x7f0ec4101820 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f0eb00a1260 0x7f0eb00a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ec40fea50 0x7f0ec4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 --2- 172.21.15.93:0/2992281132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ec40ffc40 0x7f0ec41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 >> 172.21.15.93:0/2992281132 conn(0x7f0ec40fa070 msgr2=0x7f0ec40fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 shutdown_connections 2023-07-21T22:16:05.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.108+0000 7f0ecb7b5700 1 -- 172.21.15.93:0/2992281132 wait complete. 2023-07-21T22:16:05.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1880643699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f00fe9e0 msgr2=0x7f57f00fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1880643699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f00fe9e0 0x7f57f00fedf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009990 tx=0x7f57e4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1880643699 shutdown_connections 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1880643699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f0100560 0x7f57f0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1880643699 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f00fe9e0 0x7f57f00fedf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1880643699 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00ffbd0 0x7f57f0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1880643699 >> 172.21.15.93:0/1880643699 conn(0x7f57f00fa000 msgr2=0x7f57f00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1880643699 shutdown_connections 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1880643699 wait complete. 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 Processor -- start 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 -- start start 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00fe9e0 0x7f57f0195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.244+0000 7f57f6dd0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f00ffbd0 0x7f57f0195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019d490 con 0x7f57f0100560 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019d5d0 con 0x7f57f00ffbd0 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019d760 con 0x7f57f00fe9e0 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43360/0 (socket says 172.21.15.93:43360) 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 -- 172.21.15.93:0/1600220498 learned_addr learned my addr 172.21.15.93:0/1600220498 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:05.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57effff700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f00ffbd0 0x7f57f0195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f4b6c700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00fe9e0 0x7f57f0195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 -- 172.21.15.93:0/1600220498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00fe9e0 msgr2=0x7f57f0195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00fe9e0 0x7f57f0195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 -- 172.21.15.93:0/1600220498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f00ffbd0 msgr2=0x7f57f0195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f00ffbd0 0x7f57f0195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 -- 172.21.15.93:0/1600220498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e40095f0 con 0x7f57f0100560 2023-07-21T22:16:05.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f536d700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57e8007ee0 tx=0x7f57e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017070 con 0x7f57f0100560 2023-07-21T22:16:05.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57e800f040 con 0x7f57f0100560 2023-07-21T22:16:05.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f019da40 con 0x7f57f0100560 2023-07-21T22:16:05.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8005740 con 0x7f57f0100560 2023-07-21T22:16:05.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.245+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f019df80 con 0x7f57f0100560 2023-07-21T22:16:05.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.246+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f0105960 con 0x7f57f0100560 2023-07-21T22:16:05.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.247+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f57e80058a0 con 0x7f57f0100560 2023-07-21T22:16:05.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.248+0000 7f57edffb700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 0x7f57e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.248+0000 7f57f4b6c700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 0x7f57e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.248+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f57e80cc360 con 0x7f57f0100560 2023-07-21T22:16:05.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.248+0000 7f57f4b6c700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 0x7f57e00a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000c00 tx=0x7f57e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.252+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f57e8095920 con 0x7f57f0100560 2023-07-21T22:16:05.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/846166964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1175423883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2992281132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:05.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.442+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57f002ca70 con 0x7f57f0100560 2023-07-21T22:16:05.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.443+0000 7f57edffb700 1 -- 172.21.15.93:0/1600220498 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57e8098f40 con 0x7f57f0100560 2023-07-21T22:16:05.488 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:16:05.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 msgr2=0x7f57e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 0x7f57e00a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57e4000c00 tx=0x7f57e4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 msgr2=0x7f57f019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57e8007ee0 tx=0x7f57e800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 shutdown_connections 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57f0100560 0x7f57f019cec0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57f00fe9e0 0x7f57f0195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f57e00a11e0 0x7f57e00a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 --2- 172.21.15.93:0/1600220498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57f00ffbd0 0x7f57f0195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 >> 172.21.15.93:0/1600220498 conn(0x7f57f00fa000 msgr2=0x7f57f00fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 shutdown_connections 2023-07-21T22:16:05.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.445+0000 7f57f6dd0700 1 -- 172.21.15.93:0/1600220498 wait complete. 2023-07-21T22:16:05.502 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:16:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/846166964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1175423883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2992281132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:05.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 -- 172.21.15.93:0/3993925622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac105810 msgr2=0x7fc2ac107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 --2- 172.21.15.93:0/3993925622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac105810 0x7fc2ac107c10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400b0a0 tx=0x7fc2a400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 -- 172.21.15.93:0/3993925622 shutdown_connections 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 --2- 172.21.15.93:0/3993925622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac105810 0x7fc2ac107c10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 --2- 172.21.15.93:0/3993925622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 --2- 172.21.15.93:0/3993925622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 0x7fc2ac1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 -- 172.21.15.93:0/3993925622 >> 172.21.15.93:0/3993925622 conn(0x7fc2ac0fa030 msgr2=0x7fc2ac0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 -- 172.21.15.93:0/3993925622 shutdown_connections 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.582+0000 7fc2b218b700 1 -- 172.21.15.93:0/3993925622 wait complete. 2023-07-21T22:16:05.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 Processor -- start 2023-07-21T22:16:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/846166964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1175423883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2992281132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:05.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 -- start start 2023-07-21T22:16:05.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 0x7fc2ac0fe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2ac105810 0x7fc2ac0feac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0ff260 con 0x7fc2ac1005d0 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0ff3d0 con 0x7fc2ac102ef0 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2b218b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac19f8a0 con 0x7fc2ac105810 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2ab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2ab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43370/0 (socket says 172.21.15.93:43370) 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2ab7fe700 1 -- 172.21.15.93:0/740072976 learned_addr learned my addr 172.21.15.93:0/740072976 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.583+0000 7fc2aaffd700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 0x7fc2ac0fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 -- 172.21.15.93:0/740072976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2ac105810 msgr2=0x7fc2ac0feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2ac105810 0x7fc2ac0feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 -- 172.21.15.93:0/740072976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 msgr2=0x7fc2ac0fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 0x7fc2ac0fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 -- 172.21.15.93:0/740072976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009d00 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2ab7fe700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c9b0 tx=0x7fc29c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c007b00 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29c007c60 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c018660 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac19fb80 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.584+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac1a0090 con 0x7fc2ac1005d0 2023-07-21T22:16:05.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.585+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac0fbba0 con 0x7fc2ac1005d0 2023-07-21T22:16:05.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.586+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc29c00f450 con 0x7fc2ac1005d0 2023-07-21T22:16:05.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.586+0000 7fc2a8ff9700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 0x7fc2940a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.587+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc29c0ccc90 con 0x7fc2ac1005d0 2023-07-21T22:16:05.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.587+0000 7fc2aaffd700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 0x7fc2940a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.587+0000 7fc2aaffd700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 0x7fc2940a3790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005950 tx=0x7fc2a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.590+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc29c096360 con 0x7fc2ac1005d0 2023-07-21T22:16:05.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.778+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2ac02ca00 con 0x7fc2ac1005d0 2023-07-21T22:16:05.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.778+0000 7fc2a8ff9700 1 -- 172.21.15.93:0/740072976 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc29c099980 con 0x7fc2ac1005d0 2023-07-21T22:16:05.823 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:16:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 msgr2=0x7fc2940a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 0x7fc2940a3790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0005950 tx=0x7fc2a000b410 comp rx=0 tx=0).stop 2023-07-21T22:16:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 msgr2=0x7fc2ac0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c9b0 tx=0x7fc29c00ccc0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 shutdown_connections 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2ac105810 0x7fc2ac0feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2ac1005d0 0x7fc2ac0fff40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc2940a12e0 0x7fc2940a3790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 --2- 172.21.15.93:0/740072976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2ac102ef0 0x7fc2ac0fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 >> 172.21.15.93:0/740072976 conn(0x7fc2ac0fa030 msgr2=0x7fc2ac101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 shutdown_connections 2023-07-21T22:16:05.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.780+0000 7fc2b218b700 1 -- 172.21.15.93:0/740072976 wait complete. 2023-07-21T22:16:05.837 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:16:05.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2998320439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc0fe9e0 msgr2=0x7fcefc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2998320439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc0fedf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009990 tx=0x7fcef0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2998320439 shutdown_connections 2023-07-21T22:16:05.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2998320439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc100560 0x7fcefc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2998320439 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc0fedf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2998320439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 0x7fcefc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2998320439 >> 172.21.15.93:0/2998320439 conn(0x7fcefc0fa000 msgr2=0x7fcefc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2998320439 shutdown_connections 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2998320439 wait complete. 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 Processor -- start 2023-07-21T22:16:05.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.914+0000 7fcf03fa1700 1 -- start start 2023-07-21T22:16:05.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 0x7fcefc195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc19d4a0 con 0x7fcefc100560 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc19d5e0 con 0x7fcefc0fe9e0 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc19d770 con 0x7fcefc0ffbd0 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43386/0 (socket says 172.21.15.93:43386) 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 -- 172.21.15.93:0/2335887777 learned_addr learned my addr 172.21.15.93:0/2335887777 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf01d3d700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0153c700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 0x7fcefc195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 -- 172.21.15.93:0/2335887777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 msgr2=0x7fcefc195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 0x7fcefc195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 -- 172.21.15.93:0/2335887777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc0fe9e0 msgr2=0x7fcefc195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 -- 172.21.15.93:0/2335887777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00095f0 con 0x7fcefc100560 2023-07-21T22:16:05.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf0253e700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcef4007ee0 tx=0x7fcef400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4017070 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcef400f040 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc19da50 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4005740 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.915+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc19df60 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.916+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc10a280 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.917+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcef400dc60 con 0x7fcefc100560 2023-07-21T22:16:05.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.918+0000 7fceeeffd700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 0x7fcee80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.918+0000 7fcf01d3d700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 0x7fcee80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.918+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcef40cc220 con 0x7fcefc100560 2023-07-21T22:16:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.919+0000 7fcf01d3d700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 0x7fcee80a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b5c0 tx=0x7fcef0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:05.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:05.923+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcef4095840 con 0x7fcefc100560 2023-07-21T22:16:06.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.154+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcefc19e570 con 0x7fcefc100560 2023-07-21T22:16:06.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.155+0000 7fceeeffd700 1 -- 172.21.15.93:0/2335887777 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7fcef4098e60 con 0x7fcefc100560 2023-07-21T22:16:06.200 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:06.200 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:16:06.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 msgr2=0x7fcee80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 0x7fcee80a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b5c0 tx=0x7fcef0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 msgr2=0x7fcefc19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcef4007ee0 tx=0x7fcef400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 shutdown_connections 2023-07-21T22:16:06.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcefc100560 0x7fcefc19ced0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fcee80a12a0 0x7fcee80a3750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcefc0fe9e0 0x7fcefc195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 --2- 172.21.15.93:0/2335887777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcefc0ffbd0 0x7fcefc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 >> 172.21.15.93:0/2335887777 conn(0x7fcefc0fa000 msgr2=0x7fcefc0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 shutdown_connections 2023-07-21T22:16:06.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.157+0000 7fcf03fa1700 1 -- 172.21.15.93:0/2335887777 wait complete. 2023-07-21T22:16:06.206 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:06.214 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:06.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 -- 172.21.15.93:0/2589776625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780fe8e0 msgr2=0x7fda780fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/2589776625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780fe8e0 0x7fda780fecf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fda68009990 tx=0x7fda68009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 -- 172.21.15.93:0/2589776625 shutdown_connections 2023-07-21T22:16:06.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/2589776625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda78100460 0x7fda78106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/2589776625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780fe8e0 0x7fda780fecf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/2589776625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda780ffad0 0x7fda780fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 -- 172.21.15.93:0/2589776625 >> 172.21.15.93:0/2589776625 conn(0x7fda780f9ee0 msgr2=0x7fda780fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 -- 172.21.15.93:0/2589776625 shutdown_connections 2023-07-21T22:16:06.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.293+0000 7fda7d1a2700 1 -- 172.21.15.93:0/2589776625 wait complete. 2023-07-21T22:16:06.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 Processor -- start 2023-07-21T22:16:06.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 -- start start 2023-07-21T22:16:06.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda780fe8e0 0x7fda78068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda78100460 0x7fda7819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819d0e0 con 0x7fda780ffad0 2023-07-21T22:16:06.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819d250 con 0x7fda780fe8e0 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7d1a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7819d410 con 0x7fda78100460 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43398/0 (socket says 172.21.15.93:43398) 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 -- 172.21.15.93:0/3939842661 learned_addr learned my addr 172.21.15.93:0/3939842661 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda77fff700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda780fe8e0 0x7fda78068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda7c9a1700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda78100460 0x7fda7819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 -- 172.21.15.93:0/3939842661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda78100460 msgr2=0x7fda7819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda78100460 0x7fda7819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 -- 172.21.15.93:0/3939842661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda780fe8e0 msgr2=0x7fda78068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda780fe8e0 0x7fda78068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 -- 172.21.15.93:0/3939842661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680095f0 con 0x7fda780ffad0 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.294+0000 7fda777fe700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e9a0 tx=0x7fda6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c009790 con 0x7fda780ffad0 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7819d6f0 con 0x7fda780ffad0 2023-07-21T22:16:06.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda6c00cbd0 con 0x7fda780ffad0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c018680 con 0x7fda780ffad0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7819dc00 con 0x7fda780ffad0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.295+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda78105d80 con 0x7fda780ffad0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.297+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fda6c00cd40 con 0x7fda780ffad0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.297+0000 7fda757fa700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 0x7fda600a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.297+0000 7fda77fff700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 0x7fda600a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.297+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fda6c014070 con 0x7fda780ffad0 2023-07-21T22:16:06.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.297+0000 7fda77fff700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 0x7fda600a3790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda68006010 tx=0x7fda68005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.300+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fda6c0965c0 con 0x7fda780ffad0 2023-07-21T22:16:06.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.491+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda7802c850 con 0x7fda780ffad0 2023-07-21T22:16:06.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.492+0000 7fda757fa700 1 -- 172.21.15.93:0/3939842661 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7fda6c099be0 con 0x7fda780ffad0 2023-07-21T22:16:06.537 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:06.537 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:06.539 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.494+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 msgr2=0x7fda600a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.539 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.494+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 0x7fda600a3790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fda68006010 tx=0x7fda68005960 comp rx=0 tx=0).stop 2023-07-21T22:16:06.539 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.494+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 msgr2=0x7fda7819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.494+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e9a0 tx=0x7fda6c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 shutdown_connections 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda78100460 0x7fda7819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fda600a12e0 0x7fda600a3790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda780fe8e0 0x7fda78068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 --2- 172.21.15.93:0/3939842661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda780ffad0 0x7fda7819a2b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 >> 172.21.15.93:0/3939842661 conn(0x7fda780f9ee0 msgr2=0x7fda78102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 shutdown_connections 2023-07-21T22:16:06.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.495+0000 7fda7d1a2700 1 -- 172.21.15.93:0/3939842661 wait complete. 2023-07-21T22:16:06.550 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:06 smithi093 ceph-mon[96579]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:06.551 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:06 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1600220498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.551 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:06 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/740072976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.551 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:06 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2335887777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:06.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:14:50.635520+0000', 'modified': '2023-07-21T22:14:58.012216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24484, 'mds_1': 24482, 'mds_2': 24460, 'mds_3': 24448, 'mds_4': 14589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14577': {'gid': 14577, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.93:6839/3482994338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6838', 'nonce': 3482994338}, {'type': 'v1', 'addr': '172.21.15.93:6839', 'nonce': 3482994338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14589': {'gid': 14589, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.93:6841/4095455709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6840', 'nonce': 4095455709}, {'type': 'v1', 'addr': '172.21.15.93:6841', 'nonce': 4095455709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/2616544286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2616544286}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2616544286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/3262717838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3262717838}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3262717838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/3279794938', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 3279794938}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 3279794938}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/117715569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 117715569}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 117715569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.194:6835/127707947', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 127707947}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 127707947}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.194:6837/714728190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 714728190}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 714728190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.134:6839/3043386858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3043386858}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3043386858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.194:6839/1819166614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 1819166614}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 1819166614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:16:06.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T22:16:06.555 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-21T22:16:06.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:06 smithi194 ceph-mon[98288]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:06.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:06 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1600220498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:06 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/740072976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:06 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2335887777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:06.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:06 smithi134 ceph-mon[97873]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:06.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:06 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1600220498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:06 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/740072976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:06.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:06 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2335887777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:06.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 -- 172.21.15.93:0/4123982987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00fffe0 msgr2=0x7fe3f01003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/4123982987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00fffe0 0x7fe3f01003f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009990 tx=0x7fe3e0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 -- 172.21.15.93:0/4123982987 shutdown_connections 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/4123982987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/4123982987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00fffe0 0x7fe3f01003f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/4123982987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fec30 0x7fe3f00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 -- 172.21.15.93:0/4123982987 >> 172.21.15.93:0/4123982987 conn(0x7fe3f00f9770 msgr2=0x7fe3f00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 -- 172.21.15.93:0/4123982987 shutdown_connections 2023-07-21T22:16:06.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.631+0000 7fe3f4f95700 1 -- 172.21.15.93:0/4123982987 wait complete. 2023-07-21T22:16:06.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.632+0000 7fe3f4f95700 1 Processor -- start 2023-07-21T22:16:06.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.632+0000 7fe3f4f95700 1 -- start start 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 0x7fe3f01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fffe0 0x7fe3f019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019d370 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019d4b0 con 0x7fe3f00fec30 2023-07-21T22:16:06.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019d640 con 0x7fe3f00fffe0 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43410/0 (socket says 172.21.15.93:43410) 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1232930383 learned_addr learned my addr 172.21.15.93:0/1232930383 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 0x7fe3f01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fffe0 0x7fe3f019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1232930383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fffe0 msgr2=0x7fe3f019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fffe0 0x7fe3f019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1232930383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 msgr2=0x7fe3f01957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 0x7fe3f01957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1232930383 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00095f0 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 0x7fe3f01957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:06.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c9b0 tx=0x7fe3e400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3df7fe700 1 -- 172.21.15.93:0/1232930383 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4007b00 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3df7fe700 1 -- 172.21.15.93:0/1232930383 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e4007c60 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f019d920 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3df7fe700 1 -- 172.21.15.93:0/1232930383 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4018660 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.633+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f019de80 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.635+0000 7fe3df7fe700 1 -- 172.21.15.93:0/1232930383 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe3e4018880 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.636+0000 7fe3df7fe700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 0x7fe3d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.636+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 0x7fe3d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.636+0000 7fe3df7fe700 1 -- 172.21.15.93:0/1232930383 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe3e40ccab0 con 0x7fe3f00ff5e0 2023-07-21T22:16:06.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.636+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 0x7fe3d80a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0006010 tx=0x7fe3e000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 Processor -- start 2023-07-21T22:16:06.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 -- start start 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31e7e0 0x55fc4a31ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a31d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a222c60 con 0x55fc4a31d9c0 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a2759f0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe40111ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a26ee80 con 0x55fc4a31f150 2023-07-21T22:16:06.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43418/0 (socket says 172.21.15.93:43418) 2023-07-21T22:16:06.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 -- 172.21.15.93:0/3758473892 learned_addr learned my addr 172.21.15.93:0/3758473892 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31e7e0 0x55fc4a31ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 -- 172.21.15.93:0/3758473892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 msgr2=0x55fc4a31d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a31d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.637+0000 7fe3eed9d700 1 -- 172.21.15.93:0/3758473892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31e7e0 msgr2=0x55fc4a31ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31e7e0 0x55fc4a31ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3eed9d700 1 -- 172.21.15.93:0/3758473892 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4a09cd90 con 0x55fc4a31d9c0 2023-07-21T22:16:06.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009f20 tx=0x7fe3e8009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafdab493bfda147 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3dcff9700 1 -- 172.21.15.93:0/3758473892 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8011070 con 0x55fc4a31d9c0 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3dcff9700 1 -- 172.21.15.93:0/3758473892 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e8009a50 con 0x55fc4a31d9c0 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe3dcff9700 1 -- 172.21.15.93:0/3758473892 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800a400 con 0x55fc4a31d9c0 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe40111ab80 1 -- 172.21.15.93:0/3758473892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 msgr2=0x55fc4a31de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.638+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009f20 tx=0x7fe3e8009f50 comp rx=0 tx=0).stop 2023-07-21T22:16:06.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- 172.21.15.93:0/3758473892 shutdown_connections 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31de40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31e7e0 0x55fc4a31ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3758473892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a31d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- 172.21.15.93:0/3758473892 >> 172.21.15.93:0/3758473892 conn(0x55fc4a312ea0 msgr2=0x55fc4a318310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- 172.21.15.93:0/3758473892 shutdown_connections 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- 172.21.15.93:0/3758473892 wait complete. 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- start start 2023-07-21T22:16:06.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 Processor -- start 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31d9c0 0x55fc4a3270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a32e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a20ea70 con 0x55fc4a31e7e0 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a20e770 con 0x55fc4a31d9c0 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a0f28d0 con 0x55fc4a31f150 2023-07-21T22:16:06.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe40111ab80 10 client.0 _ll_register_callbacks cb 0x55fc4a20c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43424/0 (socket says 172.21.15.93:43424) 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1863720174 learned_addr learned my addr 172.21.15.93:0/1863720174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.639+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31d9c0 0x55fc4a3270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a32e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1863720174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 msgr2=0x55fc4a32e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a32e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1863720174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31d9c0 msgr2=0x55fc4a3270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31d9c0 0x55fc4a3270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 -- 172.21.15.93:0/1863720174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e8009bd0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4096290 tx=0x7fe3e401f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0de2e1bd1d46ab5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4099480 con 0x55fc4a31e7e0 2023-07-21T22:16:06.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe3e409ddc0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc4a267e60 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e40a21e0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe40111ab80 10 client.15201 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc4a085c10 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc4a267e60 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3e4018c50 con 0x55fc4a31e7e0 2023-07-21T22:16:06.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe40111ab80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.640+0000 7fe40111ab80 10 client.15201 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe40111ab80 20 client.15201 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a332330 0x55fc4a3317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe40111ab80 4 client.15201 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc4a22a000 con 0x55fc4a332330 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a332330 0x55fc4a3317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a332330 0x55fc4a3317c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=150012f79bba793c server_cookie=6b255c2a96f819ef in_seq=0 out_seq=0 2023-07-21T22:16:06.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe3d5ffb700 10 client.15201 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/1863720174 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc4a22a000 con 0x55fc4a332330 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe3d5ffb700 10 client.15201 handle_command_reply: tid=0 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.641+0000 7fe40111ab80 1 client.15201 shutdown 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 msgr2=0x55fc4a327620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4096290 tx=0x7fe3e401f300 comp rx=0 tx=0).stop 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 shutdown_connections 2023-07-21T22:16:06.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a332330 0x55fc4a3317c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31f150 0x55fc4a32e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31d9c0 0x55fc4a3270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe3d5ffb700 0 client.15201 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 --2- 172.21.15.93:0/1863720174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a327620 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 >> 172.21.15.93:0/1863720174 conn(0x55fc4a312ea0 msgr2=0x55fc4a3179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 shutdown_connections 2023-07-21T22:16:06.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 1 -- 172.21.15.93:0/1863720174 wait complete. 2023-07-21T22:16:06.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.642+0000 7fe40111ab80 20 client.15201 trim_cache size 0 max 16384 2023-07-21T22:16:06.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 Processor -- start 2023-07-21T22:16:06.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 -- start start 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a325dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a218470 con 0x55fc4a31e7e0 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a23d790 con 0x55fc4a31f150 2023-07-21T22:16:06.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe40111ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a0fede0 con 0x55fc4a31d9c0 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43432/0 (socket says 172.21.15.93:43432) 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 -- 172.21.15.93:0/2245071203 learned_addr learned my addr 172.21.15.93:0/2245071203 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a325dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 -- 172.21.15.93:0/2245071203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 msgr2=0x55fc4a31d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 -- 172.21.15.93:0/2245071203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 msgr2=0x55fc4a325dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a325dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.659+0000 7fe3edd9b700 1 -- 172.21.15.93:0/2245071203 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4a0c3200 con 0x55fc4a31e7e0 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400afd0 tx=0x7fe3e4099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c2a6050f3be8d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe3dd7fa700 1 -- 172.21.15.93:0/2245071203 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e409ddc0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe3dd7fa700 1 -- 172.21.15.93:0/2245071203 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e4095c20 con 0x55fc4a31e7e0 2023-07-21T22:16:06.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe3dd7fa700 1 -- 172.21.15.93:0/2245071203 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40a1fb0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 -- 172.21.15.93:0/2245071203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 msgr2=0x55fc4a32e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400afd0 tx=0x7fe3e4099f10 comp rx=0 tx=0).stop 2023-07-21T22:16:06.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 -- 172.21.15.93:0/2245071203 shutdown_connections 2023-07-21T22:16:06.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a325dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a31d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 --2- 172.21.15.93:0/2245071203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32e620 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 -- 172.21.15.93:0/2245071203 >> 172.21.15.93:0/2245071203 conn(0x55fc4a312ea0 msgr2=0x55fc4a31dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 -- 172.21.15.93:0/2245071203 shutdown_connections 2023-07-21T22:16:06.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.660+0000 7fe40111ab80 1 -- 172.21.15.93:0/2245071203 wait complete. 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 -- start start 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 Processor -- start 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a32b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a32e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a32ea20 con 0x55fc4a31e7e0 2023-07-21T22:16:06.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a386190 con 0x55fc4a31f150 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4a386300 con 0x55fc4a31d9c0 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe40111ab80 10 client.0 _ll_register_callbacks cb 0x7fe3e40996f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43438/0 (socket says 172.21.15.93:43438) 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 -- 172.21.15.93:0/3015858603 learned_addr learned my addr 172.21.15.93:0/3015858603 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a32b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a32e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 -- 172.21.15.93:0/3015858603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 msgr2=0x55fc4a32b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a32b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 -- 172.21.15.93:0/3015858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 msgr2=0x55fc4a32e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a32e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 -- 172.21.15.93:0/3015858603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc4a085c10 con 0x55fc4a31e7e0 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3eed9d700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a32e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3edd9b700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400a700 tx=0x7fe3e4099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a0c12de3536dab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.661+0000 7fe3d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e409ddc0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fe3e40a21e0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc4a1c84a0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e4018d40 con 0x55fc4a31e7e0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 10 client.15207 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fc4a09b260 con 0x55fc4a31e7e0 2023-07-21T22:16:06.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc4a1c84a0 con 0x55fc4a31e7e0 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3e407e020 con 0x55fc4a31e7e0 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 10 client.15207 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 10 client.15207 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 20 client.15207 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a38ab30 0x55fc4a387550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 4 client.15207 mds_command: new command op to 14589 tid=0[{"prefix": "status"}] 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc4a0bdf20 con 0x55fc4a38ab30 2023-07-21T22:16:06.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a38ab30 0x55fc4a387550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3d5ffb700 10 client.15207 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.662+0000 7fe3ee59c700 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a38ab30 0x55fc4a387550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e22c71849ef357ef server_cookie=1ba2360bfe8ded1 in_seq=0 out_seq=0 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3d5ffb700 1 -- 172.21.15.93:0/3015858603 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fc4a0bdf20 con 0x55fc4a38ab30 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3d5ffb700 10 client.15207 handle_command_reply: tid=0 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 client.15207 shutdown 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 msgr2=0x55fc4a32ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400a700 tx=0x7fe3e4099560 comp rx=0 tx=0).stop 2023-07-21T22:16:06.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 shutdown_connections 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc4a31f150 0x55fc4a32e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55fc4a38ab30 0x55fc4a387550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fc4a31d9c0 0x55fc4a32b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 --2- 172.21.15.93:0/3015858603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fc4a31e7e0 0x55fc4a32ddf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3d5ffb700 0 client.15207 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 >> 172.21.15.93:0/3015858603 conn(0x55fc4a312ea0 msgr2=0x55fc4a3317c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 shutdown_connections 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 1 -- 172.21.15.93:0/3015858603 wait complete. 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe40111ab80 20 client.15207 trim_cache size 0 max 16384 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 4, 2023-07-21T22:16:06.722 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 14589, 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 69.668555726999998, 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-07-21T22:16:06.723 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:16:06.724 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 83.898193563999996 2023-07-21T22:16:06.724 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:06.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 msgr2=0x7fe3d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 0x7fe3d80a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0006010 tx=0x7fe3e000b540 comp rx=0 tx=0).stop 2023-07-21T22:16:06.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 msgr2=0x7fe3f0195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c9b0 tx=0x7fe3e400cd70 comp rx=0 tx=0).stop 2023-07-21T22:16:06.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 shutdown_connections 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3f00fffe0 0x7fe3f019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fe3d80a1260 0x7fe3d80a3710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3f00fec30 0x7fe3f01957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 --2- 172.21.15.93:0/1232930383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f00ff5e0 0x7fe3f0195d30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.663+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 >> 172.21.15.93:0/1232930383 conn(0x7fe3f00f9770 msgr2=0x7fe3f00fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.664+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 shutdown_connections 2023-07-21T22:16:06.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.664+0000 7fe3f4f95700 1 -- 172.21.15.93:0/1232930383 wait complete. 2023-07-21T22:16:06.727 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-21T22:16:06.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 -- 172.21.15.93:0/3760346075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40fe9e0 msgr2=0x7f5ee40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 --2- 172.21.15.93:0/3760346075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee40fedf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009990 tx=0x7f5ed4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 -- 172.21.15.93:0/3760346075 shutdown_connections 2023-07-21T22:16:06.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 --2- 172.21.15.93:0/3760346075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee4100560 0x7f5ee4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 --2- 172.21.15.93:0/3760346075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee40fedf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 --2- 172.21.15.93:0/3760346075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 -- 172.21.15.93:0/3760346075 >> 172.21.15.93:0/3760346075 conn(0x7f5ee40fa000 msgr2=0x7f5ee40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 -- 172.21.15.93:0/3760346075 shutdown_connections 2023-07-21T22:16:06.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.804+0000 7f5eea507700 1 -- 172.21.15.93:0/3760346075 wait complete. 2023-07-21T22:16:06.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 Processor -- start 2023-07-21T22:16:06.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 -- start start 2023-07-21T22:16:06.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee4102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4100560 0x7f5ee41014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4101c90 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4101e00 con 0x7f5ee4100560 2023-07-21T22:16:06.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5eea507700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee419f8b0 con 0x7f5ee40fe9e0 2023-07-21T22:16:06.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43452/0 (socket says 172.21.15.93:43452) 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 -- 172.21.15.93:0/2740116825 learned_addr learned my addr 172.21.15.93:0/2740116825 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee4102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 -- 172.21.15.93:0/2740116825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee40fe9e0 msgr2=0x7f5ee4102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4100560 0x7f5ee41014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee4102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 -- 172.21.15.93:0/2740116825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4100560 msgr2=0x7f5ee41014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4100560 0x7f5ee41014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 -- 172.21.15.93:0/2740116825 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40095f0 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.805+0000 7f5edbfff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c920 tx=0x7f5ecc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.806+0000 7f5ee1ffb700 1 -- 172.21.15.93:0/2740116825 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc007b00 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.806+0000 7f5ee1ffb700 1 -- 172.21.15.93:0/2740116825 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ecc004cf0 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.806+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee419fb90 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.806+0000 7f5ee1ffb700 1 -- 172.21.15.93:0/2740116825 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc005610 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.806+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee41a00a0 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.808+0000 7f5ee1ffb700 1 -- 172.21.15.93:0/2740116825 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5ecc004750 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.808+0000 7f5ee1ffb700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 0x7f5ed00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.808+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 0x7f5ed00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.808+0000 7f5ee1ffb700 1 -- 172.21.15.93:0/2740116825 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5ecc0cca70 con 0x7f5ee40ffbd0 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 0x7f5ed00a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4006010 tx=0x7f5ed400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 Processor -- start 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 -- start start 2023-07-21T22:16:06.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab769bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 0x55abab7683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab66dc60 con 0x55abab7689c0 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab6c09f0 con 0x55abab7697e0 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.809+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab6b9e80 con 0x55abab76a150 2023-07-21T22:16:06.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43462/0 (socket says 172.21.15.93:43462) 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/384052240 learned_addr learned my addr 172.21.15.93:0/384052240 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5edbfff700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 0x55abab7683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab769bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/384052240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 msgr2=0x55abab7683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 0x55abab7683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/384052240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 msgr2=0x55abab769bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab769bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/384052240 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abab4e7d90 con 0x55abab7689c0 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5edbfff700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 0x55abab7683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009c70 tx=0x7f5edc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3517c15a80cfed6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5eda7fc700 1 -- 172.21.15.93:0/384052240 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc014400 con 0x55abab7689c0 2023-07-21T22:16:06.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5eda7fc700 1 -- 172.21.15.93:0/384052240 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc014a20 con 0x55abab7689c0 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.810+0000 7f5eda7fc700 1 -- 172.21.15.93:0/384052240 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc013ab0 con 0x55abab7689c0 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- 172.21.15.93:0/384052240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 msgr2=0x55abab768e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009c70 tx=0x7f5edc009f80 comp rx=0 tx=0).stop 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- 172.21.15.93:0/384052240 shutdown_connections 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab768e40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab769bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/384052240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab76a150 0x55abab7683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- 172.21.15.93:0/384052240 >> 172.21.15.93:0/384052240 conn(0x55abab75dea0 msgr2=0x55abab763310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- 172.21.15.93:0/384052240 shutdown_connections 2023-07-21T22:16:06.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- 172.21.15.93:0/384052240 wait complete. 2023-07-21T22:16:06.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 -- start start 2023-07-21T22:16:06.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.811+0000 7f5ef668cb80 1 Processor -- start 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab772050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab772590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab659a70 con 0x55abab76a150 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab659770 con 0x55abab7697e0 2023-07-21T22:16:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab53d8d0 con 0x55abab7689c0 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 10 client.0 _ll_register_callbacks cb 0x55abab657710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43468/0 (socket says 172.21.15.93:43468) 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/1286520987 learned_addr learned my addr 172.21.15.93:0/1286520987 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab772050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 msgr2=0x55abab772050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab772050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/1286520987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 msgr2=0x55abab772590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5edbfff700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab772590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab772590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/1286520987 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc0098a0 con 0x55abab76a150 2023-07-21T22:16:06.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5edc017040 tx=0x7f5edc007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4286d0ec3b5ccf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ec37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc014600 con 0x55abab76a150 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5edc004e20 con 0x55abab76a150 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abab6134a0 con 0x55abab76a150 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc014760 con 0x55abab76a150 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 10 client.15219 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:06.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.812+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55abab4d0c10 con 0x55abab76a150 2023-07-21T22:16:06.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abab6134a0 con 0x55abab76a150 2023-07-21T22:16:06.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5edc005450 con 0x55abab76a150 2023-07-21T22:16:06.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:06.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 10 client.15219 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:16:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 20 client.15219 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab77d160 0x55abab77c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 4 client.15219 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abab6b2e60 con 0x55abab77d160 2023-07-21T22:16:06.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab77d160 0x55abab77c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ec37fe700 10 client.15219 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:06.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.813+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab77d160 0x55abab77c5c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94882d7c9ea63f7 server_cookie=9689ac2cf6e931f7 in_seq=0 out_seq=0 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ec37fe700 1 -- 172.21.15.93:0/1286520987 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55abab6b2e60 con 0x55abab77d160 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ec37fe700 10 client.15219 handle_command_reply: tid=0 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 client.15219 shutdown 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 msgr2=0x55abab7795d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5edc017040 tx=0x7f5edc007e80 comp rx=0 tx=0).stop 2023-07-21T22:16:06.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 shutdown_connections 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab77d160 0x55abab77c5c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7795d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab772050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/1286520987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab772590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ec37fe700 0 client.15219 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:06.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 >> 172.21.15.93:0/1286520987 conn(0x55abab75dea0 msgr2=0x55abab7629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 shutdown_connections 2023-07-21T22:16:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.814+0000 7f5ef668cb80 1 -- 172.21.15.93:0/1286520987 wait complete. 2023-07-21T22:16:06.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.815+0000 7f5ef668cb80 20 client.15219 trim_cache size 0 max 16384 2023-07-21T22:16:06.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 Processor -- start 2023-07-21T22:16:06.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 -- start start 2023-07-21T22:16:06.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab785ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab7794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab549de0 con 0x55abab76a150 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab678480 con 0x55abab7697e0 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab655d50 con 0x55abab7689c0 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43476/0 (socket says 172.21.15.93:43476) 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/2966955913 learned_addr learned my addr 172.21.15.93:0/2966955913 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5edbfff700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab7794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab785ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/2966955913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 msgr2=0x55abab785ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab785ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/2966955913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 msgr2=0x55abab7794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab7794f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5ee8aa4700 1 -- 172.21.15.93:0/2966955913 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abab50e200 con 0x55abab76a150 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.832+0000 7f5edbfff700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab7794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004b00 tx=0x7f5edc005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d1de8fdf8ebe14 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5edaffd700 1 -- 172.21.15.93:0/2966955913 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc007890 con 0x55abab76a150 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5edaffd700 1 -- 172.21.15.93:0/2966955913 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc013700 con 0x55abab76a150 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5edaffd700 1 -- 172.21.15.93:0/2966955913 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0148c0 con 0x55abab76a150 2023-07-21T22:16:06.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- 172.21.15.93:0/2966955913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 msgr2=0x55abab7629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004b00 tx=0x7f5edc005ad0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- 172.21.15.93:0/2966955913 shutdown_connections 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab76a150 0x55abab7629b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7689c0 0x55abab785ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/2966955913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab7697e0 0x55abab7794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- 172.21.15.93:0/2966955913 >> 172.21.15.93:0/2966955913 conn(0x55abab75dea0 msgr2=0x55abab7632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- 172.21.15.93:0/2966955913 shutdown_connections 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- 172.21.15.93:0/2966955913 wait complete. 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.833+0000 7f5ef668cb80 1 -- start start 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 Processor -- start 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7697e0 0x55abab76ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab76a150 0x55abab7d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab76f630 con 0x55abab7689c0 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab7d1560 con 0x55abab76a150 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abab7d1720 con 0x55abab7697e0 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ef668cb80 10 client.0 _ll_register_callbacks cb 0x7f5edc009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43478/0 (socket says 172.21.15.93:43478) 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 -- 172.21.15.93:0/149493498 learned_addr learned my addr 172.21.15.93:0/149493498 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:06.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee8aa4700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab76a150 0x55abab7d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5edbfff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7697e0 0x55abab76ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 -- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7697e0 msgr2=0x55abab76ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7697e0 0x55abab76ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 -- 172.21.15.93:0/149493498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab76a150 msgr2=0x55abab7d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab76a150 0x55abab7d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 -- 172.21.15.93:0/149493498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abab4d0c10 con 0x55abab7689c0 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ee3fff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40ffa30 tx=0x7f5ed4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e410bea57e69886a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:06.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.834+0000 7f5ec37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4033030 con 0x55abab7689c0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5ed4033190 con 0x55abab7689c0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5edc014a20 con 0x55abab7689c0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed4033350 con 0x55abab7689c0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 10 client.15225 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55abab4e6260 con 0x55abab7689c0 2023-07-21T22:16:06.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc014a20 con 0x55abab7689c0 2023-07-21T22:16:06.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ed4004690 con 0x55abab7689c0 2023-07-21T22:16:06.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:06.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:16:06.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 20 client.15225 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:06.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab7d6480 0x55abab7d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:06.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 4 client.15225 mds_command: new command op to 24448 tid=0[{"prefix": "status"}] 2023-07-21T22:16:06.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55abab675000 con 0x55abab7d6480 2023-07-21T22:16:06.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.835+0000 7f5edbfff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab7d6480 0x55abab7d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:06.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ec37fe700 10 client.15225 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:06.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5edbfff700 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab7d6480 0x55abab7d2db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36d981b0283d8a15 server_cookie=c37c381a92a1202b in_seq=0 out_seq=0 2023-07-21T22:16:06.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ec37fe700 1 -- 172.21.15.93:0/149493498 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55abab675000 con 0x55abab7d6480 2023-07-21T22:16:06.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ec37fe700 10 client.15225 handle_command_reply: tid=0 2023-07-21T22:16:06.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 client.15225 shutdown 2023-07-21T22:16:06.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 msgr2=0x55abab76ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40ffa30 tx=0x7f5ed4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 shutdown_connections 2023-07-21T22:16:06.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abab76a150 0x55abab7d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55abab7d6480 0x55abab7d2db0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55abab7689c0 0x55abab76ea00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ec37fe700 0 client.15225 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:06.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 --2- 172.21.15.93:0/149493498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abab7697e0 0x55abab76ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 >> 172.21.15.93:0/149493498 conn(0x55abab75dea0 msgr2=0x55abab77c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 shutdown_connections 2023-07-21T22:16:06.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 1 -- 172.21.15.93:0/149493498 wait complete. 2023-07-21T22:16:06.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.836+0000 7f5ef668cb80 20 client.15225 trim_cache size 0 max 16384 2023-07-21T22:16:06.897 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:06.897 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:16:06.897 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 3, 2023-07-21T22:16:06.897 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24448, 2023-07-21T22:16:06.897 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 70.561453982000003, 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 95.071383968000006 2023-07-21T22:16:06.898 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:06.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 msgr2=0x7f5ed00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 0x7f5ed00a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4006010 tx=0x7f5ed400b540 comp rx=0 tx=0).stop 2023-07-21T22:16:06.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 msgr2=0x7f5ee4100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:06.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c920 tx=0x7f5ecc00cce0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 shutdown_connections 2023-07-21T22:16:06.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f5ed00a11e0 0x7f5ed00a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4100560 0x7f5ee41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ee40fe9e0 0x7f5ee4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 --2- 172.21.15.93:0/2740116825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee40ffbd0 0x7f5ee4100fb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:06.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 >> 172.21.15.93:0/2740116825 conn(0x7f5ee40fa000 msgr2=0x7f5ee4068740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:06.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 shutdown_connections 2023-07-21T22:16:06.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.837+0000 7f5eea507700 1 -- 172.21.15.93:0/2740116825 wait complete. 2023-07-21T22:16:06.902 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-21T22:16:07.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 -- 172.21.15.93:0/322719627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0ffbd0 msgr2=0x7f715c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 --2- 172.21.15.93:0/322719627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0ffbd0 0x7f715c100020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f714c009990 tx=0x7f714c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 -- 172.21.15.93:0/322719627 shutdown_connections 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 --2- 172.21.15.93:0/322719627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 0x7f715c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 --2- 172.21.15.93:0/322719627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0fe9e0 0x7f715c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 --2- 172.21.15.93:0/322719627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0ffbd0 0x7f715c100020 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 -- 172.21.15.93:0/322719627 >> 172.21.15.93:0/322719627 conn(0x7f715c0fa000 msgr2=0x7f715c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 -- 172.21.15.93:0/322719627 shutdown_connections 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.979+0000 7f7161c38700 1 -- 172.21.15.93:0/322719627 wait complete. 2023-07-21T22:16:07.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 Processor -- start 2023-07-21T22:16:07.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 -- start start 2023-07-21T22:16:07.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 0x7f715c194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 0x7f715c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19c2c0 con 0x7f715c0fe9e0 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19c400 con 0x7f715c100560 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f7161c38700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c19c590 con 0x7f715c0ffbd0 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43492/0 (socket says 172.21.15.93:43492) 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 -- 172.21.15.93:0/2013369108 learned_addr learned my addr 172.21.15.93:0/2013369108 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715bfff700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 0x7f715c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715affd700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 0x7f715c194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 -- 172.21.15.93:0/2013369108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 msgr2=0x7f715c194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 0x7f715c194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 -- 172.21.15.93:0/2013369108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 msgr2=0x7f715c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 0x7f715c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715b7fe700 1 -- 172.21.15.93:0/2013369108 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0095f0 con 0x7f715c0fe9e0 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.980+0000 7f715affd700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 0x7f715c194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f715b7fe700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f714400c8f0 tx=0x7f714400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f7158ff9700 1 -- 172.21.15.93:0/2013369108 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144004220 con 0x7f715c0fe9e0 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f7158ff9700 1 -- 172.21.15.93:0/2013369108 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7144004380 con 0x7f715c0fe9e0 2023-07-21T22:16:07.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f7158ff9700 1 -- 172.21.15.93:0/2013369108 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144003c40 con 0x7f715c0fe9e0 2023-07-21T22:16:07.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c19c870 con 0x7f715c0fe9e0 2023-07-21T22:16:07.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.981+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c19cd30 con 0x7f715c0fe9e0 2023-07-21T22:16:07.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.983+0000 7f7158ff9700 1 -- 172.21.15.93:0/2013369108 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7144007740 con 0x7f715c0fe9e0 2023-07-21T22:16:07.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f7158ff9700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 0x7f71480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715affd700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 0x7f71480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f7158ff9700 1 -- 172.21.15.93:0/2013369108 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71440cc630 con 0x7f715c0fe9e0 2023-07-21T22:16:07.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715affd700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 0x7f71480a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b2e0 tx=0x7f714c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 Processor -- start 2023-07-21T22:16:07.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 -- start start 2023-07-21T22:16:07.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b254830 0x55832b254c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b2557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b157c60 con 0x55832b255da0 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b1aa9f0 con 0x55832b254830 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b1a3e80 con 0x55832b255180 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43504/0 (socket says 172.21.15.93:43504) 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 -- 172.21.15.93:0/1817505120 learned_addr learned my addr 172.21.15.93:0/1817505120 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 -- 172.21.15.93:0/1817505120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 msgr2=0x55832b2557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715b7fe700 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b254830 0x55832b254c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b2557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 -- 172.21.15.93:0/1817505120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b254830 msgr2=0x55832b254c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b254830 0x55832b254c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 -- 172.21.15.93:0/1817505120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832afd1d90 con 0x55832b255da0 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f715bfff700 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7154011680 tx=0x7f7154009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff40e8ea000cb5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f7151ffb700 1 -- 172.21.15.93:0/1817505120 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e040 con 0x55832b255da0 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f7151ffb700 1 -- 172.21.15.93:0/1817505120 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f715400f040 con 0x55832b255da0 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.985+0000 7f7151ffb700 1 -- 172.21.15.93:0/1817505120 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154004880 con 0x55832b255da0 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1817505120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 msgr2=0x55832b256210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7154011680 tx=0x7f7154009920 comp rx=0 tx=0).stop 2023-07-21T22:16:07.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1817505120 shutdown_connections 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b256210 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b254830 0x55832b254c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1817505120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b2557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1817505120 >> 172.21.15.93:0/1817505120 conn(0x55832b247ea0 msgr2=0x55832b24d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1817505120 shutdown_connections 2023-07-21T22:16:07.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1817505120 wait complete. 2023-07-21T22:16:07.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.986+0000 7f716ddbdb80 1 -- start start 2023-07-21T22:16:07.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 Processor -- start 2023-07-21T22:16:07.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b25e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b25e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b143a70 con 0x55832b255da0 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b143770 con 0x55832b255180 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b0278d0 con 0x55832b254830 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f716ddbdb80 10 client.0 _ll_register_callbacks cb 0x55832b141710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b25e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b25e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:58818/0 (socket says 172.21.15.93:58818) 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715b7fe700 1 -- 172.21.15.93:0/499653545 learned_addr learned my addr 172.21.15.93:0/499653545 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43510/0 (socket says 172.21.15.93:43510) 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715affd700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b25e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 -- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 msgr2=0x55832b25e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b25e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 -- 172.21.15.93:0/499653545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 msgr2=0x55832b25e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b25e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 -- 172.21.15.93:0/499653545 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154011330 con 0x55832b255da0 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715affd700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b25e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f715bfff700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7154012010 tx=0x7f7154003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b61e344c51fae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.987+0000 7f713affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e030 con 0x55832b255da0 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71540093a0 con 0x55832b255da0 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55832b0fd4a0 con 0x55832b255da0 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7154004ad0 con 0x55832b255da0 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 10 client.15237 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55832afbac10 con 0x55832b255da0 2023-07-21T22:16:07.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55832b0fd4a0 con 0x55832b255da0 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7154024020 con 0x55832b255da0 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 10 client.15237 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 20 client.15237 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b269970 0x55832b268850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 4 client.15237 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:07.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.988+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55832b19ce60 con 0x55832b269970 2023-07-21T22:16:07.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.989+0000 7f715b7fe700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b269970 0x55832b268850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.989+0000 7f713affd700 10 client.15237 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.989+0000 7f715b7fe700 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b269970 0x55832b268850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=951cad36c6ef9b65 server_cookie=d7836038863a2762 in_seq=0 out_seq=0 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f713affd700 1 -- 172.21.15.93:0/499653545 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55832b19ce60 con 0x55832b269970 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f713affd700 10 client.15237 handle_command_reply: tid=0 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 client.15237 shutdown 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 msgr2=0x55832b265800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7154012010 tx=0x7f7154003c30 comp rx=0 tx=0).stop 2023-07-21T22:16:07.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 shutdown_connections 2023-07-21T22:16:07.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b265800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b269970 0x55832b268850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b25e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/499653545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b25e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f713affd700 0 client.15237 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:07.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 >> 172.21.15.93:0/499653545 conn(0x55832b247ea0 msgr2=0x55832b24c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 shutdown_connections 2023-07-21T22:16:07.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 1 -- 172.21.15.93:0/499653545 wait complete. 2023-07-21T22:16:07.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:06.990+0000 7f716ddbdb80 20 client.15237 trim_cache size 0 max 16384 2023-07-21T22:16:07.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 Processor -- start 2023-07-21T22:16:07.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 -- start start 2023-07-21T22:16:07.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b266030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b268850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b255c20 con 0x55832b255da0 2023-07-21T22:16:07.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b26a5f0 con 0x55832b255180 2023-07-21T22:16:07.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.007+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b25d730 con 0x55832b254830 2023-07-21T22:16:07.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43518/0 (socket says 172.21.15.93:43518) 2023-07-21T22:16:07.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 -- 172.21.15.93:0/70322246 learned_addr learned my addr 172.21.15.93:0/70322246 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715affd700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b268850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715b7fe700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b266030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 -- 172.21.15.93:0/70322246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 msgr2=0x55832b266030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b266030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 -- 172.21.15.93:0/70322246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 msgr2=0x55832b268850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b268850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 -- 172.21.15.93:0/70322246 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55832aff8200 con 0x55832b255da0 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715affd700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b268850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.008+0000 7f715bfff700 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7154011680 tx=0x7f7154007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3185cb4ec7b6e7ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.010+0000 7f71527fc700 1 -- 172.21.15.93:0/70322246 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e040 con 0x55832b255da0 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.010+0000 7f71527fc700 1 -- 172.21.15.93:0/70322246 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7154003d50 con 0x55832b255da0 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.010+0000 7f71527fc700 1 -- 172.21.15.93:0/70322246 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154014480 con 0x55832b255da0 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- 172.21.15.93:0/70322246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 msgr2=0x55832b25c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7154011680 tx=0x7f7154007eb0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- 172.21.15.93:0/70322246 shutdown_connections 2023-07-21T22:16:07.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b255da0 0x55832b25c130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b254830 0x55832b266030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/70322246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255180 0x55832b268850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- 172.21.15.93:0/70322246 >> 172.21.15.93:0/70322246 conn(0x55832b247ea0 msgr2=0x55832b25cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- 172.21.15.93:0/70322246 shutdown_connections 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- 172.21.15.93:0/70322246 wait complete. 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 -- start start 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 Processor -- start 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.011+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b258320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 0x55832b2bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b2bf710 con 0x55832b254830 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b2bf850 con 0x55832b255da0 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55832b2bf990 con 0x55832b255180 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 10 client.0 _ll_register_callbacks cb 0x7f7154011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43520/0 (socket says 172.21.15.93:43520) 2023-07-21T22:16:07.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 -- 172.21.15.93:0/1745209400 learned_addr learned my addr 172.21.15.93:0/1745209400 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715affd700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b258320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715bfff700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 0x55832b2bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 -- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 msgr2=0x55832b258320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b258320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 -- 172.21.15.93:0/1745209400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 msgr2=0x55832b2bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 0x55832b2bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 -- 172.21.15.93:0/1745209400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55832afbac10 con 0x55832b254830 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715bfff700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 0x55832b2bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:07.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f715b7fe700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f71440998a0 tx=0x7f714400a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e97e3bc9171ecab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f713affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144097cb0 con 0x55832b254830 2023-07-21T22:16:07.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f71440ca260 con 0x55832b254830 2023-07-21T22:16:07.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55832b15f000 con 0x55832b254830 2023-07-21T22:16:07.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71440b7aa0 con 0x55832b254830 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 10 client.15243 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.012+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55832afd0260 con 0x55832b254830 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55832b15f000 con 0x55832b254830 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7144098e00 con 0x55832b254830 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 10 client.15243 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 20 client.15243 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b2c4640 0x55832b2c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 4 client.15243 mds_command: new command op to 24460 tid=0[{"prefix": "status"}] 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55832aff2f20 con 0x55832b2c4640 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f715affd700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b2c4640 0x55832b2c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f713affd700 10 client.15243 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:07.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.013+0000 7f715affd700 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b2c4640 0x55832b2c0fa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1676b253bb90e0fa server_cookie=71d42f26fb785347 in_seq=0 out_seq=0 2023-07-21T22:16:07.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f713affd700 1 -- 172.21.15.93:0/1745209400 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55832aff2f20 con 0x55832b2c4640 2023-07-21T22:16:07.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f713affd700 10 client.15243 handle_command_reply: tid=0 2023-07-21T22:16:07.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 client.15243 shutdown 2023-07-21T22:16:07.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 msgr2=0x55832b257de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f71440998a0 tx=0x7f714400a5d0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 shutdown_connections 2023-07-21T22:16:07.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55832b255da0 0x55832b2bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55832b254830 0x55832b257de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55832b2c4640 0x55832b2c0fa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 --2- 172.21.15.93:0/1745209400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55832b255180 0x55832b258320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f713affd700 0 client.15243 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:07.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 >> 172.21.15.93:0/1745209400 conn(0x55832b247ea0 msgr2=0x55832b268850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 shutdown_connections 2023-07-21T22:16:07.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 1 -- 172.21.15.93:0/1745209400 wait complete. 2023-07-21T22:16:07.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f716ddbdb80 20 client.15243 trim_cache size 0 max 16384 2023-07-21T22:16:07.076 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:07.076 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:16:07.076 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-07-21T22:16:07.076 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24460, 2023-07-21T22:16:07.076 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 71.745047075000002, 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-07-21T22:16:07.077 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:16:07.078 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 92.968672061999996 2023-07-21T22:16:07.078 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:07.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 msgr2=0x7f71480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 0x7f71480a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b2e0 tx=0x7f714c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 msgr2=0x7f715c194740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.014+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f714400c8f0 tx=0x7f714400cc00 comp rx=0 tx=0).stop 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 shutdown_connections 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f71480a11e0 0x7f71480a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f715c100560 0x7f715c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f715c0fe9e0 0x7f715c194740 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 --2- 172.21.15.93:0/2013369108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c0ffbd0 0x7f715c194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 >> 172.21.15.93:0/2013369108 conn(0x7f715c0fa000 msgr2=0x7f715c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 shutdown_connections 2023-07-21T22:16:07.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.015+0000 7f7161c38700 1 -- 172.21.15.93:0/2013369108 wait complete. 2023-07-21T22:16:07.081 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:16:07.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- 172.21.15.93:0/1146594675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9068100560 msgr2=0x7f9068106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- 172.21.15.93:0/1146594675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9068100560 0x7f9068106e20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f906000b0a0 tx=0x7f906000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- 172.21.15.93:0/1146594675 shutdown_connections 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- 172.21.15.93:0/1146594675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9068100560 0x7f9068106e20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- 172.21.15.93:0/1146594675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90680fe9e0 0x7f90680fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- 172.21.15.93:0/1146594675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680ffbd0 0x7f9068100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- 172.21.15.93:0/1146594675 >> 172.21.15.93:0/1146594675 conn(0x7f90680fa000 msgr2=0x7f90680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- 172.21.15.93:0/1146594675 shutdown_connections 2023-07-21T22:16:07.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- 172.21.15.93:0/1146594675 wait complete. 2023-07-21T22:16:07.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 Processor -- start 2023-07-21T22:16:07.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- start start 2023-07-21T22:16:07.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680fe9e0 0x7f9068068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 0x7f906819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819d450 con 0x7f90680ffbd0 2023-07-21T22:16:07.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819d5c0 con 0x7f9068100560 2023-07-21T22:16:07.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.159+0000 7f906ebfd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819d780 con 0x7f90680fe9e0 2023-07-21T22:16:07.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43534/0 (socket says 172.21.15.93:43534) 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 -- 172.21.15.93:0/222439868 learned_addr learned my addr 172.21.15.93:0/222439868 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f906d19a700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 0x7f906819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f906c999700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680fe9e0 0x7f9068068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 -- 172.21.15.93:0/222439868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680fe9e0 msgr2=0x7f9068068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680fe9e0 0x7f9068068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 -- 172.21.15.93:0/222439868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 msgr2=0x7f906819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 0x7f906819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 -- 172.21.15.93:0/222439868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009d00 con 0x7f90680ffbd0 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f906d19a700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 0x7f906819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9067fff700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f905400c920 tx=0x7f905400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9065ffb700 1 -- 172.21.15.93:0/222439868 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054007b00 con 0x7f90680ffbd0 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9065ffb700 1 -- 172.21.15.93:0/222439868 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9054004cf0 con 0x7f90680ffbd0 2023-07-21T22:16:07.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906819da60 con 0x7f90680ffbd0 2023-07-21T22:16:07.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f9065ffb700 1 -- 172.21.15.93:0/222439868 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054005610 con 0x7f90680ffbd0 2023-07-21T22:16:07.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.160+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906819df70 con 0x7f90680ffbd0 2023-07-21T22:16:07.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.162+0000 7f9065ffb700 1 -- 172.21.15.93:0/222439868 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9054004750 con 0x7f90680ffbd0 2023-07-21T22:16:07.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.163+0000 7f9065ffb700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 0x7f90580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.163+0000 7f9065ffb700 1 -- 172.21.15.93:0/222439868 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f90540cca70 con 0x7f90680ffbd0 2023-07-21T22:16:07.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.163+0000 7f906c999700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 0x7f90580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.163+0000 7f906c999700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 0x7f90580a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f905c005950 tx=0x7f905c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 Processor -- start 2023-07-21T22:16:07.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 -- start start 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc4ddc60 con 0x55f7fc5da150 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc5309f0 con 0x55f7fc5d89c0 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f907ad82b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc529e80 con 0x55f7fc5d97e0 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43542/0 (socket says 172.21.15.93:43542) 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 -- 172.21.15.93:0/238550061 learned_addr learned my addr 172.21.15.93:0/238550061 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f906c999700 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5d9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f906d19a700 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 -- 172.21.15.93:0/238550061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d97e0 msgr2=0x55f7fc5d9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5d9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 -- 172.21.15.93:0/238550061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 msgr2=0x55f7fc5d8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5d8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.164+0000 7f9067fff700 1 -- 172.21.15.93:0/238550061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fc357d90 con 0x55f7fc5da150 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f9067fff700 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90540074a0 tx=0x7f9054099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5700e4099af42716 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f9052ffd700 1 -- 172.21.15.93:0/238550061 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540a21a0 con 0x55f7fc5da150 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f9052ffd700 1 -- 172.21.15.93:0/238550061 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90540980f0 con 0x55f7fc5da150 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f9052ffd700 1 -- 172.21.15.93:0/238550061 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054099fb0 con 0x55f7fc5da150 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 -- 172.21.15.93:0/238550061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 msgr2=0x55f7fc5d83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f90540074a0 tx=0x7f9054099730 comp rx=0 tx=0).stop 2023-07-21T22:16:07.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 -- 172.21.15.93:0/238550061 shutdown_connections 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5d8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 --2- 172.21.15.93:0/238550061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5da150 0x55f7fc5d83e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.165+0000 7f907ad82b80 1 -- 172.21.15.93:0/238550061 >> 172.21.15.93:0/238550061 conn(0x55f7fc5cdea0 msgr2=0x55f7fc5d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 -- 172.21.15.93:0/238550061 shutdown_connections 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 -- 172.21.15.93:0/238550061 wait complete. 2023-07-21T22:16:07.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 -- start start 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 Processor -- start 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.166+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc4c9a70 con 0x55f7fc5d97e0 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc4c9770 con 0x55f7fc5da150 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc3ad8d0 con 0x55f7fc5d89c0 2023-07-21T22:16:07.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 10 client.0 _ll_register_callbacks cb 0x55f7fc4c7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43548/0 (socket says 172.21.15.93:43548) 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 -- 172.21.15.93:0/2984768810 learned_addr learned my addr 172.21.15.93:0/2984768810 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f906c999700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f906d19a700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 -- 172.21.15.93:0/2984768810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d89c0 msgr2=0x55f7fc5e20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 -- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 msgr2=0x55f7fc5e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 -- 172.21.15.93:0/2984768810 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906819df70 con 0x55f7fc5d97e0 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f906d19a700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9067fff700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f90540a1c70 tx=0x7f90540ca8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b140727eb73a9712 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9047fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540b8120 con 0x55f7fc5d97e0 2023-07-21T22:16:07.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f90540b24c0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7fc4834a0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9054099240 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 10 client.15255 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f7fc340c10 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.167+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7fc4834a0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f90540a1da0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 20 client.15255 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc5ece40 0x55f7fc5ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 4 client.15255 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7fc522e60 con 0x55f7fc5ece40 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.168+0000 7f906c999700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc5ece40 0x55f7fc5ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f9047fff700 10 client.15255 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f906c999700 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc5ece40 0x55f7fc5ec5a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=310872e57f3e217 server_cookie=14e3d4d19fc31023 in_seq=0 out_seq=0 2023-07-21T22:16:07.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f9047fff700 1 -- 172.21.15.93:0/2984768810 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7fc522e60 con 0x55f7fc5ece40 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f9047fff700 10 client.15255 handle_command_reply: tid=0 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 client.15255 shutdown 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 msgr2=0x55f7fc5e2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f90540a1c70 tx=0x7f90540ca8d0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 shutdown_connections 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc5ece40 0x55f7fc5ec5a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f9047fff700 0 client.15255 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 --2- 172.21.15.93:0/2984768810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e2630 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.169+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 >> 172.21.15.93:0/2984768810 conn(0x55f7fc5cdea0 msgr2=0x55f7fc5d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.170+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 shutdown_connections 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.170+0000 7f907ad82b80 1 -- 172.21.15.93:0/2984768810 wait complete. 2023-07-21T22:16:07.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.170+0000 7f907ad82b80 20 client.15255 trim_cache size 0 max 16384 2023-07-21T22:16:07.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 Processor -- start 2023-07-21T22:16:07.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 -- start start 2023-07-21T22:16:07.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc5ee120 con 0x55f7fc5d97e0 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc5d4010 con 0x55f7fc5d89c0 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f907ad82b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc5cb050 con 0x55f7fc5da150 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f9067fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43556/0 (socket says 172.21.15.93:43556) 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.187+0000 7f9067fff700 1 -- 172.21.15.93:0/326789749 learned_addr learned my addr 172.21.15.93:0/326789749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f906c999700 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 -- 172.21.15.93:0/326789749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 msgr2=0x55f7fc5e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 -- 172.21.15.93:0/326789749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 msgr2=0x55f7fc5e0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 -- 172.21.15.93:0/326789749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fc37e200 con 0x55f7fc5d97e0 2023-07-21T22:16:07.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f9067fff700 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9054099a50 tx=0x7f9054099ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb2dfb64a3b9aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f90537fe700 1 -- 172.21.15.93:0/326789749 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054096090 con 0x55f7fc5d97e0 2023-07-21T22:16:07.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f90537fe700 1 -- 172.21.15.93:0/326789749 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90540b24c0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f90537fe700 1 -- 172.21.15.93:0/326789749 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540994a0 con 0x55f7fc5d97e0 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 -- 172.21.15.93:0/326789749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 msgr2=0x55f7fc5e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9054099a50 tx=0x7f9054099ed0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 -- 172.21.15.93:0/326789749 shutdown_connections 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc5e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 --2- 172.21.15.93:0/326789749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e9600 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.188+0000 7f907ad82b80 1 -- 172.21.15.93:0/326789749 >> 172.21.15.93:0/326789749 conn(0x55f7fc5cdea0 msgr2=0x55f7fc5f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- 172.21.15.93:0/326789749 shutdown_connections 2023-07-21T22:16:07.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- 172.21.15.93:0/326789749 wait complete. 2023-07-21T22:16:07.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- start start 2023-07-21T22:16:07.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 Processor -- start 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc640cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc641200 con 0x55f7fc5d89c0 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc641340 con 0x55f7fc5d97e0 2023-07-21T22:16:07.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fc6414b0 con 0x55f7fc5da150 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f907ad82b80 10 client.0 _ll_register_callbacks cb 0x7f906000b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f906c999700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f906c999700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43560/0 (socket says 172.21.15.93:43560) 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f906c999700 1 -- 172.21.15.93:0/3780236617 learned_addr learned my addr 172.21.15.93:0/3780236617 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.189+0000 7f9067fff700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 -- 172.21.15.93:0/3780236617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 msgr2=0x55f7fc640cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc640cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 -- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d97e0 msgr2=0x55f7fc5e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 -- 172.21.15.93:0/3780236617 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7fc340c10 con 0x55f7fc5d89c0 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f906c999700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f905c009470 tx=0x7f905c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c9fce410ce91ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c028ce0 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f905c025070 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90540980f0 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f905c028e40 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 10 client.15261 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f7fc356260 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90540980f0 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f905c011630 con 0x55f7fc5d89c0 2023-07-21T22:16:07.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 20 client.15261 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc6462e0 0x55f7fc642bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 4 client.15261 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.190+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7fc4e5000 con 0x55f7fc6462e0 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9067fff700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc6462e0 0x55f7fc642bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9047fff700 10 client.15261 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:07.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9067fff700 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc6462e0 0x55f7fc642bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d83e05823ed9a04 server_cookie=7a03578bfac5b7de in_seq=0 out_seq=0 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9047fff700 1 -- 172.21.15.93:0/3780236617 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f7fc4e5000 con 0x55f7fc6462e0 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9047fff700 10 client.15261 handle_command_reply: tid=0 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 client.15261 shutdown 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 msgr2=0x55f7fc5e4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f905c009470 tx=0x7f905c0047c0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 shutdown_connections 2023-07-21T22:16:07.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f7fc6462e0 0x55f7fc642bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7fc5da150 0x55f7fc640cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f9047fff700 0 client.15261 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fc5d89c0 0x55f7fc5e4800 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 --2- 172.21.15.93:0/3780236617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f7fc5d97e0 0x55f7fc5e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.191+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 >> 172.21.15.93:0/3780236617 conn(0x55f7fc5cdea0 msgr2=0x55f7fc5e0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 shutdown_connections 2023-07-21T22:16:07.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f907ad82b80 1 -- 172.21.15.93:0/3780236617 wait complete. 2023-07-21T22:16:07.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f907ad82b80 20 client.15261 trim_cache size 0 max 16384 2023-07-21T22:16:07.249 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:07.249 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:16:07.249 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-07-21T22:16:07.249 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24482, 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 72.934591816999998, 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-07-21T22:16:07.250 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:16:07.251 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 82.275412770000003 2023-07-21T22:16:07.251 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:07.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 msgr2=0x7f90580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 0x7f90580a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f905c005950 tx=0x7f905c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 msgr2=0x7f906819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f905400c920 tx=0x7f905400cce0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 shutdown_connections 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f90580a11e0 0x7f90580a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9068100560 0x7f906819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90680fe9e0 0x7f9068068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 --2- 172.21.15.93:0/222439868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f90680ffbd0 0x7f906819a630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 >> 172.21.15.93:0/222439868 conn(0x7f90680fa000 msgr2=0x7f9068103040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 shutdown_connections 2023-07-21T22:16:07.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.192+0000 7f906ebfd700 1 -- 172.21.15.93:0/222439868 wait complete. 2023-07-21T22:16:07.254 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:16:07.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 -- 172.21.15.93:0/2265153543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 msgr2=0x7fc4e00fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 --2- 172.21.15.93:0/2265153543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e00fff60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009990 tx=0x7fc4d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 -- 172.21.15.93:0/2265153543 shutdown_connections 2023-07-21T22:16:07.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 --2- 172.21.15.93:0/2265153543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e01016d0 0x7fc4e0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 --2- 172.21.15.93:0/2265153543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e00fff60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 --2- 172.21.15.93:0/2265153543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e0100d40 0x7fc4e0101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 -- 172.21.15.93:0/2265153543 >> 172.21.15.93:0/2265153543 conn(0x7fc4e00fb170 msgr2=0x7fc4e00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 -- 172.21.15.93:0/2265153543 shutdown_connections 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.332+0000 7fc4e6496700 1 -- 172.21.15.93:0/2265153543 wait complete. 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 Processor -- start 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 -- start start 2023-07-21T22:16:07.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0100d40 0x7fc4e0195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e01016d0 0x7fc4e019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d3c0 con 0x7fc4e00ffb50 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d500 con 0x7fc4e01016d0 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e6496700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d690 con 0x7fc4e0100d40 2023-07-21T22:16:07.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43574/0 (socket says 172.21.15.93:43574) 2023-07-21T22:16:07.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 -- 172.21.15.93:0/313756206 learned_addr learned my addr 172.21.15.93:0/313756206 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0100d40 0x7fc4e0195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e01016d0 0x7fc4e019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 -- 172.21.15.93:0/313756206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0100d40 msgr2=0x7fc4e0195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0100d40 0x7fc4e0195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 -- 172.21.15.93:0/313756206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e01016d0 msgr2=0x7fc4e019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e01016d0 0x7fc4e019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 -- 172.21.15.93:0/313756206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00095f0 con 0x7fc4e00ffb50 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.333+0000 7fc4dffff700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.334+0000 7fc4dd7fa700 1 -- 172.21.15.93:0/313756206 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d001d070 con 0x7fc4e00ffb50 2023-07-21T22:16:07.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.334+0000 7fc4dd7fa700 1 -- 172.21.15.93:0/313756206 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d0004180 con 0x7fc4e00ffb50 2023-07-21T22:16:07.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.334+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e019d910 con 0x7fc4e00ffb50 2023-07-21T22:16:07.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.334+0000 7fc4dd7fa700 1 -- 172.21.15.93:0/313756206 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0022e00 con 0x7fc4e00ffb50 2023-07-21T22:16:07.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.334+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e019de10 con 0x7fc4e00ffb50 2023-07-21T22:16:07.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.336+0000 7fc4dd7fa700 1 -- 172.21.15.93:0/313756206 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc4d0022500 con 0x7fc4e00ffb50 2023-07-21T22:16:07.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.336+0000 7fc4dd7fa700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 0x7fc4c80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.337+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 0x7fc4c80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.337+0000 7fc4dd7fa700 1 -- 172.21.15.93:0/313756206 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc4d00cdc90 con 0x7fc4e00ffb50 2023-07-21T22:16:07.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.337+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 0x7fc4c80a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4005950 tx=0x7fc4d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 Processor -- start 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 -- start start 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf3dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf3c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae41c60 con 0x561bdaf3c9c0 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae949f0 con 0x561bdaf3e150 2023-07-21T22:16:07.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae8de80 con 0x561bdaf3d7e0 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43586/0 (socket says 172.21.15.93:43586) 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 -- 172.21.15.93:0/3329092220 learned_addr learned my addr 172.21.15.93:0/3329092220 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4dffff700 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf3dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 -- 172.21.15.93:0/3329092220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 msgr2=0x561bdaf3dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf3dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 -- 172.21.15.93:0/3329092220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 msgr2=0x561bdaf3c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf3c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf3c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 -- 172.21.15.93:0/3329092220 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdacbbd90 con 0x561bdaf3c9c0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009dc0 tx=0x7fc4d80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d90a09ccdff86d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4c67fc700 1 -- 172.21.15.93:0/3329092220 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8014610 con 0x561bdaf3c9c0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4c67fc700 1 -- 172.21.15.93:0/3329092220 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d800f050 con 0x561bdaf3c9c0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.338+0000 7fc4c67fc700 1 -- 172.21.15.93:0/3329092220 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80139a0 con 0x561bdaf3c9c0 2023-07-21T22:16:07.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- 172.21.15.93:0/3329092220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 msgr2=0x561bdaf3ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009dc0 tx=0x7fc4d80077e0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- 172.21.15.93:0/3329092220 shutdown_connections 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3ce40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf3dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/3329092220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf3c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- 172.21.15.93:0/3329092220 >> 172.21.15.93:0/3329092220 conn(0x561bdaf31ea0 msgr2=0x561bdaf37310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- 172.21.15.93:0/3329092220 shutdown_connections 2023-07-21T22:16:07.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- 172.21.15.93:0/3329092220 wait complete. 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.339+0000 7fc4f261bb80 1 -- start start 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 Processor -- start 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae2da70 con 0x561bdaf3c9c0 2023-07-21T22:16:07.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae2d770 con 0x561bdaf3e150 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdad118d0 con 0x561bdaf3d7e0 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4f261bb80 10 client.0 _ll_register_callbacks cb 0x561bdae2b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43588/0 (socket says 172.21.15.93:43588) 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4dffff700 1 -- 172.21.15.93:0/2525437791 learned_addr learned my addr 172.21.15.93:0/2525437791 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.340+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 -- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 msgr2=0x561bdaf465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 -- 172.21.15.93:0/2525437791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 msgr2=0x561bdaf4d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 -- 172.21.15.93:0/2525437791 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d8009a20 con 0x561bdaf3c9c0 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4dffff700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c010 tx=0x7fc4d0005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ab0f379a729da server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a3130 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc4d009b000 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bdade74a0 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d009f030 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 10 client.15273 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561bdaca4c10 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d00c9080 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4d0096f80 con 0x561bdaf3c9c0 2023-07-21T22:16:07.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 10 client.15273 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 20 client.15273 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdaf50f70 0x561bdaf50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 4 client.15273 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.341+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bdae86e60 con 0x561bdaf50f70 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.342+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdaf50f70 0x561bdaf50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.342+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdaf50f70 0x561bdaf50540 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afaaffa6c8431ca8 server_cookie=5b109ed05413fdc7 in_seq=0 out_seq=0 2023-07-21T22:16:07.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.342+0000 7fc4bf7fe700 10 client.15273 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/2525437791 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561bdae86e60 con 0x561bdaf50f70 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4bf7fe700 10 client.15273 handle_command_reply: tid=0 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 client.15273 shutdown 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 msgr2=0x561bdaf46090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c010 tx=0x7fc4d0005ad0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 shutdown_connections 2023-07-21T22:16:07.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdaf50f70 0x561bdaf50540 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf46090 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4bf7fe700 0 client.15273 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/2525437791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 >> 172.21.15.93:0/2525437791 conn(0x561bdaf31ea0 msgr2=0x561bdaf369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 shutdown_connections 2023-07-21T22:16:07.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 1 -- 172.21.15.93:0/2525437791 wait complete. 2023-07-21T22:16:07.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.343+0000 7fc4f261bb80 20 client.15273 trim_cache size 0 max 16384 2023-07-21T22:16:07.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 Processor -- start 2023-07-21T22:16:07.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 -- start start 2023-07-21T22:16:07.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3d7e0 0x561bdaf4d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdaf45c60 con 0x561bdaf3e150 2023-07-21T22:16:07.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdad1dde0 con 0x561bdaf3d7e0 2023-07-21T22:16:07.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdae4c480 con 0x561bdaf3c9c0 2023-07-21T22:16:07.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43600/0 (socket says 172.21.15.93:43600) 2023-07-21T22:16:07.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 -- 172.21.15.93:0/897622734 learned_addr learned my addr 172.21.15.93:0/897622734 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4dffff700 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 -- 172.21.15.93:0/897622734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3c9c0 msgr2=0x561bdaf3c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 -- 172.21.15.93:0/897622734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3d7e0 msgr2=0x561bdaf4d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3d7e0 0x561bdaf4d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 -- 172.21.15.93:0/897622734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bdaf47260 con 0x561bdaf3e150 2023-07-21T22:16:07.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80090b0 tx=0x7fc4d8009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506d698abca3baa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4c6ffd700 1 -- 172.21.15.93:0/897622734 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8023070 con 0x561bdaf3e150 2023-07-21T22:16:07.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4c6ffd700 1 -- 172.21.15.93:0/897622734 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d8004b80 con 0x561bdaf3e150 2023-07-21T22:16:07.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.361+0000 7fc4c6ffd700 1 -- 172.21.15.93:0/897622734 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8028680 con 0x561bdaf3e150 2023-07-21T22:16:07.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- 172.21.15.93:0/897622734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 msgr2=0x561bdaf59d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80090b0 tx=0x7fc4d8009970 comp rx=0 tx=0).stop 2023-07-21T22:16:07.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- 172.21.15.93:0/897622734 shutdown_connections 2023-07-21T22:16:07.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3e150 0x561bdaf59d00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3c9c0 0x561bdaf3c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/897622734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3d7e0 0x561bdaf4d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- 172.21.15.93:0/897622734 >> 172.21.15.93:0/897622734 conn(0x561bdaf31ea0 msgr2=0x561bdaf44dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- 172.21.15.93:0/897622734 shutdown_connections 2023-07-21T22:16:07.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- 172.21.15.93:0/897622734 wait complete. 2023-07-21T22:16:07.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.362+0000 7fc4f261bb80 1 -- start start 2023-07-21T22:16:07.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 Processor -- start 2023-07-21T22:16:07.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdaf4c080 con 0x561bdaf3c9c0 2023-07-21T22:16:07.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdaf4c1f0 con 0x561bdaf3e150 2023-07-21T22:16:07.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bdaf4c3b0 con 0x561bdaf3d7e0 2023-07-21T22:16:07.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4f261bb80 10 client.0 _ll_register_callbacks cb 0x7fc4d009aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:07.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43602/0 (socket says 172.21.15.93:43602) 2023-07-21T22:16:07.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 -- 172.21.15.93:0/1192244486 learned_addr learned my addr 172.21.15.93:0/1192244486 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:07.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4e4a33700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 -- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 msgr2=0x561bdaf48df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf48df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 -- 172.21.15.93:0/1192244486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 msgr2=0x561bdaf4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:07.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.429 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 -- 172.21.15.93:0/1192244486 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bdacbbd90 con 0x561bdaf3c9c0 2023-07-21T22:16:07.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4dffff700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0003940 tx=0x7fc4d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd16830d2d94998d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:07.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:07.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.363+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a3130 con 0x561bdaf3c9c0 2023-07-21T22:16:07.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc4d00ca060 con 0x561bdaf3c9c0 2023-07-21T22:16:07.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d800f050 con 0x561bdaf3c9c0 2023-07-21T22:16:07.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d0097340 con 0x561bdaf3c9c0 2023-07-21T22:16:07.431 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 10 client.15279 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:07.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561bdace2200 con 0x561bdaf3c9c0 2023-07-21T22:16:07.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d800f050 con 0x561bdaf3c9c0 2023-07-21T22:16:07.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4d0096df0 con 0x561bdaf3c9c0 2023-07-21T22:16:07.432 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:07.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:16:07.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 20 client.15279 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:07.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdafae270 0x561bdafab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:07.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 4 client.15279 mds_command: new command op to 24484 tid=0[{"prefix": "status"}] 2023-07-21T22:16:07.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561bdae49000 con 0x561bdafae270 2023-07-21T22:16:07.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.364+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdafae270 0x561bdafab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:07.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4bf7fe700 10 client.15279 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:07.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4df7fe700 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdafae270 0x561bdafab250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b2ef60023a1744 server_cookie=a0c7d3ab3b16628a in_seq=0 out_seq=0 2023-07-21T22:16:07.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4bf7fe700 1 -- 172.21.15.93:0/1192244486 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561bdae49000 con 0x561bdafae270 2023-07-21T22:16:07.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4bf7fe700 10 client.15279 handle_command_reply: tid=0 2023-07-21T22:16:07.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 client.15279 shutdown 2023-07-21T22:16:07.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 msgr2=0x561bdaf48810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0003940 tx=0x7fc4d00042f0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 shutdown_connections 2023-07-21T22:16:07.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bdafae270 0x561bdafab250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bdaf3e150 0x561bdaf4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4bf7fe700 0 client.15279 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:07.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bdaf3c9c0 0x561bdaf48810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 --2- 172.21.15.93:0/1192244486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bdaf3d7e0 0x561bdaf48df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 >> 172.21.15.93:0/1192244486 conn(0x561bdaf31ea0 msgr2=0x561bdaf59d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 shutdown_connections 2023-07-21T22:16:07.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 1 -- 172.21.15.93:0/1192244486 wait complete. 2023-07-21T22:16:07.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.365+0000 7fc4f261bb80 20 client.15279 trim_cache size 0 max 16384 2023-07-21T22:16:07.439 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:07.439 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "de1cec7a-2812-11ee-9b34-001a4aab830c", 2023-07-21T22:16:07.440 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-07-21T22:16:07.440 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24484, 2023-07-21T22:16:07.440 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-07-21T22:16:07.440 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-07-21T22:16:07.441 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-07-21T22:16:07.441 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 76.123015901000002, 2023-07-21T22:16:07.441 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 26, 2023-07-21T22:16:07.441 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-07-21T22:16:07.442 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:16:07.442 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 88.820931642999994 2023-07-21T22:16:07.442 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:07.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 msgr2=0x7fc4c80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 0x7fc4c80a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4005950 tx=0x7fc4d40058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 msgr2=0x7fc4e0195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:07.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0009f90 comp rx=0 tx=0).stop 2023-07-21T22:16:07.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 shutdown_connections 2023-07-21T22:16:07.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc4c80a1260 0x7fc4c80a3710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e01016d0 0x7fc4e019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4e00ffb50 0x7fc4e0195840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 --2- 172.21.15.93:0/313756206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0100d40 0x7fc4e0195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:07.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 >> 172.21.15.93:0/313756206 conn(0x7fc4e00fb170 msgr2=0x7fc4e00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:07.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 shutdown_connections 2023-07-21T22:16:07.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:07.366+0000 7fc4e6496700 1 -- 172.21.15.93:0/313756206 wait complete. 2023-07-21T22:16:07.448 INFO:teuthology.run_tasks:Running task kclient... 2023-07-21T22:16:07.480 INFO:tasks.kclient:Mounting kernel clients... 2023-07-21T22:16:07.480 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-21T22:16:07.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:07.481 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-07-21T22:16:07.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:07.499 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-07-21T22:16:07.561 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:07 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3939842661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:07.567 INFO:teuthology.orchestra.run.smithi093.stderr:Cannot find device "ceph-brx" 2023-07-21T22:16:07.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:16:07.568 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-07-21T22:16:07.569 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T22:16:07.569 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:07.569 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T22:16:07.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:07 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3939842661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:07.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:07 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3939842661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:08.159 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:16:08.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:08 smithi093 ceph-mon[96579]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:08.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:08 smithi134 ceph-mon[97873]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:08.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:08 smithi194 ceph-mon[98288]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:16:09.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 -- 172.21.15.93:0/2934364524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300feab0 msgr2=0x7f20300fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:09.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 --2- 172.21.15.93:0/2934364524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300feab0 0x7f20300fef20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2024009990 tx=0x7f2024009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 -- 172.21.15.93:0/2934364524 shutdown_connections 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 --2- 172.21.15.93:0/2934364524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300ff460 0x7f203010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 --2- 172.21.15.93:0/2934364524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2030104ab0 0x7f2030104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 --2- 172.21.15.93:0/2934364524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300feab0 0x7f20300fef20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 -- 172.21.15.93:0/2934364524 >> 172.21.15.93:0/2934364524 conn(0x7f2030075b70 msgr2=0x7f2030075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 -- 172.21.15.93:0/2934364524 shutdown_connections 2023-07-21T22:16:09.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.188+0000 7f2036854700 1 -- 172.21.15.93:0/2934364524 wait complete. 2023-07-21T22:16:09.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 Processor -- start 2023-07-21T22:16:09.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 -- start start 2023-07-21T22:16:09.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300feab0 0x7f203019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:09.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:09.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2030104ab0 0x7f2030195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030196130 con 0x7f20300ff460 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301962a0 con 0x7f20300feab0 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f2036854700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030196460 con 0x7f2030104ab0 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43616/0 (socket says 172.21.15.93:43616) 2023-07-21T22:16:09.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f202f7fe700 1 -- 172.21.15.93:0/2753899242 learned_addr learned my addr 172.21.15.93:0/2753899242 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.189+0000 7f202ffff700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300feab0 0x7f203019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 -- 172.21.15.93:0/2753899242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2030104ab0 msgr2=0x7f2030195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2030104ab0 0x7f2030195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 -- 172.21.15.93:0/2753899242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300feab0 msgr2=0x7f203019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300feab0 0x7f203019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 -- 172.21.15.93:0/2753899242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240095f0 con 0x7f20300ff460 2023-07-21T22:16:09.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202f7fe700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f20240095c0 tx=0x7f202400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202401d070 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202400fb50 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20301966e0 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20240173f0 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.190+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20301a2a30 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.191+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203004e210 con 0x7f20300ff460 2023-07-21T22:16:09.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.192+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f202400fcc0 con 0x7f20300ff460 2023-07-21T22:16:09.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.192+0000 7f202d7fa700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 0x7f20180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:09.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.192+0000 7f202ffff700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 0x7f20180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:09.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.193+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20240cdb10 con 0x7f20300ff460 2023-07-21T22:16:09.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.193+0000 7f202ffff700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 0x7f20180a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20300ffbf0 tx=0x7f2020009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:09.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.197+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f202405c770 con 0x7f20300ff460 2023-07-21T22:16:09.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.425+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2030196fd0 con 0x7f20300ff460 2023-07-21T22:16:09.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.425+0000 7f202d7fa700 1 -- 172.21.15.93:0/2753899242 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2024027070 con 0x7f20300ff460 2023-07-21T22:16:09.469 INFO:teuthology.orchestra.run.smithi093.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-21T22:16:09.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 msgr2=0x7f20180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:09.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 0x7f20180a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f20300ffbf0 tx=0x7f2020009190 comp rx=0 tx=0).stop 2023-07-21T22:16:09.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 msgr2=0x7f203019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:09.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f20240095c0 tx=0x7f202400f710 comp rx=0 tx=0).stop 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 shutdown_connections 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2030104ab0 0x7f2030195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f20180a1260 0x7f20180a3710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20300feab0 0x7f203019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 --2- 172.21.15.93:0/2753899242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20300ff460 0x7f203019bf80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.427+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 >> 172.21.15.93:0/2753899242 conn(0x7f2030075b70 msgr2=0x7f20301028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.428+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 shutdown_connections 2023-07-21T22:16:09.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:09.428+0000 7f2036854700 1 -- 172.21.15.93:0/2753899242 wait complete. 2023-07-21T22:16:09.899 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-21T22:16:09.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T22:16:09.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-07-21T22:16:09.900 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-21T22:16:09.900 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:09.900 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-21T22:16:09.900 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T22:16:09.900 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T22:16:09.901 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:16:09.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:09.901 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-07-21T22:16:09.920 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T22:16:09.920 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T22:16:09.920 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:16:09.921 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 41462sec preferred_lft 41462sec 2023-07-21T22:16:09.922 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::1d18:7f22:62a7:7e48/64 scope link noprefixroute 2023-07-21T22:16:09.923 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:16:09.923 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T22:16:09.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:09.924 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-07-21T22:16:09.924 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-07-21T22:16:09.924 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-07-21T22:16:09.924 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-07-21T22:16:09.925 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-07-21T22:16:09.925 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T22:16:09.925 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-07-21T22:16:10.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.100 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T22:16:10.128 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-07-21T22:16:10.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.129 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-07-21T22:16:10.205 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-07-21T22:16:10.206 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T22:16:10.206 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-21T22:16:10.206 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-21T22:16:10.206 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T22:16:10.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.207 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-07-21T22:16:10.207 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-07-21T22:16:10.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-21T22:16:10.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-21T22:16:10.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-21T22:16:10.208 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-07-21T22:16:10.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.483 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-07-21T22:16:10.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.500 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-07-21T22:16:10.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.557 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-07-21T22:16:10.557 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-07-21T22:16:10.557 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:16:10.557 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-21T22:16:10.558 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-07-21T22:16:10.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:10 smithi134 ceph-mon[97873]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:16:10.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:10 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2753899242' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:16:10.635 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:10 smithi093 ceph-mon[96579]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:16:10.635 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:10 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2753899242' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:16:10.651 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-21T22:16:10.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.652 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-07-21T22:16:10.652 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-07-21T22:16:10.652 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-21T22:16:10.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T22:16:10.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-21T22:16:10.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-21T22:16:10.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-21T22:16:10.654 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-07-21T22:16:10.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:10 smithi194 ceph-mon[98288]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:16:10.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:10 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2753899242' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:16:10.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:10.933 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-07-21T22:16:10.933 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-07-21T22:16:10.934 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-07-21T22:16:10.934 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T22:16:10.934 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-07-21T22:16:11.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:16:11.031 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:11.048 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:16:11.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:16:11.049 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:11.105 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-21T22:16:11.106 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-21T22:16:11.106 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880 2023-07-21T22:16:11.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.107 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:16:11.226 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:16:11.226 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T22:16:11.399 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:16:11.400 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T22:16:11.400 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.93:6789,172.21.15.134:6789,172.21.15.194:6789" 2023-07-21T22:16:11.400 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with new device syntax: 0@de1cec7a-2812-11ee-9b34-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880 2023-07-21T22:16:11.401 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.93:6789/172.21.15.134:6789/172.21.15.194:6789" will pass to kernel 2023-07-21T22:16:11.401 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: v2 mount support check returned 2 2023-07-21T22:16:11.401 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-21T22:16:11.401 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.93:6789,172.21.15.134:6789,172.21.15.194:6789:/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880 2023-07-21T22:16:11.401 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=de1cec7a-2812-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-21T22:16:11.402 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-21T22:16:11.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.403 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:11.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.432 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:16:11.524 INFO:teuthology.orchestra.run.smithi093.stdout:client24968 2023-07-21T22:16:11.525 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:16:11.525 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:16:11.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.542 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:16:11.640 INFO:teuthology.orchestra.run.smithi093.stdout:client24968 2023-07-21T22:16:11.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.641 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:16:11.692 INFO:teuthology.orchestra.run.smithi093.stdout:client24968 2023-07-21T22:16:11.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:16:11.693 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c.client24968/status') 2023-07-21T22:16:11.747 INFO:teuthology.orchestra.run.smithi093.stdout:instance: client.24968 (3)172.21.15.93:0/952484535 2023-07-21T22:16:11.748 INFO:teuthology.orchestra.run.smithi093.stdout:blocklisted: false 2023-07-21T22:16:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records in 2023-07-21T22:16:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records out 2023-07-21T22:16:11.748 INFO:teuthology.orchestra.run.smithi093.stderr:70 bytes copied, 2.0958e-05 s, 3.3 MB/s 2023-07-21T22:16:11.749 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-21T22:16:11.760 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-21T22:16:11.771 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:11.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.848+0000 7fc1321be700 1 -- 172.21.15.93:0/4083257187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c0ffb00 msgr2=0x7fc12c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.848+0000 7fc1321be700 1 --2- 172.21.15.93:0/4083257187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c0ffb00 0x7fc12c0fff10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009990 tx=0x7fc11c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.848+0000 7fc1321be700 1 -- 172.21.15.93:0/4083257187 shutdown_connections 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.848+0000 7fc1321be700 1 --2- 172.21.15.93:0/4083257187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c101680 0x7fc12c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 --2- 172.21.15.93:0/4083257187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c0ffb00 0x7fc12c0fff10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 --2- 172.21.15.93:0/4083257187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c100cf0 0x7fc12c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 -- 172.21.15.93:0/4083257187 >> 172.21.15.93:0/4083257187 conn(0x7fc12c0fb0e0 msgr2=0x7fc12c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:11.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 -- 172.21.15.93:0/4083257187 shutdown_connections 2023-07-21T22:16:11.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 -- 172.21.15.93:0/4083257187 wait complete. 2023-07-21T22:16:11.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 Processor -- start 2023-07-21T22:16:11.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 -- start start 2023-07-21T22:16:11.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0ffb00 0x7fc12c1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.849+0000 7fc1321be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c101680 0x7fc12c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d330 con 0x7fc12c100cf0 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d470 con 0x7fc12c0ffb00 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19d600 con 0x7fc12c101680 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:11.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43640/0 (socket says 172.21.15.93:43640) 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 -- 172.21.15.93:0/2263963613 learned_addr learned my addr 172.21.15.93:0/2263963613 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12b7fe700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0ffb00 0x7fc12c1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 -- 172.21.15.93:0/2263963613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c101680 msgr2=0x7fc12c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c101680 0x7fc12c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 -- 172.21.15.93:0/2263963613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0ffb00 msgr2=0x7fc12c1957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0ffb00 0x7fc12c1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 -- 172.21.15.93:0/2263963613 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0095f0 con 0x7fc12c100cf0 2023-07-21T22:16:11.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc12affd700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d950 tx=0x7fc12000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120009b40 con 0x7fc12c100cf0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc120009ca0 con 0x7fc12c100cf0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c19d8e0 con 0x7fc12c100cf0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12000f620 con 0x7fc12c100cf0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.850+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c19ddf0 con 0x7fc12c100cf0 2023-07-21T22:16:11.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.851+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c10b2d0 con 0x7fc12c100cf0 2023-07-21T22:16:11.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.852+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc120010460 con 0x7fc12c100cf0 2023-07-21T22:16:11.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.853+0000 7fc128ff9700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 0x7fc1140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:11.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.853+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc1200ccdf0 con 0x7fc12c100cf0 2023-07-21T22:16:11.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.853+0000 7fc12b7fe700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 0x7fc1140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:11.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.853+0000 7fc12b7fe700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 0x7fc1140a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00b5c0 tx=0x7fc11c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:11.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:11.857+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc120096410 con 0x7fc12c100cf0 2023-07-21T22:16:12.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.048+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc12c1069b0 con 0x7fc12c100cf0 2023-07-21T22:16:12.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.049+0000 7fc128ff9700 1 -- 172.21.15.93:0/2263963613 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7fc120020090 con 0x7fc12c100cf0 2023-07-21T22:16:12.093 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:12.093 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:12.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 msgr2=0x7fc1140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 0x7fc1140a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00b5c0 tx=0x7fc11c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 msgr2=0x7fc12c195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d950 tx=0x7fc12000dd10 comp rx=0 tx=0).stop 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 shutdown_connections 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc12c101680 0x7fc12c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc1140a1220 0x7fc1140a36d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc12c0ffb00 0x7fc12c1957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 --2- 172.21.15.93:0/2263963613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc12c100cf0 0x7fc12c195cf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 >> 172.21.15.93:0/2263963613 conn(0x7fc12c0fb0e0 msgr2=0x7fc12c0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 shutdown_connections 2023-07-21T22:16:12.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.052+0000 7fc1321be700 1 -- 172.21.15.93:0/2263963613 wait complete. 2023-07-21T22:16:12.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:16:12.109 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:16:12.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 -- 172.21.15.93:0/644106607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800fe9f0 msgr2=0x7f3f800fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 --2- 172.21.15.93:0/644106607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800fe9f0 0x7f3f800fee00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 -- 172.21.15.93:0/644106607 shutdown_connections 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 --2- 172.21.15.93:0/644106607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 0x7f3f80106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 --2- 172.21.15.93:0/644106607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800fe9f0 0x7f3f800fee00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 --2- 172.21.15.93:0/644106607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800ffbe0 0x7f3f80100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 -- 172.21.15.93:0/644106607 >> 172.21.15.93:0/644106607 conn(0x7f3f800fa030 msgr2=0x7f3f800fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 -- 172.21.15.93:0/644106607 shutdown_connections 2023-07-21T22:16:12.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 -- 172.21.15.93:0/644106607 wait complete. 2023-07-21T22:16:12.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.191+0000 7f3f861fb700 1 Processor -- start 2023-07-21T22:16:12.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 -- start start 2023-07-21T22:16:12.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800fe9f0 0x7f3f80069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 0x7f3f8019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019d270 con 0x7f3f800ffbe0 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019d3e0 con 0x7f3f800fe9f0 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f861fb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8019d5a0 con 0x7f3f80100570 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43654/0 (socket says 172.21.15.93:43654) 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 -- 172.21.15.93:0/1142800677 learned_addr learned my addr 172.21.15.93:0/1142800677 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:12.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7f7fe700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800fe9f0 0x7f3f80069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7ffff700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 0x7f3f8019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 -- 172.21.15.93:0/1142800677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 msgr2=0x7f3f8019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 0x7f3f8019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 -- 172.21.15.93:0/1142800677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800fe9f0 msgr2=0x7f3f80069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800fe9f0 0x7f3f80069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 -- 172.21.15.93:0/1142800677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700095f0 con 0x7f3f800ffbe0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.192+0000 7f3f7effd700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c9b0 tx=0x7f3f7400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.193+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74007b00 con 0x7f3f800ffbe0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.193+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f74007c60 con 0x7f3f800ffbe0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.193+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8019d880 con 0x7f3f800ffbe0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.193+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74018660 con 0x7f3f800ffbe0 2023-07-21T22:16:12.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.193+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8019dd90 con 0x7f3f800ffbe0 2023-07-21T22:16:12.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.194+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f80105e90 con 0x7f3f800ffbe0 2023-07-21T22:16:12.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.195+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3f74018880 con 0x7f3f800ffbe0 2023-07-21T22:16:12.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.196+0000 7f3f7cff9700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 0x7f3f680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.196+0000 7f3f7f7fe700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 0x7f3f680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.196+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3f740cce70 con 0x7f3f800ffbe0 2023-07-21T22:16:12.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.196+0000 7f3f7f7fe700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 0x7f3f680a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006010 tx=0x7f3f7000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.199+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3f740964c0 con 0x7f3f800ffbe0 2023-07-21T22:16:12.388 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:12 smithi093 ceph-mon[96579]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T22:16:12.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:12 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2263963613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:12.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.390+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f8002c9e0 con 0x7f3f800ffbe0 2023-07-21T22:16:12.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.390+0000 7f3f7cff9700 1 -- 172.21.15.93:0/1142800677 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f74099ae0 con 0x7f3f800ffbe0 2023-07-21T22:16:12.435 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:16:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 msgr2=0x7f3f680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 0x7f3f680a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006010 tx=0x7f3f7000b540 comp rx=0 tx=0).stop 2023-07-21T22:16:12.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 msgr2=0x7f3f8019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400c9b0 tx=0x7f3f7400cd70 comp rx=0 tx=0).stop 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 shutdown_connections 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f80100570 0x7f3f8019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f680a11e0 0x7f3f680a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f800fe9f0 0x7f3f80069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 --2- 172.21.15.93:0/1142800677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f800ffbe0 0x7f3f8019a450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.392+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 >> 172.21.15.93:0/1142800677 conn(0x7f3f800fa030 msgr2=0x7f3f80102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.393+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 shutdown_connections 2023-07-21T22:16:12.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.393+0000 7f3f861fb700 1 -- 172.21.15.93:0/1142800677 wait complete. 2023-07-21T22:16:12.448 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:16:12.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 -- 172.21.15.93:0/2168455673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80ff5e0 msgr2=0x7f76d8106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 --2- 172.21.15.93:0/2168455673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80ff5e0 0x7f76d8106e90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f76d000b0a0 tx=0x7f76d000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 -- 172.21.15.93:0/2168455673 shutdown_connections 2023-07-21T22:16:12.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 --2- 172.21.15.93:0/2168455673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80ff5e0 0x7f76d8106e90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 --2- 172.21.15.93:0/2168455673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fffe0 0x7f76d81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 --2- 172.21.15.93:0/2168455673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80fec30 0x7f76d80ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 -- 172.21.15.93:0/2168455673 >> 172.21.15.93:0/2168455673 conn(0x7f76d80f9770 msgr2=0x7f76d80fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 -- 172.21.15.93:0/2168455673 shutdown_connections 2023-07-21T22:16:12.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.529+0000 7f76e0555700 1 -- 172.21.15.93:0/2168455673 wait complete. 2023-07-21T22:16:12.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 Processor -- start 2023-07-21T22:16:12.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 -- start start 2023-07-21T22:16:12.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fec30 0x7f76d8194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80ff5e0 0x7f76d8194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819c300 con 0x7f76d80fffe0 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819c440 con 0x7f76d80ff5e0 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76e0555700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d819c5d0 con 0x7f76d80fec30 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43670/0 (socket says 172.21.15.93:43670) 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 -- 172.21.15.93:0/2487543615 learned_addr learned my addr 172.21.15.93:0/2487543615 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:12.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76ddaf0700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80ff5e0 0x7f76d8194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76de2f1700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fec30 0x7f76d8194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 -- 172.21.15.93:0/2487543615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fec30 msgr2=0x7f76d8194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fec30 0x7f76d8194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 -- 172.21.15.93:0/2487543615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80ff5e0 msgr2=0x7f76d8194cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80ff5e0 0x7f76d8194cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 -- 172.21.15.93:0/2487543615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009d00 con 0x7f76d80fffe0 2023-07-21T22:16:12.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.530+0000 7f76deaf2700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.531+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000e040 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.531+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d001a6d0 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.531+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80fdb20 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.531+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004390 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.531+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d80fe080 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.532+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d80fea10 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.533+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f76d001ab20 con 0x7f76d80fffe0 2023-07-21T22:16:12.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.533+0000 7f76cf7fe700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 0x7f76c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.533+0000 7f76de2f1700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 0x7f76c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.533+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76d00cffd0 con 0x7f76d80fffe0 2023-07-21T22:16:12.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.534+0000 7f76de2f1700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 0x7f76c40a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005fd0 tx=0x7f76c8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.537+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f76d0102770 con 0x7f76d80fffe0 2023-07-21T22:16:12.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:12 smithi134 ceph-mon[97873]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T22:16:12.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:12 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2263963613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:12.769 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.725+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76d81059e0 con 0x7f76d80fffe0 2023-07-21T22:16:12.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.726+0000 7f76cf7fe700 1 -- 172.21.15.93:0/2487543615 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76d0099680 con 0x7f76d80fffe0 2023-07-21T22:16:12.770 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:16:12.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 msgr2=0x7f76c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 0x7f76c40a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005fd0 tx=0x7f76c8005e50 comp rx=0 tx=0).stop 2023-07-21T22:16:12.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 msgr2=0x7f76d819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0004b20 comp rx=0 tx=0).stop 2023-07-21T22:16:12.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 shutdown_connections 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d80fffe0 0x7f76d819bd30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76d80fec30 0x7f76d8194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f76c40a1260 0x7f76c40a3710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 --2- 172.21.15.93:0/2487543615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d80ff5e0 0x7f76d8194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 >> 172.21.15.93:0/2487543615 conn(0x7f76d80f9770 msgr2=0x7f76d8102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 shutdown_connections 2023-07-21T22:16:12.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.728+0000 7f76e0555700 1 -- 172.21.15.93:0/2487543615 wait complete. 2023-07-21T22:16:12.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:16:12.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:12 smithi194 ceph-mon[98288]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-21T22:16:12.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:12 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2263963613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:12.907 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.863+0000 7ffa22599700 1 -- 172.21.15.93:0/926584760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0fffe0 msgr2=0x7ffa1c1003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.863+0000 7ffa22599700 1 --2- 172.21.15.93:0/926584760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c1003f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009990 tx=0x7ffa10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 -- 172.21.15.93:0/926584760 shutdown_connections 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- 172.21.15.93:0/926584760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- 172.21.15.93:0/926584760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c1003f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- 172.21.15.93:0/926584760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fec30 0x7ffa1c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 -- 172.21.15.93:0/926584760 >> 172.21.15.93:0/926584760 conn(0x7ffa1c0f9770 msgr2=0x7ffa1c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:12.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 -- 172.21.15.93:0/926584760 shutdown_connections 2023-07-21T22:16:12.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 -- 172.21.15.93:0/926584760 wait complete. 2023-07-21T22:16:12.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 Processor -- start 2023-07-21T22:16:12.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 -- start start 2023-07-21T22:16:12.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0fec30 0x7ffa1c0fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.864+0000 7ffa22599700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa22599700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19d350 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa22599700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19d4c0 con 0x7ffa1c0fec30 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa22599700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19d680 con 0x7ffa1c0fffe0 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43682/0 (socket says 172.21.15.93:43682) 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 -- 172.21.15.93:0/2835450733 learned_addr learned my addr 172.21.15.93:0/2835450733 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1bfff700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0fec30 0x7ffa1c0fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa20b36700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 -- 172.21.15.93:0/2835450733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 msgr2=0x7ffa1c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 -- 172.21.15.93:0/2835450733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0fec30 msgr2=0x7ffa1c0fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0fec30 0x7ffa1c0fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 -- 172.21.15.93:0/2835450733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100095f0 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa20b36700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa1b7fe700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00d980 tx=0x7ffa0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004a30 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa0c0054e0 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c19d960 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c009c90 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.865+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19de70 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.866+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c1059d0 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.867+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ffa0c004c20 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.868+0000 7ffa197fa700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 0x7ffa040a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.868+0000 7ffa1bfff700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 0x7ffa040a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.868+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffa0c0cccc0 con 0x7ffa1c0ff5e0 2023-07-21T22:16:12.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.868+0000 7ffa1bfff700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 0x7ffa040a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:12.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:12.871+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffa0c096360 con 0x7ffa1c0ff5e0 2023-07-21T22:16:13.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.060+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa1c02c9e0 con 0x7ffa1c0ff5e0 2023-07-21T22:16:13.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.060+0000 7ffa197fa700 1 -- 172.21.15.93:0/2835450733 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa0c017030 con 0x7ffa1c0ff5e0 2023-07-21T22:16:13.104 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:16:13.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.062+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 msgr2=0x7ffa040a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.062+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 0x7ffa040a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005960 comp rx=0 tx=0).stop 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.062+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 msgr2=0x7ffa1c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.062+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00d980 tx=0x7ffa0c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 shutdown_connections 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa1c0fffe0 0x7ffa1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7ffa040a1260 0x7ffa040a3710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa1c0fec30 0x7ffa1c0fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 --2- 172.21.15.93:0/2835450733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa1c0ff5e0 0x7ffa1c19a570 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 >> 172.21.15.93:0/2835450733 conn(0x7ffa1c0f9770 msgr2=0x7ffa1c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 shutdown_connections 2023-07-21T22:16:13.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.063+0000 7ffa22599700 1 -- 172.21.15.93:0/2835450733 wait complete. 2023-07-21T22:16:13.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:13.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 -- 172.21.15.93:0/1138529527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea24100560 msgr2=0x7fea24106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 --2- 172.21.15.93:0/1138529527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea24100560 0x7fea24106e20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b0a0 tx=0x7fea1c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 -- 172.21.15.93:0/1138529527 shutdown_connections 2023-07-21T22:16:13.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 --2- 172.21.15.93:0/1138529527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea24100560 0x7fea24106e20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 --2- 172.21.15.93:0/1138529527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea240fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 --2- 172.21.15.93:0/1138529527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea240ffbd0 0x7fea24100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 -- 172.21.15.93:0/1138529527 >> 172.21.15.93:0/1138529527 conn(0x7fea240fa000 msgr2=0x7fea240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 -- 172.21.15.93:0/1138529527 shutdown_connections 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.196+0000 7fea2c432700 1 -- 172.21.15.93:0/1138529527 wait complete. 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 Processor -- start 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 -- start start 2023-07-21T22:16:13.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea241947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 0x7fea2419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2419c320 con 0x7fea240ffbd0 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2419c460 con 0x7fea24100560 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2c432700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2419c5f0 con 0x7fea240fe9e0 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea299cd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea299cd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43696/0 (socket says 172.21.15.93:43696) 2023-07-21T22:16:13.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea299cd700 1 -- 172.21.15.93:0/594736758 learned_addr learned my addr 172.21.15.93:0/594736758 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:13.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2a1ce700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea241947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.197+0000 7fea2a9cf700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 0x7fea2419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 -- 172.21.15.93:0/594736758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 msgr2=0x7fea241947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea241947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 -- 172.21.15.93:0/594736758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 msgr2=0x7fea2419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 0x7fea2419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 -- 172.21.15.93:0/594736758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c009d00 con 0x7fea240ffbd0 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea2a9cf700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 0x7fea2419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea2a1ce700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea241947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea299cd700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d950 tx=0x7fea2000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20009b40 con 0x7fea240ffbd0 2023-07-21T22:16:13.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2419c8d0 con 0x7fea240ffbd0 2023-07-21T22:16:13.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea20009ca0 con 0x7fea240ffbd0 2023-07-21T22:16:13.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000f620 con 0x7fea240ffbd0 2023-07-21T22:16:13.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.198+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2419ce30 con 0x7fea240ffbd0 2023-07-21T22:16:13.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.199+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2410a1c0 con 0x7fea240ffbd0 2023-07-21T22:16:13.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.200+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fea20010460 con 0x7fea240ffbd0 2023-07-21T22:16:13.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.200+0000 7fea177fe700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 0x7fea100a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.200+0000 7fea2a1ce700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 0x7fea100a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.201+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fea200ccd70 con 0x7fea240ffbd0 2023-07-21T22:16:13.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.201+0000 7fea2a1ce700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 0x7fea100a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea240ffa30 tx=0x7fea1800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.205+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fea20096390 con 0x7fea240ffbd0 2023-07-21T22:16:13.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:13 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1142800677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:13 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2487543615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:13 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2835450733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.397+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea241057c0 con 0x7fea240ffbd0 2023-07-21T22:16:13.442 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.399+0000 7fea177fe700 1 -- 172.21.15.93:0/594736758 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7fea20020090 con 0x7fea240ffbd0 2023-07-21T22:16:13.442 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:13.443 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 msgr2=0x7fea100a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 0x7fea100a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea240ffa30 tx=0x7fea1800b670 comp rx=0 tx=0).stop 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 msgr2=0x7fea24194ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d950 tx=0x7fea2000dd10 comp rx=0 tx=0).stop 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 shutdown_connections 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fea100a1220 0x7fea100a36d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea24100560 0x7fea2419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea240fe9e0 0x7fea241947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 --2- 172.21.15.93:0/594736758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea240ffbd0 0x7fea24194ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 >> 172.21.15.93:0/594736758 conn(0x7fea240fa000 msgr2=0x7fea240fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 shutdown_connections 2023-07-21T22:16:13.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.401+0000 7fea2c432700 1 -- 172.21.15.93:0/594736758 wait complete. 2023-07-21T22:16:13.457 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:16:13.458 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:16:13.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 -- 172.21.15.93:0/3840159557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 msgr2=0x7f821c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 --2- 172.21.15.93:0/3840159557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c107c00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f821400b0a0 tx=0x7f821400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 -- 172.21.15.93:0/3840159557 shutdown_connections 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 --2- 172.21.15.93:0/3840159557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c107c00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 --2- 172.21.15.93:0/3840159557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 0x7f821c102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 --2- 172.21.15.93:0/3840159557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 -- 172.21.15.93:0/3840159557 >> 172.21.15.93:0/3840159557 conn(0x7f821c0fa000 msgr2=0x7f821c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 -- 172.21.15.93:0/3840159557 shutdown_connections 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.538+0000 7f8223254700 1 -- 172.21.15.93:0/3840159557 wait complete. 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 Processor -- start 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 -- start start 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 0x7f821c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c19d440 con 0x7f821c1057e0 2023-07-21T22:16:13.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c19d580 con 0x7f821c102ec0 2023-07-21T22:16:13.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f8223254700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c19d710 con 0x7f821c1005a0 2023-07-21T22:16:13.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f82217f1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f82217f1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43712/0 (socket says 172.21.15.93:43712) 2023-07-21T22:16:13.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f82217f1700 1 -- 172.21.15.93:0/124734033 learned_addr learned my addr 172.21.15.93:0/124734033 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:13.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.539+0000 7f821bfff700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8220ff0700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 0x7f821c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 -- 172.21.15.93:0/124734033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 msgr2=0x7f821c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 0x7f821c1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 -- 172.21.15.93:0/124734033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 msgr2=0x7f821c195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 -- 172.21.15.93:0/124734033 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214009d00 con 0x7f821c1057e0 2023-07-21T22:16:13.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f821bfff700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:13.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f82217f1700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f82140062a0 tx=0x7f8214004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e040 con 0x7f821c1057e0 2023-07-21T22:16:13.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f821401db50 con 0x7f821c1057e0 2023-07-21T22:16:13.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821401bc40 con 0x7f821c1057e0 2023-07-21T22:16:13.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c19d990 con 0x7f821c1057e0 2023-07-21T22:16:13.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.540+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c19ddc0 con 0x7f821c1057e0 2023-07-21T22:16:13.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.541+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c0fbb70 con 0x7f821c1057e0 2023-07-21T22:16:13.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.542+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f821401d690 con 0x7f821c1057e0 2023-07-21T22:16:13.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.542+0000 7f8219ffb700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 0x7f820c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.543+0000 7f8220ff0700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 0x7f820c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.543+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f82140ce6d0 con 0x7f821c1057e0 2023-07-21T22:16:13.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.543+0000 7f8220ff0700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 0x7f820c0a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8210005950 tx=0x7f82100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.547+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8214097d20 con 0x7f821c1057e0 2023-07-21T22:16:13.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:13 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1142800677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:13 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2487543615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:13 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2835450733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.735+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f821c02ca90 con 0x7f821c1057e0 2023-07-21T22:16:13.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.735+0000 7f8219ffb700 1 -- 172.21.15.93:0/124734033 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f821409b340 con 0x7f821c1057e0 2023-07-21T22:16:13.779 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:16:13.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 msgr2=0x7f820c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 0x7f820c0a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8210005950 tx=0x7f82100058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 msgr2=0x7f821c19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f82140062a0 tx=0x7f8214004db0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 shutdown_connections 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f821c1057e0 0x7f821c19ce70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f821c1005a0 0x7f821c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f820c0a11e0 0x7f820c0a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 --2- 172.21.15.93:0/124734033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f821c102ec0 0x7f821c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 >> 172.21.15.93:0/124734033 conn(0x7f821c0fa000 msgr2=0x7f821c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 shutdown_connections 2023-07-21T22:16:13.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.738+0000 7f8223254700 1 -- 172.21.15.93:0/124734033 wait complete. 2023-07-21T22:16:13.793 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:16:13.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:13 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1142800677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:13 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2487543615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:13 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2835450733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:13.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 -- 172.21.15.93:0/2691086584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0ff5e0 msgr2=0x7fde6c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 --2- 172.21.15.93:0/2691086584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c106e90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde6400b0a0 tx=0x7fde6400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 -- 172.21.15.93:0/2691086584 shutdown_connections 2023-07-21T22:16:13.918 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 --2- 172.21.15.93:0/2691086584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c106e90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 --2- 172.21.15.93:0/2691086584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 0x7fde6c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 --2- 172.21.15.93:0/2691086584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0fec30 0x7fde6c0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 -- 172.21.15.93:0/2691086584 >> 172.21.15.93:0/2691086584 conn(0x7fde6c0f9770 msgr2=0x7fde6c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 -- 172.21.15.93:0/2691086584 shutdown_connections 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.874+0000 7fde7362a700 1 -- 172.21.15.93:0/2691086584 wait complete. 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 Processor -- start 2023-07-21T22:16:13.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 -- start start 2023-07-21T22:16:13.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 0x7fde6c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c19c3f0 con 0x7fde6c0fec30 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c19c530 con 0x7fde6c0ff5e0 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde7362a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c19c6c0 con 0x7fde6c0fffe0 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43722/0 (socket says 172.21.15.93:43722) 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 -- 172.21.15.93:0/3966060226 learned_addr learned my addr 172.21.15.93:0/3966060226 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde71bc7700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 0x7fde6c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde70bc5700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 -- 172.21.15.93:0/3966060226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 msgr2=0x7fde6c19be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 0x7fde6c19be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 -- 172.21.15.93:0/3966060226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0ff5e0 msgr2=0x7fde6c194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 -- 172.21.15.93:0/3966060226 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64009d00 con 0x7fde6c0fec30 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde713c6700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d980 tx=0x7fde5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.875+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004020 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.876+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5c009730 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.876+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c19c9a0 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.876+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c003a80 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.876+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c19ceb0 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.877+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c105980 con 0x7fde6c0fec30 2023-07-21T22:16:13.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.878+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fde5c0098a0 con 0x7fde6c0fec30 2023-07-21T22:16:13.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.878+0000 7fde627fc700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 0x7fde580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:13.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.878+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fde5c0ccab0 con 0x7fde6c0fec30 2023-07-21T22:16:13.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.878+0000 7fde70bc5700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 0x7fde580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:13.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.879+0000 7fde70bc5700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 0x7fde580a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde68005fd0 tx=0x7fde6800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:13.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:13.881+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fde5c0960d0 con 0x7fde6c0fec30 2023-07-21T22:16:14.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.069+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde6c02cb00 con 0x7fde6c0fec30 2023-07-21T22:16:14.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.069+0000 7fde627fc700 1 -- 172.21.15.93:0/3966060226 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde5c0996f0 con 0x7fde6c0fec30 2023-07-21T22:16:14.113 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:16:14.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 msgr2=0x7fde580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 0x7fde580a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde68005fd0 tx=0x7fde6800a560 comp rx=0 tx=0).stop 2023-07-21T22:16:14.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 msgr2=0x7fde6c194870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00d980 tx=0x7fde5c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:14.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 shutdown_connections 2023-07-21T22:16:14.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde6c0fffe0 0x7fde6c19be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde6c0fec30 0x7fde6c194870 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fde580a11e0 0x7fde580a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 --2- 172.21.15.93:0/3966060226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde6c0ff5e0 0x7fde6c194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 >> 172.21.15.93:0/3966060226 conn(0x7fde6c0f9770 msgr2=0x7fde6c0fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 shutdown_connections 2023-07-21T22:16:14.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.071+0000 7fde7362a700 1 -- 172.21.15.93:0/3966060226 wait complete. 2023-07-21T22:16:14.127 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:16:14.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 -- 172.21.15.93:0/3763958536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 msgr2=0x7f41d81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 --2- 172.21.15.93:0/3763958536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d81029a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009990 tx=0x7f41cc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 -- 172.21.15.93:0/3763958536 shutdown_connections 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 --2- 172.21.15.93:0/3763958536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 --2- 172.21.15.93:0/3763958536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d81029a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 --2- 172.21.15.93:0/3763958536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 0x7f41d81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 -- 172.21.15.93:0/3763958536 >> 172.21.15.93:0/3763958536 conn(0x7f41d80fa000 msgr2=0x7f41d80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 -- 172.21.15.93:0/3763958536 shutdown_connections 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.208+0000 7f41de46d700 1 -- 172.21.15.93:0/3763958536 wait complete. 2023-07-21T22:16:14.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 Processor -- start 2023-07-21T22:16:14.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 -- start start 2023-07-21T22:16:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 0x7f41d8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d819d480 con 0x7f41d81005c0 2023-07-21T22:16:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d819d5c0 con 0x7f41d8105800 2023-07-21T22:16:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41de46d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d819d750 con 0x7f41d8102ee0 2023-07-21T22:16:14.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43736/0 (socket says 172.21.15.93:43736) 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 -- 172.21.15.93:0/2696727738 learned_addr learned my addr 172.21.15.93:0/2696727738 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41dca0a700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d77fe700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 0x7f41d8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 -- 172.21.15.93:0/2696727738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 msgr2=0x7f41d8195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 0x7f41d8195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 -- 172.21.15.93:0/2696727738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 msgr2=0x7f41d819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41d7fff700 1 -- 172.21.15.93:0/2696727738 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0095f0 con 0x7f41d81005c0 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.209+0000 7f41dca0a700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41d7fff700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000c00 tx=0x7f41cc00f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc01d070 con 0x7f41d81005c0 2023-07-21T22:16:14.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41cc00fce0 con 0x7f41d81005c0 2023-07-21T22:16:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d819d9d0 con 0x7f41d81005c0 2023-07-21T22:16:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0176e0 con 0x7f41d81005c0 2023-07-21T22:16:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.210+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d819df00 con 0x7f41d81005c0 2023-07-21T22:16:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.211+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d80fbb70 con 0x7f41d81005c0 2023-07-21T22:16:14.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.212+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f41cc022470 con 0x7f41d81005c0 2023-07-21T22:16:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.212+0000 7f41d57fa700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 0x7f41c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.212+0000 7f41d77fe700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 0x7f41c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.212+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f41cc0cdeb0 con 0x7f41d81005c0 2023-07-21T22:16:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.213+0000 7f41d77fe700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 0x7f41c00a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.216+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f41cc05c770 con 0x7f41d81005c0 2023-07-21T22:16:14.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:14 smithi093 ceph-mon[96579]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T22:16:14.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:14 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/594736758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:14.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:14 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/124734033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:14 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3966060226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.407+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41d802ca70 con 0x7f41d81005c0 2023-07-21T22:16:14.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.407+0000 7f41d57fa700 1 -- 172.21.15.93:0/2696727738 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41cc0270a0 con 0x7f41d81005c0 2023-07-21T22:16:14.451 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:16:14.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 msgr2=0x7f41c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 0x7f41c00a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2023-07-21T22:16:14.453 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 msgr2=0x7f41d8195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000c00 tx=0x7f41cc00f830 comp rx=0 tx=0).stop 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 shutdown_connections 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f41c00a11e0 0x7f41c00a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41d8105800 0x7f41d819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41d81005c0 0x7f41d8195900 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 --2- 172.21.15.93:0/2696727738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41d8102ee0 0x7f41d8195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 >> 172.21.15.93:0/2696727738 conn(0x7f41d80fa000 msgr2=0x7f41d81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 shutdown_connections 2023-07-21T22:16:14.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.409+0000 7f41de46d700 1 -- 172.21.15.93:0/2696727738 wait complete. 2023-07-21T22:16:14.465 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-21T22:16:14.465 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:16:14.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- 172.21.15.93:0/1863622564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200ffbc0 msgr2=0x7f1020100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 --2- 172.21.15.93:0/1863622564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200ffbc0 0x7f1020100010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- 172.21.15.93:0/1863622564 shutdown_connections 2023-07-21T22:16:14.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 --2- 172.21.15.93:0/1863622564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1020100550 0x7f1020106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 --2- 172.21.15.93:0/1863622564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10200fe9d0 0x7f10200fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 --2- 172.21.15.93:0/1863622564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200ffbc0 0x7f1020100010 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- 172.21.15.93:0/1863622564 >> 172.21.15.93:0/1863622564 conn(0x7f10200f9fd0 msgr2=0x7f10200fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- 172.21.15.93:0/1863622564 shutdown_connections 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- 172.21.15.93:0/1863622564 wait complete. 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 Processor -- start 2023-07-21T22:16:14.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.542+0000 7f102695d700 1 -- start start 2023-07-21T22:16:14.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 0x7f102019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020100550 0x7f102019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102019d360 con 0x7f10200fe9d0 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102019d4d0 con 0x7f1020100550 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102019d690 con 0x7f10200ffbc0 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43750/0 (socket says 172.21.15.93:43750) 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 -- 172.21.15.93:0/1357030736 learned_addr learned my addr 172.21.15.93:0/1357030736 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:14.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101f7fe700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 0x7f102019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 -- 172.21.15.93:0/1357030736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 msgr2=0x7f102019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 0x7f102019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 -- 172.21.15.93:0/1357030736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020100550 msgr2=0x7f102019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f1024efa700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020100550 0x7f102019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020100550 0x7f102019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 -- 172.21.15.93:0/1357030736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100095f0 con 0x7f10200fe9d0 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101f7fe700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 0x7f102019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:14.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101ffff700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f101400d980 tx=0x7f101400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400de60 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10140044e0 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102019d970 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014003be0 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.543+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102019ded0 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.544+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1020105f60 con 0x7f10200fe9d0 2023-07-21T22:16:14.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.545+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1014009bb0 con 0x7f10200fe9d0 2023-07-21T22:16:14.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.546+0000 7f101d7fa700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 0x7f10080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.546+0000 7f101f7fe700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 0x7f10080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.546+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f10140cc800 con 0x7f10200fe9d0 2023-07-21T22:16:14.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.546+0000 7f101f7fe700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 0x7f10080a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f101000b5c0 tx=0x7f1010005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.550+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1014095dc0 con 0x7f10200fe9d0 2023-07-21T22:16:14.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:14 smithi134 ceph-mon[97873]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T22:16:14.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:14 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/594736758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:14.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:14 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/124734033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:14 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3966060226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.779+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f102002c850 con 0x7f10200fe9d0 2023-07-21T22:16:14.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.780+0000 7f101d7fa700 1 -- 172.21.15.93:0/1357030736 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f1014009940 con 0x7f10200fe9d0 2023-07-21T22:16:14.824 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:14.824 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:16:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 msgr2=0x7f10080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 0x7f10080a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f101000b5c0 tx=0x7f1010005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 msgr2=0x7f1020068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f101400d980 tx=0x7f101400dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:14.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 shutdown_connections 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f10080a12a0 0x7f10080a3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020100550 0x7f102019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10200fe9d0 0x7f1020068200 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 --2- 172.21.15.93:0/1357030736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10200ffbc0 0x7f102019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 >> 172.21.15.93:0/1357030736 conn(0x7f10200f9fd0 msgr2=0x7f10201030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 shutdown_connections 2023-07-21T22:16:14.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.782+0000 7f102695d700 1 -- 172.21.15.93:0/1357030736 wait complete. 2023-07-21T22:16:14.828 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:14.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:14 smithi194 ceph-mon[98288]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-21T22:16:14.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:14 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/594736758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:14.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:14 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/124734033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:14 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3966060226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:14.838 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-21T22:16:14.838 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-21T22:16:14.839 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:16:14.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- 172.21.15.93:0/3667278671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b701057c0 msgr2=0x7f2b70107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 --2- 172.21.15.93:0/3667278671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b701057c0 0x7f2b70107c20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800b0a0 tx=0x7f2b6800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- 172.21.15.93:0/3667278671 shutdown_connections 2023-07-21T22:16:14.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 --2- 172.21.15.93:0/3667278671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b701057c0 0x7f2b70107c20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 --2- 172.21.15.93:0/3667278671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b70100580 0x7f2b70102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 --2- 172.21.15.93:0/3667278671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 0x7f2b70105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- 172.21.15.93:0/3667278671 >> 172.21.15.93:0/3667278671 conn(0x7f2b700fa020 msgr2=0x7f2b700fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:14.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- 172.21.15.93:0/3667278671 shutdown_connections 2023-07-21T22:16:14.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- 172.21.15.93:0/3667278671 wait complete. 2023-07-21T22:16:14.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 Processor -- start 2023-07-21T22:16:14.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.916+0000 7f2b76420700 1 -- start start 2023-07-21T22:16:14.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 0x7f2b70195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b701057c0 0x7f2b7019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7019d3d0 con 0x7f2b70100580 2023-07-21T22:16:14.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7019d510 con 0x7f2b70102ea0 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7019d6a0 con 0x7f2b701057c0 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43764/0 (socket says 172.21.15.93:43764) 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 -- 172.21.15.93:0/3104935852 learned_addr learned my addr 172.21.15.93:0/3104935852 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6f7fe700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 0x7f2b70195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 -- 172.21.15.93:0/3104935852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b701057c0 msgr2=0x7f2b7019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b749bd700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b701057c0 0x7f2b7019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b701057c0 0x7f2b7019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 -- 172.21.15.93:0/3104935852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 msgr2=0x7f2b70195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 0x7f2b70195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 -- 172.21.15.93:0/3104935852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009d00 con 0x7f2b70100580 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6ffff700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000d980 tx=0x7f2b6000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000bb60 con 0x7f2b70100580 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7019d980 con 0x7f2b70100580 2023-07-21T22:16:14.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b60007640 con 0x7f2b70100580 2023-07-21T22:16:14.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004e90 con 0x7f2b70100580 2023-07-21T22:16:14.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.917+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7019dee0 con 0x7f2b70100580 2023-07-21T22:16:14.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.918+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700fbb90 con 0x7f2b70100580 2023-07-21T22:16:14.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.919+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2b6000bcc0 con 0x7f2b70100580 2023-07-21T22:16:14.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.920+0000 7f2b6d7fa700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 0x7f2b580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:14.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.920+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2b600cc980 con 0x7f2b70100580 2023-07-21T22:16:14.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.920+0000 7f2b6f7fe700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 0x7f2b580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:14.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.920+0000 7f2b6f7fe700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 0x7f2b580a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009b10 tx=0x7f2b64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:14.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:14.924+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2b60099a30 con 0x7f2b70100580 2023-07-21T22:16:15.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.112+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b70060950 con 0x7f2b580a1220 2023-07-21T22:16:15.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.122+0000 7f2b6d7fa700 1 -- 172.21.15.93:0/3104935852 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343472 (secure 0 0 0) 0x7f2b70060950 con 0x7f2b580a1220 2023-07-21T22:16:15.169 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:15.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 msgr2=0x7f2b580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 0x7f2b580a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009b10 tx=0x7f2b64009260 comp rx=0 tx=0).stop 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 msgr2=0x7f2b70195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000d980 tx=0x7f2b6000dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 shutdown_connections 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.125+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b701057c0 0x7f2b7019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b70100580 0x7f2b70195850 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f2b580a1220 0x7f2b580a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 --2- 172.21.15.93:0/3104935852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b70102ea0 0x7f2b70195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 >> 172.21.15.93:0/3104935852 conn(0x7f2b700fa020 msgr2=0x7f2b701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:15.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 shutdown_connections 2023-07-21T22:16:15.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.126+0000 7f2b76420700 1 -- 172.21.15.93:0/3104935852 wait complete. 2023-07-21T22:16:15.172 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:16:15.181 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-21T22:16:13.228492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654660,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":925,"num_read_kb":587,"num_write":418,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336324,"kb_used_data":12900,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605500,"statfs":{"total":1151940427776,"available":1151596032000,"internally_reserved":0,"allocated":13209600,"data_stored":7243053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043777+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:15:53.043777+0000","last_peered":"2023-07-21T22:15:53.043777+0000","last_clean":"2023-07-21T22:15:53.043777+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:15:53.043777+0000","last_undegraded":"2023-07-21T22:15:53.043777+0000","last_fullsized":"2023-07-21T22:15:53.043777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:19:32.645830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152152+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:15:53.152152+0000","last_peered":"2023-07-21T22:15:53.152152+0000","last_clean":"2023-07-21T22:15:53.152152+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:15:53.152152+0000","last_undegraded":"2023-07-21T22:15:53.152152+0000","last_fullsized":"2023-07-21T22:15:53.152152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:12.886060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152220+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.152220+0000","last_peered":"2023-07-21T22:15:53.152220+0000","last_clean":"2023-07-21T22:15:53.152220+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:15:53.152220+0000","last_undegraded":"2023-07-21T22:15:53.152220+0000","last_fullsized":"2023-07-21T22:15:53.152220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:27.406439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082127+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:15:53.082127+0000","last_peered":"2023-07-21T22:15:53.082127+0000","last_clean":"2023-07-21T22:15:53.082127+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:15:53.082127+0000","last_undegraded":"2023-07-21T22:15:53.082127+0000","last_fullsized":"2023-07-21T22:15:53.082127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:48.099744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040768+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:15:53.040768+0000","last_peered":"2023-07-21T22:15:53.040768+0000","last_clean":"2023-07-21T22:15:53.040768+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:15:53.040768+0000","last_undegraded":"2023-07-21T22:15:53.040768+0000","last_fullsized":"2023-07-21T22:15:53.040768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:03:07.972409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:40:56.830770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035947+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:15:53.035947+0000","last_peered":"2023-07-21T22:15:53.035947+0000","last_clean":"2023-07-21T22:15:53.035947+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:15:53.035947+0000","last_undegraded":"2023-07-21T22:15:53.035947+0000","last_fullsized":"2023-07-21T22:15:53.035947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:47.456515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040964+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:15:53.040964+0000","last_peered":"2023-07-21T22:15:53.040964+0000","last_clean":"2023-07-21T22:15:53.040964+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:15:53.040964+0000","last_undegraded":"2023-07-21T22:15:53.040964+0000","last_fullsized":"2023-07-21T22:15:53.040964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:30.005830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038418+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:15:53.038418+0000","last_peered":"2023-07-21T22:15:53.038418+0000","last_clean":"2023-07-21T22:15:53.038418+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:15:53.038418+0000","last_undegraded":"2023-07-21T22:15:53.038418+0000","last_fullsized":"2023-07-21T22:15:53.038418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:58:57.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080966+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:15:53.080966+0000","last_peered":"2023-07-21T22:15:53.080966+0000","last_clean":"2023-07-21T22:15:53.080966+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:15:53.080966+0000","last_undegraded":"2023-07-21T22:15:53.080966+0000","last_fullsized":"2023-07-21T22:15:53.080966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:50.746825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036344+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:15:53.036344+0000","last_peered":"2023-07-21T22:15:53.036344+0000","last_clean":"2023-07-21T22:15:53.036344+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:15:53.036344+0000","last_undegraded":"2023-07-21T22:15:53.036344+0000","last_fullsized":"2023-07-21T22:15:53.036344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:24:07.779545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082003+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.082003+0000","last_peered":"2023-07-21T22:15:53.082003+0000","last_clean":"2023-07-21T22:15:53.082003+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:15:53.082003+0000","last_undegraded":"2023-07-21T22:15:53.082003+0000","last_fullsized":"2023-07-21T22:15:53.082003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:40.403683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082836+0000","last_change":"2023-07-21T22:15:53.082836+0000","last_active":"2023-07-21T22:15:53.082836+0000","last_peered":"2023-07-21T22:15:53.082836+0000","last_clean":"2023-07-21T22:15:53.082836+0000","last_became_active":"2023-07-21T22:15:53.082492+0000","last_became_peered":"2023-07-21T22:15:53.082492+0000","last_unstale":"2023-07-21T22:15:53.082836+0000","last_undegraded":"2023-07-21T22:15:53.082836+0000","last_fullsized":"2023-07-21T22:15:53.082836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:37.252281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152025+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:15:53.152025+0000","last_peered":"2023-07-21T22:15:53.152025+0000","last_clean":"2023-07-21T22:15:53.152025+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:15:53.152025+0000","last_undegraded":"2023-07-21T22:15:53.152025+0000","last_fullsized":"2023-07-21T22:15:53.152025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:00:20.418659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.151988+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:15:53.151988+0000","last_peered":"2023-07-21T22:15:53.151988+0000","last_clean":"2023-07-21T22:15:53.151988+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:15:53.151988+0000","last_undegraded":"2023-07-21T22:15:53.151988+0000","last_fullsized":"2023-07-21T22:15:53.151988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:30:09.436268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037879+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:15:53.037879+0000","last_peered":"2023-07-21T22:15:53.037879+0000","last_clean":"2023-07-21T22:15:53.037879+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:15:53.037879+0000","last_undegraded":"2023-07-21T22:15:53.037879+0000","last_fullsized":"2023-07-21T22:15:53.037879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:12.939463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043953+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:15:53.043953+0000","last_peered":"2023-07-21T22:15:53.043953+0000","last_clean":"2023-07-21T22:15:53.043953+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:15:53.043953+0000","last_undegraded":"2023-07-21T22:15:53.043953+0000","last_fullsized":"2023-07-21T22:15:53.043953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:23.203665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039040+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:15:53.039040+0000","last_peered":"2023-07-21T22:15:53.039040+0000","last_clean":"2023-07-21T22:15:53.039040+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:15:53.039040+0000","last_undegraded":"2023-07-21T22:15:53.039040+0000","last_fullsized":"2023-07-21T22:15:53.039040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:32.580202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044944+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:15:53.044944+0000","last_peered":"2023-07-21T22:15:53.044944+0000","last_clean":"2023-07-21T22:15:53.044944+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:15:53.044944+0000","last_undegraded":"2023-07-21T22:15:53.044944+0000","last_fullsized":"2023-07-21T22:15:53.044944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:34.324418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.116122+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:16:11.116122+0000","last_peered":"2023-07-21T22:16:11.116122+0000","last_clean":"2023-07-21T22:16:11.116122+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:16:11.116122+0000","last_undegraded":"2023-07-21T22:16:11.116122+0000","last_fullsized":"2023-07-21T22:16:11.116122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:37.453186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041881+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:15:53.041881+0000","last_peered":"2023-07-21T22:15:53.041881+0000","last_clean":"2023-07-21T22:15:53.041881+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:15:53.041881+0000","last_undegraded":"2023-07-21T22:15:53.041881+0000","last_fullsized":"2023-07-21T22:15:53.041881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:08:06.854091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042034+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:15:53.042034+0000","last_peered":"2023-07-21T22:15:53.042034+0000","last_clean":"2023-07-21T22:15:53.042034+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:15:53.042034+0000","last_undegraded":"2023-07-21T22:15:53.042034+0000","last_fullsized":"2023-07-21T22:15:53.042034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:50.856377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152334+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:15:53.152334+0000","last_peered":"2023-07-21T22:15:53.152334+0000","last_clean":"2023-07-21T22:15:53.152334+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:15:53.152334+0000","last_undegraded":"2023-07-21T22:15:53.152334+0000","last_fullsized":"2023-07-21T22:15:53.152334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:48:22.931628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041052+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:15:53.041052+0000","last_peered":"2023-07-21T22:15:53.041052+0000","last_clean":"2023-07-21T22:15:53.041052+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:15:53.041052+0000","last_undegraded":"2023-07-21T22:15:53.041052+0000","last_fullsized":"2023-07-21T22:15:53.041052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:54:35.987213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038150+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:15:53.038150+0000","last_peered":"2023-07-21T22:15:53.038150+0000","last_clean":"2023-07-21T22:15:53.038150+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:15:53.038150+0000","last_undegraded":"2023-07-21T22:15:53.038150+0000","last_fullsized":"2023-07-21T22:15:53.038150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:50.632781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040901+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:15:53.040901+0000","last_peered":"2023-07-21T22:15:53.040901+0000","last_clean":"2023-07-21T22:15:53.040901+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:15:53.040901+0000","last_undegraded":"2023-07-21T22:15:53.040901+0000","last_fullsized":"2023-07-21T22:15:53.040901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:50.098672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039265+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.039265+0000","last_peered":"2023-07-21T22:15:53.039265+0000","last_clean":"2023-07-21T22:15:53.039265+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:15:53.039265+0000","last_undegraded":"2023-07-21T22:15:53.039265+0000","last_fullsized":"2023-07-21T22:15:53.039265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:43.599400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.105103+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:16:10.105103+0000","last_peered":"2023-07-21T22:16:10.105103+0000","last_clean":"2023-07-21T22:16:10.105103+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:16:10.105103+0000","last_undegraded":"2023-07-21T22:16:10.105103+0000","last_fullsized":"2023-07-21T22:16:10.105103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:01:43.880287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040613+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:15:53.040613+0000","last_peered":"2023-07-21T22:15:53.040613+0000","last_clean":"2023-07-21T22:15:53.040613+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:15:53.040613+0000","last_undegraded":"2023-07-21T22:15:53.040613+0000","last_fullsized":"2023-07-21T22:15:53.040613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:22.887613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:15:40.848961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039177+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:15:53.039177+0000","last_peered":"2023-07-21T22:15:53.039177+0000","last_clean":"2023-07-21T22:15:53.039177+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:15:53.039177+0000","last_undegraded":"2023-07-21T22:15:53.039177+0000","last_fullsized":"2023-07-21T22:15:53.039177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:07.499178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038020+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:15:53.038020+0000","last_peered":"2023-07-21T22:15:53.038020+0000","last_clean":"2023-07-21T22:15:53.038020+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:15:53.038020+0000","last_undegraded":"2023-07-21T22:15:53.038020+0000","last_fullsized":"2023-07-21T22:15:53.038020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:26.375743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043928+0000","last_change":"2023-07-21T22:15:53.043434+0000","last_active":"2023-07-21T22:15:53.043928+0000","last_peered":"2023-07-21T22:15:53.043928+0000","last_clean":"2023-07-21T22:15:53.043928+0000","last_became_active":"2023-07-21T22:15:53.043161+0000","last_became_peered":"2023-07-21T22:15:53.043161+0000","last_unstale":"2023-07-21T22:15:53.043928+0000","last_undegraded":"2023-07-21T22:15:53.043928+0000","last_fullsized":"2023-07-21T22:15:53.043928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:15.187751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038994+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:15:53.038994+0000","last_peered":"2023-07-21T22:15:53.038994+0000","last_clean":"2023-07-21T22:15:53.038994+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:15:53.038994+0000","last_undegraded":"2023-07-21T22:15:53.038994+0000","last_fullsized":"2023-07-21T22:15:53.038994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:48:54.003434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038084+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:15:53.038084+0000","last_peered":"2023-07-21T22:15:53.038084+0000","last_clean":"2023-07-21T22:15:53.038084+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:15:53.038084+0000","last_undegraded":"2023-07-21T22:15:53.038084+0000","last_fullsized":"2023-07-21T22:15:53.038084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:24:58.235810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":174,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.106811+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:16:12.106811+0000","last_peered":"2023-07-21T22:16:12.106811+0000","last_clean":"2023-07-21T22:16:12.106811+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:16:12.106811+0000","last_undegraded":"2023-07-21T22:16:12.106811+0000","last_fullsized":"2023-07-21T22:16:12.106811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:55.908002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038324+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:15:53.038324+0000","last_peered":"2023-07-21T22:15:53.038324+0000","last_clean":"2023-07-21T22:15:53.038324+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:15:53.038324+0000","last_undegraded":"2023-07-21T22:15:53.038324+0000","last_fullsized":"2023-07-21T22:15:53.038324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:48:35.468546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036139+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:15:53.036139+0000","last_peered":"2023-07-21T22:15:53.036139+0000","last_clean":"2023-07-21T22:15:53.036139+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:15:53.036139+0000","last_undegraded":"2023-07-21T22:15:53.036139+0000","last_fullsized":"2023-07-21T22:15:53.036139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:44.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152319+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:15:53.152319+0000","last_peered":"2023-07-21T22:15:53.152319+0000","last_clean":"2023-07-21T22:15:53.152319+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:15:53.152319+0000","last_undegraded":"2023-07-21T22:15:53.152319+0000","last_fullsized":"2023-07-21T22:15:53.152319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:50:28.765938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039401+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:15:53.039401+0000","last_peered":"2023-07-21T22:15:53.039401+0000","last_clean":"2023-07-21T22:15:53.039401+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:15:53.039401+0000","last_undegraded":"2023-07-21T22:15:53.039401+0000","last_fullsized":"2023-07-21T22:15:53.039401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:13.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041812+0000","last_change":"2023-07-21T22:15:53.041812+0000","last_active":"2023-07-21T22:15:53.041812+0000","last_peered":"2023-07-21T22:15:53.041812+0000","last_clean":"2023-07-21T22:15:53.041812+0000","last_became_active":"2023-07-21T22:15:53.041531+0000","last_became_peered":"2023-07-21T22:15:53.041531+0000","last_unstale":"2023-07-21T22:15:53.041812+0000","last_undegraded":"2023-07-21T22:15:53.041812+0000","last_fullsized":"2023-07-21T22:15:53.041812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:42.585690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.107270+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:16:12.107270+0000","last_peered":"2023-07-21T22:16:12.107270+0000","last_clean":"2023-07-21T22:16:12.107270+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:16:12.107270+0000","last_undegraded":"2023-07-21T22:16:12.107270+0000","last_fullsized":"2023-07-21T22:16:12.107270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:11.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57816,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":161,"num_read_kb":138,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040868+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:15:53.040868+0000","last_peered":"2023-07-21T22:15:53.040868+0000","last_clean":"2023-07-21T22:15:53.040868+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:15:53.040868+0000","last_undegraded":"2023-07-21T22:15:53.040868+0000","last_fullsized":"2023-07-21T22:15:53.040868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:47.935173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044645+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:15:53.044645+0000","last_peered":"2023-07-21T22:15:53.044645+0000","last_clean":"2023-07-21T22:15:53.044645+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:15:53.044645+0000","last_undegraded":"2023-07-21T22:15:53.044645+0000","last_fullsized":"2023-07-21T22:15:53.044645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:22.972635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038296+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:15:53.038296+0000","last_peered":"2023-07-21T22:15:53.038296+0000","last_clean":"2023-07-21T22:15:53.038296+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:15:53.038296+0000","last_undegraded":"2023-07-21T22:15:53.038296+0000","last_fullsized":"2023-07-21T22:15:53.038296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:13.685376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039022+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:15:53.039022+0000","last_peered":"2023-07-21T22:15:53.039022+0000","last_clean":"2023-07-21T22:15:53.039022+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:15:53.039022+0000","last_undegraded":"2023-07-21T22:15:53.039022+0000","last_fullsized":"2023-07-21T22:15:53.039022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:59.623892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040566+0000","last_change":"2023-07-21T22:15:53.040566+0000","last_active":"2023-07-21T22:15:53.040566+0000","last_peered":"2023-07-21T22:15:53.040566+0000","last_clean":"2023-07-21T22:15:53.040566+0000","last_became_active":"2023-07-21T22:15:53.040142+0000","last_became_peered":"2023-07-21T22:15:53.040142+0000","last_unstale":"2023-07-21T22:15:53.040566+0000","last_undegraded":"2023-07-21T22:15:53.040566+0000","last_fullsized":"2023-07-21T22:15:53.040566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:49.679368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.104617+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:16:10.104617+0000","last_peered":"2023-07-21T22:16:10.104617+0000","last_clean":"2023-07-21T22:16:10.104617+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:16:10.104617+0000","last_undegraded":"2023-07-21T22:16:10.104617+0000","last_fullsized":"2023-07-21T22:16:10.104617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:41.232090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036126+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:15:53.036126+0000","last_peered":"2023-07-21T22:15:53.036126+0000","last_clean":"2023-07-21T22:15:53.036126+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:15:53.036126+0000","last_undegraded":"2023-07-21T22:15:53.036126+0000","last_fullsized":"2023-07-21T22:15:53.036126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:00.043258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040503+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:15:53.040503+0000","last_peered":"2023-07-21T22:15:53.040503+0000","last_clean":"2023-07-21T22:15:53.040503+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:15:53.040503+0000","last_undegraded":"2023-07-21T22:15:53.040503+0000","last_fullsized":"2023-07-21T22:15:53.040503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:16:27.945731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042070+0000","last_change":"2023-07-21T22:15:53.042070+0000","last_active":"2023-07-21T22:15:53.042070+0000","last_peered":"2023-07-21T22:15:53.042070+0000","last_clean":"2023-07-21T22:15:53.042070+0000","last_became_active":"2023-07-21T22:15:53.041704+0000","last_became_peered":"2023-07-21T22:15:53.041704+0000","last_unstale":"2023-07-21T22:15:53.042070+0000","last_undegraded":"2023-07-21T22:15:53.042070+0000","last_fullsized":"2023-07-21T22:15:53.042070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:09.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041217+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:15:53.041217+0000","last_peered":"2023-07-21T22:15:53.041217+0000","last_clean":"2023-07-21T22:15:53.041217+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:15:53.041217+0000","last_undegraded":"2023-07-21T22:15:53.041217+0000","last_fullsized":"2023-07-21T22:15:53.041217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:07.204911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043616+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:15:53.043616+0000","last_peered":"2023-07-21T22:15:53.043616+0000","last_clean":"2023-07-21T22:15:53.043616+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:15:53.043616+0000","last_undegraded":"2023-07-21T22:15:53.043616+0000","last_fullsized":"2023-07-21T22:15:53.043616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.352059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038528+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:15:53.038528+0000","last_peered":"2023-07-21T22:15:53.038528+0000","last_clean":"2023-07-21T22:15:53.038528+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:15:53.038528+0000","last_undegraded":"2023-07-21T22:15:53.038528+0000","last_fullsized":"2023-07-21T22:15:53.038528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:31.287806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080667+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:15:53.080667+0000","last_peered":"2023-07-21T22:15:53.080667+0000","last_clean":"2023-07-21T22:15:53.080667+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:15:53.080667+0000","last_undegraded":"2023-07-21T22:15:53.080667+0000","last_fullsized":"2023-07-21T22:15:53.080667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:37.888682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:08.113031+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:16:08.113031+0000","last_peered":"2023-07-21T22:16:08.113031+0000","last_clean":"2023-07-21T22:16:08.113031+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:16:08.113031+0000","last_undegraded":"2023-07-21T22:16:08.113031+0000","last_fullsized":"2023-07-21T22:16:08.113031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:03:12.172610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080778+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:15:53.080778+0000","last_peered":"2023-07-21T22:15:53.080778+0000","last_clean":"2023-07-21T22:15:53.080778+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:15:53.080778+0000","last_undegraded":"2023-07-21T22:15:53.080778+0000","last_fullsized":"2023-07-21T22:15:53.080778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:50.140355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039406+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:15:53.039406+0000","last_peered":"2023-07-21T22:15:53.039406+0000","last_clean":"2023-07-21T22:15:53.039406+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:15:53.039406+0000","last_undegraded":"2023-07-21T22:15:53.039406+0000","last_fullsized":"2023-07-21T22:15:53.039406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:49:49.868555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":97,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:08.101354+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:16:08.101354+0000","last_peered":"2023-07-21T22:16:08.101354+0000","last_clean":"2023-07-21T22:16:08.101354+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:16:08.101354+0000","last_undegraded":"2023-07-21T22:16:08.101354+0000","last_fullsized":"2023-07-21T22:16:08.101354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:13:07.058472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041428+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:15:53.041428+0000","last_peered":"2023-07-21T22:15:53.041428+0000","last_clean":"2023-07-21T22:15:53.041428+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:15:53.041428+0000","last_undegraded":"2023-07-21T22:15:53.041428+0000","last_fullsized":"2023-07-21T22:15:53.041428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:51.070606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081766+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:15:53.081766+0000","last_peered":"2023-07-21T22:15:53.081766+0000","last_clean":"2023-07-21T22:15:53.081766+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:15:53.081766+0000","last_undegraded":"2023-07-21T22:15:53.081766+0000","last_fullsized":"2023-07-21T22:15:53.081766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:22:41.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035963+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:15:53.035963+0000","last_peered":"2023-07-21T22:15:53.035963+0000","last_clean":"2023-07-21T22:15:53.035963+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:15:53.035963+0000","last_undegraded":"2023-07-21T22:15:53.035963+0000","last_fullsized":"2023-07-21T22:15:53.035963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:40:40.251027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037702+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:15:53.037702+0000","last_peered":"2023-07-21T22:15:53.037702+0000","last_clean":"2023-07-21T22:15:53.037702+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:15:53.037702+0000","last_undegraded":"2023-07-21T22:15:53.037702+0000","last_fullsized":"2023-07-21T22:15:53.037702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:44:27.102398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035899+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:15:53.035899+0000","last_peered":"2023-07-21T22:15:53.035899+0000","last_clean":"2023-07-21T22:15:53.035899+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:15:53.035899+0000","last_undegraded":"2023-07-21T22:15:53.035899+0000","last_fullsized":"2023-07-21T22:15:53.035899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:21.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040944+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:15:53.040944+0000","last_peered":"2023-07-21T22:15:53.040944+0000","last_clean":"2023-07-21T22:15:53.040944+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:15:53.040944+0000","last_undegraded":"2023-07-21T22:15:53.040944+0000","last_fullsized":"2023-07-21T22:15:53.040944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:00:57.736308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038301+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:15:53.038301+0000","last_peered":"2023-07-21T22:15:53.038301+0000","last_clean":"2023-07-21T22:15:53.038301+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:15:53.038301+0000","last_undegraded":"2023-07-21T22:15:53.038301+0000","last_fullsized":"2023-07-21T22:15:53.038301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:15:20.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.049655+0000","last_change":"2023-07-21T22:15:53.049655+0000","last_active":"2023-07-21T22:15:53.049655+0000","last_peered":"2023-07-21T22:15:53.049655+0000","last_clean":"2023-07-21T22:15:53.049655+0000","last_became_active":"2023-07-21T22:15:53.044176+0000","last_became_peered":"2023-07-21T22:15:53.044176+0000","last_unstale":"2023-07-21T22:15:53.049655+0000","last_undegraded":"2023-07-21T22:15:53.049655+0000","last_fullsized":"2023-07-21T22:15:53.049655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:18.559755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041952+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:15:53.041952+0000","last_peered":"2023-07-21T22:15:53.041952+0000","last_clean":"2023-07-21T22:15:53.041952+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:15:53.041952+0000","last_undegraded":"2023-07-21T22:15:53.041952+0000","last_fullsized":"2023-07-21T22:15:53.041952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:53.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081167+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:15:53.081167+0000","last_peered":"2023-07-21T22:15:53.081167+0000","last_clean":"2023-07-21T22:15:53.081167+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:15:53.081167+0000","last_undegraded":"2023-07-21T22:15:53.081167+0000","last_fullsized":"2023-07-21T22:15:53.081167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:32:22.375441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152195+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:15:53.152195+0000","last_peered":"2023-07-21T22:15:53.152195+0000","last_clean":"2023-07-21T22:15:53.152195+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:15:53.152195+0000","last_undegraded":"2023-07-21T22:15:53.152195+0000","last_fullsized":"2023-07-21T22:15:53.152195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:33:50.917618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038042+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:15:53.038042+0000","last_peered":"2023-07-21T22:15:53.038042+0000","last_clean":"2023-07-21T22:15:53.038042+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:15:53.038042+0000","last_undegraded":"2023-07-21T22:15:53.038042+0000","last_fullsized":"2023-07-21T22:15:53.038042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:25.720248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040485+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:15:53.040485+0000","last_peered":"2023-07-21T22:15:53.040485+0000","last_clean":"2023-07-21T22:15:53.040485+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:15:53.040485+0000","last_undegraded":"2023-07-21T22:15:53.040485+0000","last_fullsized":"2023-07-21T22:15:53.040485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:09.802817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041150+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:15:53.041150+0000","last_peered":"2023-07-21T22:15:53.041150+0000","last_clean":"2023-07-21T22:15:53.041150+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:15:53.041150+0000","last_undegraded":"2023-07-21T22:15:53.041150+0000","last_fullsized":"2023-07-21T22:15:53.041150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:00:32.552673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043438+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:15:53.043438+0000","last_peered":"2023-07-21T22:15:53.043438+0000","last_clean":"2023-07-21T22:15:53.043438+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:15:53.043438+0000","last_undegraded":"2023-07-21T22:15:53.043438+0000","last_fullsized":"2023-07-21T22:15:53.043438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:10.540110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.053218+0000","last_change":"2023-07-21T22:15:53.052828+0000","last_active":"2023-07-21T22:15:53.053218+0000","last_peered":"2023-07-21T22:15:53.053218+0000","last_clean":"2023-07-21T22:15:53.053218+0000","last_became_active":"2023-07-21T22:15:53.044310+0000","last_became_peered":"2023-07-21T22:15:53.044310+0000","last_unstale":"2023-07-21T22:15:53.053218+0000","last_undegraded":"2023-07-21T22:15:53.053218+0000","last_fullsized":"2023-07-21T22:15:53.053218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:06.298156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041487+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:15:53.041487+0000","last_peered":"2023-07-21T22:15:53.041487+0000","last_clean":"2023-07-21T22:15:53.041487+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:15:53.041487+0000","last_undegraded":"2023-07-21T22:15:53.041487+0000","last_fullsized":"2023-07-21T22:15:53.041487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:20:04.518114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040652+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:15:53.040652+0000","last_peered":"2023-07-21T22:15:53.040652+0000","last_clean":"2023-07-21T22:15:53.040652+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:15:53.040652+0000","last_undegraded":"2023-07-21T22:15:53.040652+0000","last_fullsized":"2023-07-21T22:15:53.040652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:39.235625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036234+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:15:53.036234+0000","last_peered":"2023-07-21T22:15:53.036234+0000","last_clean":"2023-07-21T22:15:53.036234+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:15:53.036234+0000","last_undegraded":"2023-07-21T22:15:53.036234+0000","last_fullsized":"2023-07-21T22:15:53.036234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:46:13.244517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.106020+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:16:11.106020+0000","last_peered":"2023-07-21T22:16:11.106020+0000","last_clean":"2023-07-21T22:16:11.106020+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:16:11.106020+0000","last_undegraded":"2023-07-21T22:16:11.106020+0000","last_fullsized":"2023-07-21T22:16:11.106020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:48.941886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036358+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:15:53.036358+0000","last_peered":"2023-07-21T22:15:53.036358+0000","last_clean":"2023-07-21T22:15:53.036358+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:15:53.036358+0000","last_undegraded":"2023-07-21T22:15:53.036358+0000","last_fullsized":"2023-07-21T22:15:53.036358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:07.030261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039125+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:15:53.039125+0000","last_peered":"2023-07-21T22:15:53.039125+0000","last_clean":"2023-07-21T22:15:53.039125+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:15:53.039125+0000","last_undegraded":"2023-07-21T22:15:53.039125+0000","last_fullsized":"2023-07-21T22:15:53.039125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:55:02.807962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036062+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:15:53.036062+0000","last_peered":"2023-07-21T22:15:53.036062+0000","last_clean":"2023-07-21T22:15:53.036062+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:15:53.036062+0000","last_undegraded":"2023-07-21T22:15:53.036062+0000","last_fullsized":"2023-07-21T22:15:53.036062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:04:20.299792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038109+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:15:53.038109+0000","last_peered":"2023-07-21T22:15:53.038109+0000","last_clean":"2023-07-21T22:15:53.038109+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:15:53.038109+0000","last_undegraded":"2023-07-21T22:15:53.038109+0000","last_fullsized":"2023-07-21T22:15:53.038109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:40.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036311+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:15:53.036311+0000","last_peered":"2023-07-21T22:15:53.036311+0000","last_clean":"2023-07-21T22:15:53.036311+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:15:53.036311+0000","last_undegraded":"2023-07-21T22:15:53.036311+0000","last_fullsized":"2023-07-21T22:15:53.036311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:28:05.342503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043455+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:15:53.043455+0000","last_peered":"2023-07-21T22:15:53.043455+0000","last_clean":"2023-07-21T22:15:53.043455+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:15:53.043455+0000","last_undegraded":"2023-07-21T22:15:53.043455+0000","last_fullsized":"2023-07-21T22:15:53.043455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:56:12.123011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044352+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:15:53.044352+0000","last_peered":"2023-07-21T22:15:53.044352+0000","last_clean":"2023-07-21T22:15:53.044352+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:15:53.044352+0000","last_undegraded":"2023-07-21T22:15:53.044352+0000","last_fullsized":"2023-07-21T22:15:53.044352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:18:56.552168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044446+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:15:53.044446+0000","last_peered":"2023-07-21T22:15:53.044446+0000","last_clean":"2023-07-21T22:15:53.044446+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:15:53.044446+0000","last_undegraded":"2023-07-21T22:15:53.044446+0000","last_fullsized":"2023-07-21T22:15:53.044446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:01.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152181+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:15:53.152181+0000","last_peered":"2023-07-21T22:15:53.152181+0000","last_clean":"2023-07-21T22:15:53.152181+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:15:53.152181+0000","last_undegraded":"2023-07-21T22:15:53.152181+0000","last_fullsized":"2023-07-21T22:15:53.152181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:42.437133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037860+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:15:53.037860+0000","last_peered":"2023-07-21T22:15:53.037860+0000","last_clean":"2023-07-21T22:15:53.037860+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:15:53.037860+0000","last_undegraded":"2023-07-21T22:15:53.037860+0000","last_fullsized":"2023-07-21T22:15:53.037860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:46:12.184590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043738+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:15:53.043738+0000","last_peered":"2023-07-21T22:15:53.043738+0000","last_clean":"2023-07-21T22:15:53.043738+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:15:53.043738+0000","last_undegraded":"2023-07-21T22:15:53.043738+0000","last_fullsized":"2023-07-21T22:15:53.043738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:17.735466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.045030+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:15:53.045030+0000","last_peered":"2023-07-21T22:15:53.045030+0000","last_clean":"2023-07-21T22:15:53.045030+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:15:53.045030+0000","last_undegraded":"2023-07-21T22:15:53.045030+0000","last_fullsized":"2023-07-21T22:15:53.045030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:35:20.503504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041943+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:15:53.041943+0000","last_peered":"2023-07-21T22:15:53.041943+0000","last_clean":"2023-07-21T22:15:53.041943+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:15:53.041943+0000","last_undegraded":"2023-07-21T22:15:53.041943+0000","last_fullsized":"2023-07-21T22:15:53.041943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:03.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039144+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:15:53.039144+0000","last_peered":"2023-07-21T22:15:53.039144+0000","last_clean":"2023-07-21T22:15:53.039144+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:15:53.039144+0000","last_undegraded":"2023-07-21T22:15:53.039144+0000","last_fullsized":"2023-07-21T22:15:53.039144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:52:43.313944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041119+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:15:53.041119+0000","last_peered":"2023-07-21T22:15:53.041119+0000","last_clean":"2023-07-21T22:15:53.041119+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:15:53.041119+0000","last_undegraded":"2023-07-21T22:15:53.041119+0000","last_fullsized":"2023-07-21T22:15:53.041119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:12.043605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037985+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.037985+0000","last_peered":"2023-07-21T22:15:53.037985+0000","last_clean":"2023-07-21T22:15:53.037985+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:15:53.037985+0000","last_undegraded":"2023-07-21T22:15:53.037985+0000","last_fullsized":"2023-07-21T22:15:53.037985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:49.251492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038404+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:15:53.038404+0000","last_peered":"2023-07-21T22:15:53.038404+0000","last_clean":"2023-07-21T22:15:53.038404+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:15:53.038404+0000","last_undegraded":"2023-07-21T22:15:53.038404+0000","last_fullsized":"2023-07-21T22:15:53.038404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:28:33.815947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043592+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:15:53.043592+0000","last_peered":"2023-07-21T22:15:53.043592+0000","last_clean":"2023-07-21T22:15:53.043592+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:15:53.043592+0000","last_undegraded":"2023-07-21T22:15:53.043592+0000","last_fullsized":"2023-07-21T22:15:53.043592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:36:28.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64046,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":789,"num_read_kb":468,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":441176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.626}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.89000000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.68600000000000005,"5min":0.75,"15min":0.75},"last":0.499},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.51000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82999999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.54200000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":1.034,"15min":1.034},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.625,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.61299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91300000000000003,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.04,"5min":1.248,"15min":1.248},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91800000000000004,"5min":1.125,"15min":1.125},"last":0.68600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.871},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82299999999999995}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":441154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.29199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.499}]},{"osd":2,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.308,"15min":0.308},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.311}]},{"osd":4,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.496},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.375},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.48999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70499999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.52300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.9480000000000004,"5min":5.9480000000000004,"15min":5.9480000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.2400000000000002,"5min":4.2400000000000002,"15min":4.2400000000000002},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1820000000000004,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.69299999999999995,"15min":0.63400000000000001},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.4550000000000001,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.745}]},{"osd":9,"last update":"Fri Jul 21 22:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.71399999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.081,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.97099999999999997},"min":{"1min":0.79800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0680000000000001,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.80500000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.84399999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1033190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.309,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.52800000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.62}]},{"osd":3,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":1.024,"15min":1.024},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77400000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44}]},{"osd":5,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.624},"min":{"1min":0.57599999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":1.143},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.66700000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81799999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93200000000000005,"5min":1.173,"15min":1.173},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94699999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89000000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.61499999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97599999999999998,"5min":1.036,"15min":1.036},"last":0.83199999999999996}]},{"osd":11,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84899999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.49399999999999999,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":13.989000000000001,"5min":13.989000000000001,"15min":13.989000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.28399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57099999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.36599999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.70799999999999996,"15min":0.622},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.68600000000000005,"15min":0.59399999999999997},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":13.789999999999999,"5min":13.789999999999999,"15min":13.789999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.56899999999999995,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":14.023,"5min":14.023,"15min":14.023},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.435}]},{"osd":4,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.64500000000000002,"15min":0.59999999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":7.1189999999999998,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.75800000000000001,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":13.956,"5min":13.956,"15min":13.956},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.65600000000000003,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.70099999999999996,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":7.1529999999999996,"5min":7.1529999999999996,"15min":7.1529999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.204,"5min":0.81799999999999995,"15min":0.749},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":13.885999999999999,"5min":13.885999999999999,"15min":13.885999999999999},"last":0.72599999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.70699999999999996,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.78200000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.77900000000000003,"15min":0.755},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":7.2489999999999997,"5min":7.2489999999999997,"15min":7.2489999999999997},"last":0.495}]},{"osd":9,"last update":"Fri Jul 21 22:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.0609999999999999,"5min":7.0609999999999999,"15min":7.0609999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.137,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":14.058999999999999,"5min":14.058999999999999,"15min":14.058999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.274,"5min":0.878,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":13.920999999999999,"5min":13.920999999999999,"15min":13.920999999999999},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":499234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.50700000000000001},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78700000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.439,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.61899999999999999,"15min":0.57099999999999995},"min":{"1min":0.52300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.44700000000000001,"15min":0.38700000000000001},"min":{"1min":0.36099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.40799999999999997},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76400000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.372}]},{"osd":5,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.61199999999999999,"15min":0.54600000000000004},"min":{"1min":0.47599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91900000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.60999999999999999,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.75,"5min":0.629,"15min":0.60199999999999998},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90800000000000003,"5min":0.995,"15min":0.995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.59099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.75800000000000001,"15min":0.748},"min":{"1min":0.628,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75600000000000001,"5min":0.871,"15min":0.871},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71699999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.44800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59199999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65000000000000002,"5min":0.877,"15min":0.877},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74099999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.747,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76200000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76900000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":11,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.86199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":499724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.75,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63200000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.504,"15min":0.45100000000000001},"min":{"1min":0.36099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95499999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.45200000000000001},"min":{"1min":0.32900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.627,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98799999999999999,"5min":1.012,"15min":1.012},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71399999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97499999999999998,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1200000000000001,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.749}]}]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35099999999999998,"15min":0.32300000000000001},"min":{"1min":0.255,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.378,"15min":0.33600000000000002},"min":{"1min":0.28000000000000003,"5min":0.19,"15min":0.19},"max":{"1min":1.232,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":1.512,"15min":1.512},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.034,"5min":0.63300000000000001,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.545,"5min":10.545,"15min":10.545},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.434,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0329999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.42499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67300000000000004,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.67000000000000004,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":10.446999999999999,"5min":10.446999999999999,"15min":10.446999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63200000000000001,"15min":0.61499999999999999},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":1.839,"15min":1.839},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0409999999999999,"5min":1.653,"15min":1.653},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.48999999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.69899999999999995,"15min":0.64500000000000002},"min":{"1min":0.46999999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.579000000000001,"5min":10.579000000000001,"15min":10.579000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79400000000000004,"5min":0.998,"15min":0.998},"last":0.436},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.68100000000000005,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.869999999999999,"5min":10.869999999999999,"15min":10.869999999999999},"last":0.38400000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.76500000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73299999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69299999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.38100000000000001},"min":{"1min":0.34399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57499999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.34300000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.307}]},{"osd":2,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.628,"15min":0.59799999999999998},"min":{"1min":0.51400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":1.22,"15min":1.22},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.016,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.57199999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624}]},{"osd":6,"last update":"Fri Jul 21 22:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.435},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.55500000000000005,"5min":0.372,"15min":0.372},"max":{"1min":0.95099999999999996,"5min":1.105,"15min":1.105},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.5,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0169999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66200000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":1.286,"15min":1.286},"last":0.48399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51300000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67400000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1033295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83199999999999996,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.5,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82399999999999995,"5min":1.917,"15min":1.917},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.55000000000000004,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.948,"15min":1.948},"last":0.874}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59099999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.59299999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88800000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.58599999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.64500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97399999999999998,"5min":1.98,"15min":1.98},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:16:15.190 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 -- 172.21.15.93:0/2242894842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 msgr2=0x7f678c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 --2- 172.21.15.93:0/2242894842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c100010 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6774009990 tx=0x7f6774009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 -- 172.21.15.93:0/2242894842 shutdown_connections 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 --2- 172.21.15.93:0/2242894842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 0x7f678c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 --2- 172.21.15.93:0/2242894842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 0x7f678c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 --2- 172.21.15.93:0/2242894842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c100010 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 -- 172.21.15.93:0/2242894842 >> 172.21.15.93:0/2242894842 conn(0x7f678c0f9fd0 msgr2=0x7f678c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:15.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 -- 172.21.15.93:0/2242894842 shutdown_connections 2023-07-21T22:16:15.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.299+0000 7f679319e700 1 -- 172.21.15.93:0/2242894842 wait complete. 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 Processor -- start 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 -- start start 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 0x7f678c1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 0x7f678c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c19d370 con 0x7f678c0ffbc0 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c19d4b0 con 0x7f678c0fe9d0 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679319e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c19d640 con 0x7f678c100550 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43780/0 (socket says 172.21.15.93:43780) 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 -- 172.21.15.93:0/489148460 learned_addr learned my addr 172.21.15.93:0/489148460 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f6790f3a700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 0x7f678c1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f679173b700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 0x7f678c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 -- 172.21.15.93:0/489148460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 msgr2=0x7f678c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 0x7f678c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 -- 172.21.15.93:0/489148460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 msgr2=0x7f678c1957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 0x7f678c1957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 -- 172.21.15.93:0/489148460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740095f0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.300+0000 7f678bfff700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6774005910 tx=0x7f6774004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.301+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677401d070 con 0x7f678c0ffbc0 2023-07-21T22:16:15.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.301+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6774003f60 con 0x7f678c0ffbc0 2023-07-21T22:16:15.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.301+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774022da0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.301+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c19d8c0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.301+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c19dce0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.302+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c105ec0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.304+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6774003aa0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.304+0000 7f6789ffb700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 0x7f67780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.304+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67740ce5e0 con 0x7f678c0ffbc0 2023-07-21T22:16:15.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.304+0000 7f6790f3a700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 0x7f67780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.305+0000 7f6790f3a700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 0x7f67780a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6784009550 tx=0x7f67840091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:15.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.307+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6774097c90 con 0x7f678c0ffbc0 2023-07-21T22:16:15.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:15 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2696727738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:15.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:15 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1357030736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:15.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.493+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f678c060950 con 0x7f67780a12a0 2023-07-21T22:16:15.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.502+0000 7f6789ffb700 1 -- 172.21.15.93:0/489148460 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343490 (secure 0 0 0) 0x7f678c060950 con 0x7f67780a12a0 2023-07-21T22:16:15.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:15.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 msgr2=0x7f67780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 0x7f67780a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6784009550 tx=0x7f67840091b0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 msgr2=0x7f678c195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6774005910 tx=0x7f6774004730 comp rx=0 tx=0).stop 2023-07-21T22:16:15.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 shutdown_connections 2023-07-21T22:16:15.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f678c100550 0x7f678c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f67780a12a0 0x7f67780a3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f678c0fe9d0 0x7f678c1957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 --2- 172.21.15.93:0/489148460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f678c0ffbc0 0x7f678c195d30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 >> 172.21.15.93:0/489148460 conn(0x7f678c0f9fd0 msgr2=0x7f678c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:15.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 shutdown_connections 2023-07-21T22:16:15.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.506+0000 7f679319e700 1 -- 172.21.15.93:0/489148460 wait complete. 2023-07-21T22:16:15.556 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:16:15.562 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-07-21T22:16:15.228965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654660,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":935,"num_read_kb":592,"num_write":418,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336324,"kb_used_data":12900,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605500,"statfs":{"total":1151940427776,"available":1151596032000,"internally_reserved":0,"allocated":13209600,"data_stored":7243053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043777+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:15:53.043777+0000","last_peered":"2023-07-21T22:15:53.043777+0000","last_clean":"2023-07-21T22:15:53.043777+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:15:53.043777+0000","last_undegraded":"2023-07-21T22:15:53.043777+0000","last_fullsized":"2023-07-21T22:15:53.043777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:19:32.645830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152152+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:15:53.152152+0000","last_peered":"2023-07-21T22:15:53.152152+0000","last_clean":"2023-07-21T22:15:53.152152+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:15:53.152152+0000","last_undegraded":"2023-07-21T22:15:53.152152+0000","last_fullsized":"2023-07-21T22:15:53.152152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:12.886060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152220+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.152220+0000","last_peered":"2023-07-21T22:15:53.152220+0000","last_clean":"2023-07-21T22:15:53.152220+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:15:53.152220+0000","last_undegraded":"2023-07-21T22:15:53.152220+0000","last_fullsized":"2023-07-21T22:15:53.152220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:27.406439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082127+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:15:53.082127+0000","last_peered":"2023-07-21T22:15:53.082127+0000","last_clean":"2023-07-21T22:15:53.082127+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:15:53.082127+0000","last_undegraded":"2023-07-21T22:15:53.082127+0000","last_fullsized":"2023-07-21T22:15:53.082127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:48.099744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040768+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:15:53.040768+0000","last_peered":"2023-07-21T22:15:53.040768+0000","last_clean":"2023-07-21T22:15:53.040768+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:15:53.040768+0000","last_undegraded":"2023-07-21T22:15:53.040768+0000","last_fullsized":"2023-07-21T22:15:53.040768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:03:07.972409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:40:56.830770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035947+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:15:53.035947+0000","last_peered":"2023-07-21T22:15:53.035947+0000","last_clean":"2023-07-21T22:15:53.035947+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:15:53.035947+0000","last_undegraded":"2023-07-21T22:15:53.035947+0000","last_fullsized":"2023-07-21T22:15:53.035947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:47.456515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040964+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:15:53.040964+0000","last_peered":"2023-07-21T22:15:53.040964+0000","last_clean":"2023-07-21T22:15:53.040964+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:15:53.040964+0000","last_undegraded":"2023-07-21T22:15:53.040964+0000","last_fullsized":"2023-07-21T22:15:53.040964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:30.005830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038418+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:15:53.038418+0000","last_peered":"2023-07-21T22:15:53.038418+0000","last_clean":"2023-07-21T22:15:53.038418+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:15:53.038418+0000","last_undegraded":"2023-07-21T22:15:53.038418+0000","last_fullsized":"2023-07-21T22:15:53.038418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:58:57.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080966+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:15:53.080966+0000","last_peered":"2023-07-21T22:15:53.080966+0000","last_clean":"2023-07-21T22:15:53.080966+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:15:53.080966+0000","last_undegraded":"2023-07-21T22:15:53.080966+0000","last_fullsized":"2023-07-21T22:15:53.080966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:50.746825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036344+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:15:53.036344+0000","last_peered":"2023-07-21T22:15:53.036344+0000","last_clean":"2023-07-21T22:15:53.036344+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:15:53.036344+0000","last_undegraded":"2023-07-21T22:15:53.036344+0000","last_fullsized":"2023-07-21T22:15:53.036344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:24:07.779545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082003+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.082003+0000","last_peered":"2023-07-21T22:15:53.082003+0000","last_clean":"2023-07-21T22:15:53.082003+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:15:53.082003+0000","last_undegraded":"2023-07-21T22:15:53.082003+0000","last_fullsized":"2023-07-21T22:15:53.082003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:40.403683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082836+0000","last_change":"2023-07-21T22:15:53.082836+0000","last_active":"2023-07-21T22:15:53.082836+0000","last_peered":"2023-07-21T22:15:53.082836+0000","last_clean":"2023-07-21T22:15:53.082836+0000","last_became_active":"2023-07-21T22:15:53.082492+0000","last_became_peered":"2023-07-21T22:15:53.082492+0000","last_unstale":"2023-07-21T22:15:53.082836+0000","last_undegraded":"2023-07-21T22:15:53.082836+0000","last_fullsized":"2023-07-21T22:15:53.082836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:37.252281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152025+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:15:53.152025+0000","last_peered":"2023-07-21T22:15:53.152025+0000","last_clean":"2023-07-21T22:15:53.152025+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:15:53.152025+0000","last_undegraded":"2023-07-21T22:15:53.152025+0000","last_fullsized":"2023-07-21T22:15:53.152025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:00:20.418659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.151988+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:15:53.151988+0000","last_peered":"2023-07-21T22:15:53.151988+0000","last_clean":"2023-07-21T22:15:53.151988+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:15:53.151988+0000","last_undegraded":"2023-07-21T22:15:53.151988+0000","last_fullsized":"2023-07-21T22:15:53.151988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:30:09.436268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037879+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:15:53.037879+0000","last_peered":"2023-07-21T22:15:53.037879+0000","last_clean":"2023-07-21T22:15:53.037879+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:15:53.037879+0000","last_undegraded":"2023-07-21T22:15:53.037879+0000","last_fullsized":"2023-07-21T22:15:53.037879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:12.939463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043953+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:15:53.043953+0000","last_peered":"2023-07-21T22:15:53.043953+0000","last_clean":"2023-07-21T22:15:53.043953+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:15:53.043953+0000","last_undegraded":"2023-07-21T22:15:53.043953+0000","last_fullsized":"2023-07-21T22:15:53.043953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:23.203665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039040+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:15:53.039040+0000","last_peered":"2023-07-21T22:15:53.039040+0000","last_clean":"2023-07-21T22:15:53.039040+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:15:53.039040+0000","last_undegraded":"2023-07-21T22:15:53.039040+0000","last_fullsized":"2023-07-21T22:15:53.039040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:32.580202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044944+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:15:53.044944+0000","last_peered":"2023-07-21T22:15:53.044944+0000","last_clean":"2023-07-21T22:15:53.044944+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:15:53.044944+0000","last_undegraded":"2023-07-21T22:15:53.044944+0000","last_fullsized":"2023-07-21T22:15:53.044944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:34.324418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.116122+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:16:11.116122+0000","last_peered":"2023-07-21T22:16:11.116122+0000","last_clean":"2023-07-21T22:16:11.116122+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:16:11.116122+0000","last_undegraded":"2023-07-21T22:16:11.116122+0000","last_fullsized":"2023-07-21T22:16:11.116122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:37.453186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041881+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:15:53.041881+0000","last_peered":"2023-07-21T22:15:53.041881+0000","last_clean":"2023-07-21T22:15:53.041881+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:15:53.041881+0000","last_undegraded":"2023-07-21T22:15:53.041881+0000","last_fullsized":"2023-07-21T22:15:53.041881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:08:06.854091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042034+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:15:53.042034+0000","last_peered":"2023-07-21T22:15:53.042034+0000","last_clean":"2023-07-21T22:15:53.042034+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:15:53.042034+0000","last_undegraded":"2023-07-21T22:15:53.042034+0000","last_fullsized":"2023-07-21T22:15:53.042034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:50.856377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152334+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:15:53.152334+0000","last_peered":"2023-07-21T22:15:53.152334+0000","last_clean":"2023-07-21T22:15:53.152334+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:15:53.152334+0000","last_undegraded":"2023-07-21T22:15:53.152334+0000","last_fullsized":"2023-07-21T22:15:53.152334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:48:22.931628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041052+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:15:53.041052+0000","last_peered":"2023-07-21T22:15:53.041052+0000","last_clean":"2023-07-21T22:15:53.041052+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:15:53.041052+0000","last_undegraded":"2023-07-21T22:15:53.041052+0000","last_fullsized":"2023-07-21T22:15:53.041052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:54:35.987213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038150+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:15:53.038150+0000","last_peered":"2023-07-21T22:15:53.038150+0000","last_clean":"2023-07-21T22:15:53.038150+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:15:53.038150+0000","last_undegraded":"2023-07-21T22:15:53.038150+0000","last_fullsized":"2023-07-21T22:15:53.038150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:50.632781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040901+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:15:53.040901+0000","last_peered":"2023-07-21T22:15:53.040901+0000","last_clean":"2023-07-21T22:15:53.040901+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:15:53.040901+0000","last_undegraded":"2023-07-21T22:15:53.040901+0000","last_fullsized":"2023-07-21T22:15:53.040901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:50.098672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039265+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.039265+0000","last_peered":"2023-07-21T22:15:53.039265+0000","last_clean":"2023-07-21T22:15:53.039265+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:15:53.039265+0000","last_undegraded":"2023-07-21T22:15:53.039265+0000","last_fullsized":"2023-07-21T22:15:53.039265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:43.599400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.105103+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:16:10.105103+0000","last_peered":"2023-07-21T22:16:10.105103+0000","last_clean":"2023-07-21T22:16:10.105103+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:16:10.105103+0000","last_undegraded":"2023-07-21T22:16:10.105103+0000","last_fullsized":"2023-07-21T22:16:10.105103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:01:43.880287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040613+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:15:53.040613+0000","last_peered":"2023-07-21T22:15:53.040613+0000","last_clean":"2023-07-21T22:15:53.040613+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:15:53.040613+0000","last_undegraded":"2023-07-21T22:15:53.040613+0000","last_fullsized":"2023-07-21T22:15:53.040613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:22.887613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:15:40.848961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039177+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:15:53.039177+0000","last_peered":"2023-07-21T22:15:53.039177+0000","last_clean":"2023-07-21T22:15:53.039177+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:15:53.039177+0000","last_undegraded":"2023-07-21T22:15:53.039177+0000","last_fullsized":"2023-07-21T22:15:53.039177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:07.499178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038020+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:15:53.038020+0000","last_peered":"2023-07-21T22:15:53.038020+0000","last_clean":"2023-07-21T22:15:53.038020+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:15:53.038020+0000","last_undegraded":"2023-07-21T22:15:53.038020+0000","last_fullsized":"2023-07-21T22:15:53.038020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:26.375743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043928+0000","last_change":"2023-07-21T22:15:53.043434+0000","last_active":"2023-07-21T22:15:53.043928+0000","last_peered":"2023-07-21T22:15:53.043928+0000","last_clean":"2023-07-21T22:15:53.043928+0000","last_became_active":"2023-07-21T22:15:53.043161+0000","last_became_peered":"2023-07-21T22:15:53.043161+0000","last_unstale":"2023-07-21T22:15:53.043928+0000","last_undegraded":"2023-07-21T22:15:53.043928+0000","last_fullsized":"2023-07-21T22:15:53.043928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:15.187751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038994+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:15:53.038994+0000","last_peered":"2023-07-21T22:15:53.038994+0000","last_clean":"2023-07-21T22:15:53.038994+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:15:53.038994+0000","last_undegraded":"2023-07-21T22:15:53.038994+0000","last_fullsized":"2023-07-21T22:15:53.038994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:48:54.003434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038084+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:15:53.038084+0000","last_peered":"2023-07-21T22:15:53.038084+0000","last_clean":"2023-07-21T22:15:53.038084+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:15:53.038084+0000","last_undegraded":"2023-07-21T22:15:53.038084+0000","last_fullsized":"2023-07-21T22:15:53.038084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:24:58.235810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":174,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.106811+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:16:12.106811+0000","last_peered":"2023-07-21T22:16:12.106811+0000","last_clean":"2023-07-21T22:16:12.106811+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:16:12.106811+0000","last_undegraded":"2023-07-21T22:16:12.106811+0000","last_fullsized":"2023-07-21T22:16:12.106811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:55.908002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038324+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:15:53.038324+0000","last_peered":"2023-07-21T22:15:53.038324+0000","last_clean":"2023-07-21T22:15:53.038324+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:15:53.038324+0000","last_undegraded":"2023-07-21T22:15:53.038324+0000","last_fullsized":"2023-07-21T22:15:53.038324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:48:35.468546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036139+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:15:53.036139+0000","last_peered":"2023-07-21T22:15:53.036139+0000","last_clean":"2023-07-21T22:15:53.036139+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:15:53.036139+0000","last_undegraded":"2023-07-21T22:15:53.036139+0000","last_fullsized":"2023-07-21T22:15:53.036139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:44.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152319+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:15:53.152319+0000","last_peered":"2023-07-21T22:15:53.152319+0000","last_clean":"2023-07-21T22:15:53.152319+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:15:53.152319+0000","last_undegraded":"2023-07-21T22:15:53.152319+0000","last_fullsized":"2023-07-21T22:15:53.152319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:50:28.765938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039401+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:15:53.039401+0000","last_peered":"2023-07-21T22:15:53.039401+0000","last_clean":"2023-07-21T22:15:53.039401+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:15:53.039401+0000","last_undegraded":"2023-07-21T22:15:53.039401+0000","last_fullsized":"2023-07-21T22:15:53.039401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:13.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041812+0000","last_change":"2023-07-21T22:15:53.041812+0000","last_active":"2023-07-21T22:15:53.041812+0000","last_peered":"2023-07-21T22:15:53.041812+0000","last_clean":"2023-07-21T22:15:53.041812+0000","last_became_active":"2023-07-21T22:15:53.041531+0000","last_became_peered":"2023-07-21T22:15:53.041531+0000","last_unstale":"2023-07-21T22:15:53.041812+0000","last_undegraded":"2023-07-21T22:15:53.041812+0000","last_fullsized":"2023-07-21T22:15:53.041812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:42.585690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.107270+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:16:12.107270+0000","last_peered":"2023-07-21T22:16:12.107270+0000","last_clean":"2023-07-21T22:16:12.107270+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:16:12.107270+0000","last_undegraded":"2023-07-21T22:16:12.107270+0000","last_fullsized":"2023-07-21T22:16:12.107270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:11.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57816,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":161,"num_read_kb":138,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040868+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:15:53.040868+0000","last_peered":"2023-07-21T22:15:53.040868+0000","last_clean":"2023-07-21T22:15:53.040868+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:15:53.040868+0000","last_undegraded":"2023-07-21T22:15:53.040868+0000","last_fullsized":"2023-07-21T22:15:53.040868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:47.935173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044645+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:15:53.044645+0000","last_peered":"2023-07-21T22:15:53.044645+0000","last_clean":"2023-07-21T22:15:53.044645+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:15:53.044645+0000","last_undegraded":"2023-07-21T22:15:53.044645+0000","last_fullsized":"2023-07-21T22:15:53.044645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:22.972635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038296+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:15:53.038296+0000","last_peered":"2023-07-21T22:15:53.038296+0000","last_clean":"2023-07-21T22:15:53.038296+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:15:53.038296+0000","last_undegraded":"2023-07-21T22:15:53.038296+0000","last_fullsized":"2023-07-21T22:15:53.038296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:13.685376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039022+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:15:53.039022+0000","last_peered":"2023-07-21T22:15:53.039022+0000","last_clean":"2023-07-21T22:15:53.039022+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:15:53.039022+0000","last_undegraded":"2023-07-21T22:15:53.039022+0000","last_fullsized":"2023-07-21T22:15:53.039022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:59.623892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040566+0000","last_change":"2023-07-21T22:15:53.040566+0000","last_active":"2023-07-21T22:15:53.040566+0000","last_peered":"2023-07-21T22:15:53.040566+0000","last_clean":"2023-07-21T22:15:53.040566+0000","last_became_active":"2023-07-21T22:15:53.040142+0000","last_became_peered":"2023-07-21T22:15:53.040142+0000","last_unstale":"2023-07-21T22:15:53.040566+0000","last_undegraded":"2023-07-21T22:15:53.040566+0000","last_fullsized":"2023-07-21T22:15:53.040566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:49.679368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.104617+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:16:10.104617+0000","last_peered":"2023-07-21T22:16:10.104617+0000","last_clean":"2023-07-21T22:16:10.104617+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:16:10.104617+0000","last_undegraded":"2023-07-21T22:16:10.104617+0000","last_fullsized":"2023-07-21T22:16:10.104617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:41.232090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036126+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:15:53.036126+0000","last_peered":"2023-07-21T22:15:53.036126+0000","last_clean":"2023-07-21T22:15:53.036126+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:15:53.036126+0000","last_undegraded":"2023-07-21T22:15:53.036126+0000","last_fullsized":"2023-07-21T22:15:53.036126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:00.043258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040503+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:15:53.040503+0000","last_peered":"2023-07-21T22:15:53.040503+0000","last_clean":"2023-07-21T22:15:53.040503+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:15:53.040503+0000","last_undegraded":"2023-07-21T22:15:53.040503+0000","last_fullsized":"2023-07-21T22:15:53.040503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:16:27.945731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042070+0000","last_change":"2023-07-21T22:15:53.042070+0000","last_active":"2023-07-21T22:15:53.042070+0000","last_peered":"2023-07-21T22:15:53.042070+0000","last_clean":"2023-07-21T22:15:53.042070+0000","last_became_active":"2023-07-21T22:15:53.041704+0000","last_became_peered":"2023-07-21T22:15:53.041704+0000","last_unstale":"2023-07-21T22:15:53.042070+0000","last_undegraded":"2023-07-21T22:15:53.042070+0000","last_fullsized":"2023-07-21T22:15:53.042070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:09.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041217+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:15:53.041217+0000","last_peered":"2023-07-21T22:15:53.041217+0000","last_clean":"2023-07-21T22:15:53.041217+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:15:53.041217+0000","last_undegraded":"2023-07-21T22:15:53.041217+0000","last_fullsized":"2023-07-21T22:15:53.041217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:07.204911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043616+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:15:53.043616+0000","last_peered":"2023-07-21T22:15:53.043616+0000","last_clean":"2023-07-21T22:15:53.043616+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:15:53.043616+0000","last_undegraded":"2023-07-21T22:15:53.043616+0000","last_fullsized":"2023-07-21T22:15:53.043616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.352059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038528+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:15:53.038528+0000","last_peered":"2023-07-21T22:15:53.038528+0000","last_clean":"2023-07-21T22:15:53.038528+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:15:53.038528+0000","last_undegraded":"2023-07-21T22:15:53.038528+0000","last_fullsized":"2023-07-21T22:15:53.038528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:31.287806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080667+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:15:53.080667+0000","last_peered":"2023-07-21T22:15:53.080667+0000","last_clean":"2023-07-21T22:15:53.080667+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:15:53.080667+0000","last_undegraded":"2023-07-21T22:15:53.080667+0000","last_fullsized":"2023-07-21T22:15:53.080667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:37.888682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:13.118907+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:16:13.118907+0000","last_peered":"2023-07-21T22:16:13.118907+0000","last_clean":"2023-07-21T22:16:13.118907+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:16:13.118907+0000","last_undegraded":"2023-07-21T22:16:13.118907+0000","last_fullsized":"2023-07-21T22:16:13.118907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:03:12.172610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080778+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:15:53.080778+0000","last_peered":"2023-07-21T22:15:53.080778+0000","last_clean":"2023-07-21T22:15:53.080778+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:15:53.080778+0000","last_undegraded":"2023-07-21T22:15:53.080778+0000","last_fullsized":"2023-07-21T22:15:53.080778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:50.140355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039406+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:15:53.039406+0000","last_peered":"2023-07-21T22:15:53.039406+0000","last_clean":"2023-07-21T22:15:53.039406+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:15:53.039406+0000","last_undegraded":"2023-07-21T22:15:53.039406+0000","last_fullsized":"2023-07-21T22:15:53.039406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:49:49.868555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:13.108088+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:16:13.108088+0000","last_peered":"2023-07-21T22:16:13.108088+0000","last_clean":"2023-07-21T22:16:13.108088+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:16:13.108088+0000","last_undegraded":"2023-07-21T22:16:13.108088+0000","last_fullsized":"2023-07-21T22:16:13.108088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:13:07.058472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041428+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:15:53.041428+0000","last_peered":"2023-07-21T22:15:53.041428+0000","last_clean":"2023-07-21T22:15:53.041428+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:15:53.041428+0000","last_undegraded":"2023-07-21T22:15:53.041428+0000","last_fullsized":"2023-07-21T22:15:53.041428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:51.070606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081766+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:15:53.081766+0000","last_peered":"2023-07-21T22:15:53.081766+0000","last_clean":"2023-07-21T22:15:53.081766+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:15:53.081766+0000","last_undegraded":"2023-07-21T22:15:53.081766+0000","last_fullsized":"2023-07-21T22:15:53.081766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:22:41.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035963+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:15:53.035963+0000","last_peered":"2023-07-21T22:15:53.035963+0000","last_clean":"2023-07-21T22:15:53.035963+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:15:53.035963+0000","last_undegraded":"2023-07-21T22:15:53.035963+0000","last_fullsized":"2023-07-21T22:15:53.035963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:40:40.251027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037702+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:15:53.037702+0000","last_peered":"2023-07-21T22:15:53.037702+0000","last_clean":"2023-07-21T22:15:53.037702+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:15:53.037702+0000","last_undegraded":"2023-07-21T22:15:53.037702+0000","last_fullsized":"2023-07-21T22:15:53.037702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:44:27.102398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035899+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:15:53.035899+0000","last_peered":"2023-07-21T22:15:53.035899+0000","last_clean":"2023-07-21T22:15:53.035899+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:15:53.035899+0000","last_undegraded":"2023-07-21T22:15:53.035899+0000","last_fullsized":"2023-07-21T22:15:53.035899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:21.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040944+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:15:53.040944+0000","last_peered":"2023-07-21T22:15:53.040944+0000","last_clean":"2023-07-21T22:15:53.040944+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:15:53.040944+0000","last_undegraded":"2023-07-21T22:15:53.040944+0000","last_fullsized":"2023-07-21T22:15:53.040944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:00:57.736308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038301+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:15:53.038301+0000","last_peered":"2023-07-21T22:15:53.038301+0000","last_clean":"2023-07-21T22:15:53.038301+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:15:53.038301+0000","last_undegraded":"2023-07-21T22:15:53.038301+0000","last_fullsized":"2023-07-21T22:15:53.038301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:15:20.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.049655+0000","last_change":"2023-07-21T22:15:53.049655+0000","last_active":"2023-07-21T22:15:53.049655+0000","last_peered":"2023-07-21T22:15:53.049655+0000","last_clean":"2023-07-21T22:15:53.049655+0000","last_became_active":"2023-07-21T22:15:53.044176+0000","last_became_peered":"2023-07-21T22:15:53.044176+0000","last_unstale":"2023-07-21T22:15:53.049655+0000","last_undegraded":"2023-07-21T22:15:53.049655+0000","last_fullsized":"2023-07-21T22:15:53.049655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:18.559755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041952+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:15:53.041952+0000","last_peered":"2023-07-21T22:15:53.041952+0000","last_clean":"2023-07-21T22:15:53.041952+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:15:53.041952+0000","last_undegraded":"2023-07-21T22:15:53.041952+0000","last_fullsized":"2023-07-21T22:15:53.041952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:53.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081167+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:15:53.081167+0000","last_peered":"2023-07-21T22:15:53.081167+0000","last_clean":"2023-07-21T22:15:53.081167+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:15:53.081167+0000","last_undegraded":"2023-07-21T22:15:53.081167+0000","last_fullsized":"2023-07-21T22:15:53.081167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:32:22.375441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152195+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:15:53.152195+0000","last_peered":"2023-07-21T22:15:53.152195+0000","last_clean":"2023-07-21T22:15:53.152195+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:15:53.152195+0000","last_undegraded":"2023-07-21T22:15:53.152195+0000","last_fullsized":"2023-07-21T22:15:53.152195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:33:50.917618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038042+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:15:53.038042+0000","last_peered":"2023-07-21T22:15:53.038042+0000","last_clean":"2023-07-21T22:15:53.038042+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:15:53.038042+0000","last_undegraded":"2023-07-21T22:15:53.038042+0000","last_fullsized":"2023-07-21T22:15:53.038042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:25.720248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040485+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:15:53.040485+0000","last_peered":"2023-07-21T22:15:53.040485+0000","last_clean":"2023-07-21T22:15:53.040485+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:15:53.040485+0000","last_undegraded":"2023-07-21T22:15:53.040485+0000","last_fullsized":"2023-07-21T22:15:53.040485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:09.802817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041150+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:15:53.041150+0000","last_peered":"2023-07-21T22:15:53.041150+0000","last_clean":"2023-07-21T22:15:53.041150+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:15:53.041150+0000","last_undegraded":"2023-07-21T22:15:53.041150+0000","last_fullsized":"2023-07-21T22:15:53.041150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:00:32.552673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043438+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:15:53.043438+0000","last_peered":"2023-07-21T22:15:53.043438+0000","last_clean":"2023-07-21T22:15:53.043438+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:15:53.043438+0000","last_undegraded":"2023-07-21T22:15:53.043438+0000","last_fullsized":"2023-07-21T22:15:53.043438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:10.540110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.053218+0000","last_change":"2023-07-21T22:15:53.052828+0000","last_active":"2023-07-21T22:15:53.053218+0000","last_peered":"2023-07-21T22:15:53.053218+0000","last_clean":"2023-07-21T22:15:53.053218+0000","last_became_active":"2023-07-21T22:15:53.044310+0000","last_became_peered":"2023-07-21T22:15:53.044310+0000","last_unstale":"2023-07-21T22:15:53.053218+0000","last_undegraded":"2023-07-21T22:15:53.053218+0000","last_fullsized":"2023-07-21T22:15:53.053218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:06.298156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041487+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:15:53.041487+0000","last_peered":"2023-07-21T22:15:53.041487+0000","last_clean":"2023-07-21T22:15:53.041487+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:15:53.041487+0000","last_undegraded":"2023-07-21T22:15:53.041487+0000","last_fullsized":"2023-07-21T22:15:53.041487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:20:04.518114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040652+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:15:53.040652+0000","last_peered":"2023-07-21T22:15:53.040652+0000","last_clean":"2023-07-21T22:15:53.040652+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:15:53.040652+0000","last_undegraded":"2023-07-21T22:15:53.040652+0000","last_fullsized":"2023-07-21T22:15:53.040652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:39.235625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036234+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:15:53.036234+0000","last_peered":"2023-07-21T22:15:53.036234+0000","last_clean":"2023-07-21T22:15:53.036234+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:15:53.036234+0000","last_undegraded":"2023-07-21T22:15:53.036234+0000","last_fullsized":"2023-07-21T22:15:53.036234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:46:13.244517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.106020+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:16:11.106020+0000","last_peered":"2023-07-21T22:16:11.106020+0000","last_clean":"2023-07-21T22:16:11.106020+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:16:11.106020+0000","last_undegraded":"2023-07-21T22:16:11.106020+0000","last_fullsized":"2023-07-21T22:16:11.106020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:48.941886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036358+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:15:53.036358+0000","last_peered":"2023-07-21T22:15:53.036358+0000","last_clean":"2023-07-21T22:15:53.036358+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:15:53.036358+0000","last_undegraded":"2023-07-21T22:15:53.036358+0000","last_fullsized":"2023-07-21T22:15:53.036358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:07.030261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039125+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:15:53.039125+0000","last_peered":"2023-07-21T22:15:53.039125+0000","last_clean":"2023-07-21T22:15:53.039125+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:15:53.039125+0000","last_undegraded":"2023-07-21T22:15:53.039125+0000","last_fullsized":"2023-07-21T22:15:53.039125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:55:02.807962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036062+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:15:53.036062+0000","last_peered":"2023-07-21T22:15:53.036062+0000","last_clean":"2023-07-21T22:15:53.036062+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:15:53.036062+0000","last_undegraded":"2023-07-21T22:15:53.036062+0000","last_fullsized":"2023-07-21T22:15:53.036062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:04:20.299792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038109+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:15:53.038109+0000","last_peered":"2023-07-21T22:15:53.038109+0000","last_clean":"2023-07-21T22:15:53.038109+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:15:53.038109+0000","last_undegraded":"2023-07-21T22:15:53.038109+0000","last_fullsized":"2023-07-21T22:15:53.038109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:40.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036311+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:15:53.036311+0000","last_peered":"2023-07-21T22:15:53.036311+0000","last_clean":"2023-07-21T22:15:53.036311+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:15:53.036311+0000","last_undegraded":"2023-07-21T22:15:53.036311+0000","last_fullsized":"2023-07-21T22:15:53.036311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:28:05.342503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043455+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:15:53.043455+0000","last_peered":"2023-07-21T22:15:53.043455+0000","last_clean":"2023-07-21T22:15:53.043455+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:15:53.043455+0000","last_undegraded":"2023-07-21T22:15:53.043455+0000","last_fullsized":"2023-07-21T22:15:53.043455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:56:12.123011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044352+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:15:53.044352+0000","last_peered":"2023-07-21T22:15:53.044352+0000","last_clean":"2023-07-21T22:15:53.044352+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:15:53.044352+0000","last_undegraded":"2023-07-21T22:15:53.044352+0000","last_fullsized":"2023-07-21T22:15:53.044352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:18:56.552168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044446+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:15:53.044446+0000","last_peered":"2023-07-21T22:15:53.044446+0000","last_clean":"2023-07-21T22:15:53.044446+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:15:53.044446+0000","last_undegraded":"2023-07-21T22:15:53.044446+0000","last_fullsized":"2023-07-21T22:15:53.044446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:01.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152181+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:15:53.152181+0000","last_peered":"2023-07-21T22:15:53.152181+0000","last_clean":"2023-07-21T22:15:53.152181+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:15:53.152181+0000","last_undegraded":"2023-07-21T22:15:53.152181+0000","last_fullsized":"2023-07-21T22:15:53.152181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:42.437133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037860+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:15:53.037860+0000","last_peered":"2023-07-21T22:15:53.037860+0000","last_clean":"2023-07-21T22:15:53.037860+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:15:53.037860+0000","last_undegraded":"2023-07-21T22:15:53.037860+0000","last_fullsized":"2023-07-21T22:15:53.037860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:46:12.184590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043738+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:15:53.043738+0000","last_peered":"2023-07-21T22:15:53.043738+0000","last_clean":"2023-07-21T22:15:53.043738+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:15:53.043738+0000","last_undegraded":"2023-07-21T22:15:53.043738+0000","last_fullsized":"2023-07-21T22:15:53.043738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:17.735466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.045030+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:15:53.045030+0000","last_peered":"2023-07-21T22:15:53.045030+0000","last_clean":"2023-07-21T22:15:53.045030+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:15:53.045030+0000","last_undegraded":"2023-07-21T22:15:53.045030+0000","last_fullsized":"2023-07-21T22:15:53.045030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:35:20.503504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041943+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:15:53.041943+0000","last_peered":"2023-07-21T22:15:53.041943+0000","last_clean":"2023-07-21T22:15:53.041943+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:15:53.041943+0000","last_undegraded":"2023-07-21T22:15:53.041943+0000","last_fullsized":"2023-07-21T22:15:53.041943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:03.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039144+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:15:53.039144+0000","last_peered":"2023-07-21T22:15:53.039144+0000","last_clean":"2023-07-21T22:15:53.039144+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:15:53.039144+0000","last_undegraded":"2023-07-21T22:15:53.039144+0000","last_fullsized":"2023-07-21T22:15:53.039144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:52:43.313944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041119+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:15:53.041119+0000","last_peered":"2023-07-21T22:15:53.041119+0000","last_clean":"2023-07-21T22:15:53.041119+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:15:53.041119+0000","last_undegraded":"2023-07-21T22:15:53.041119+0000","last_fullsized":"2023-07-21T22:15:53.041119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:12.043605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037985+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.037985+0000","last_peered":"2023-07-21T22:15:53.037985+0000","last_clean":"2023-07-21T22:15:53.037985+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:15:53.037985+0000","last_undegraded":"2023-07-21T22:15:53.037985+0000","last_fullsized":"2023-07-21T22:15:53.037985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:49.251492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038404+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:15:53.038404+0000","last_peered":"2023-07-21T22:15:53.038404+0000","last_clean":"2023-07-21T22:15:53.038404+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:15:53.038404+0000","last_undegraded":"2023-07-21T22:15:53.038404+0000","last_fullsized":"2023-07-21T22:15:53.038404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:28:33.815947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043592+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:15:53.043592+0000","last_peered":"2023-07-21T22:15:53.043592+0000","last_clean":"2023-07-21T22:15:53.043592+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:15:53.043592+0000","last_undegraded":"2023-07-21T22:15:53.043592+0000","last_fullsized":"2023-07-21T22:15:53.043592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:36:28.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64046,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":799,"num_read_kb":473,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":441176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.626}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.89000000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.68600000000000005,"5min":0.75,"15min":0.75},"last":0.499},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.51000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82999999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.54200000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":1.034,"15min":1.034},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.625,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.61299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91300000000000003,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.04,"5min":1.248,"15min":1.248},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91800000000000004,"5min":1.125,"15min":1.125},"last":0.68600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.871},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82299999999999995}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":441154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.29199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.499}]},{"osd":2,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.308,"15min":0.308},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.311}]},{"osd":4,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.496},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.375},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.48999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70499999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.52300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.9480000000000004,"5min":5.9480000000000004,"15min":5.9480000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.2400000000000002,"5min":4.2400000000000002,"15min":4.2400000000000002},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1820000000000004,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.69299999999999995,"15min":0.63400000000000001},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.4550000000000001,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.745}]},{"osd":9,"last update":"Fri Jul 21 22:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.71399999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.081,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.97099999999999997},"min":{"1min":0.79800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0680000000000001,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.80500000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.84399999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1033190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.309,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.52800000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":1.024,"15min":1.024},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77400000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.628}]},{"osd":4,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.624},"min":{"1min":0.57599999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":1.143},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.71899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.66700000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93200000000000005,"5min":1.173,"15min":1.173},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94699999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89000000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.61499999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97599999999999998,"5min":1.036,"15min":1.036},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.878}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.49399999999999999,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":13.989000000000001,"5min":13.989000000000001,"15min":13.989000000000001},"last":0.308},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.28399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57099999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.70799999999999996,"15min":0.622},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.68600000000000005,"15min":0.59399999999999997},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":13.789999999999999,"5min":13.789999999999999,"15min":13.789999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.56899999999999995,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":14.023,"5min":14.023,"15min":14.023},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.64500000000000002,"15min":0.59999999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":7.1189999999999998,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.75800000000000001,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":13.956,"5min":13.956,"15min":13.956},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.65600000000000003,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.70099999999999996,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":7.1529999999999996,"5min":7.1529999999999996,"15min":7.1529999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.204,"5min":0.81799999999999995,"15min":0.749},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":13.885999999999999,"5min":13.885999999999999,"15min":13.885999999999999},"last":0.49199999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71599999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.748,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.74299999999999999,"5min":7.2489999999999997,"15min":7.2489999999999997},"last":0.65600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.0609999999999999,"5min":7.0609999999999999,"15min":7.0609999999999999},"last":0.503},{"interface":"front","average":{"1min":1.137,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":14.058999999999999,"5min":14.058999999999999,"15min":14.058999999999999},"last":0.64700000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.274,"5min":0.878,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":13.920999999999999,"5min":13.920999999999999,"15min":13.920999999999999},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":499234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.50700000000000001},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78700000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.439,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.61899999999999999,"15min":0.57099999999999995},"min":{"1min":0.52300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.44700000000000001,"15min":0.38700000000000001},"min":{"1min":0.36099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.40799999999999997},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76400000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.372}]},{"osd":5,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.61199999999999999,"15min":0.54600000000000004},"min":{"1min":0.47599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91900000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.60999999999999999,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.75,"5min":0.629,"15min":0.60199999999999998},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90800000000000003,"5min":0.995,"15min":0.995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.59099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.75800000000000001,"15min":0.748},"min":{"1min":0.628,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75600000000000001,"5min":0.871,"15min":0.871},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71699999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.44800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59199999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65000000000000002,"5min":0.877,"15min":0.877},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74099999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.747,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76200000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76900000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":11,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.86199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":499724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.75,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63200000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.504,"15min":0.45100000000000001},"min":{"1min":0.36099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95499999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.45200000000000001},"min":{"1min":0.32900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.627,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98799999999999999,"5min":1.012,"15min":1.012},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71399999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97499999999999998,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1200000000000001,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.749}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35099999999999998,"15min":0.32300000000000001},"min":{"1min":0.255,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.438},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.378,"15min":0.33600000000000002},"min":{"1min":0.28000000000000003,"5min":0.19,"15min":0.19},"max":{"1min":1.232,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":1.512,"15min":1.512},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.034,"5min":0.63300000000000001,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.545,"5min":10.545,"15min":10.545},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.434,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0329999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.42499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67300000000000004,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":10.446999999999999,"15min":10.446999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0409999999999999,"5min":1.653,"15min":1.653},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.69899999999999995,"15min":0.64500000000000002},"min":{"1min":0.46999999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.579000000000001,"5min":10.579000000000001,"15min":10.579000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79400000000000004,"5min":0.998,"15min":0.998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.68100000000000005,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.869999999999999,"5min":10.869999999999999,"15min":10.869999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.76500000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60899999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.38100000000000001},"min":{"1min":0.34399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57499999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.34300000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.307}]},{"osd":2,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.628,"15min":0.59799999999999998},"min":{"1min":0.51400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":1.22,"15min":1.22},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.016,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.57199999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624}]},{"osd":6,"last update":"Fri Jul 21 22:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.435},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.55500000000000005,"5min":0.372,"15min":0.372},"max":{"1min":0.95099999999999996,"5min":1.105,"15min":1.105},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.5,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0169999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66200000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":1.286,"15min":1.286},"last":0.48399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51300000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67400000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1033295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83199999999999996,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.5,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82399999999999995,"5min":1.917,"15min":1.917},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.55000000000000004,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.948,"15min":1.948},"last":0.874}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59099999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.59299999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88800000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.58599999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.64500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97399999999999998,"5min":1.98,"15min":1.98},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:16:15.570 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-21T22:16:15.570 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:16:15.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:15 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2696727738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:15.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:15 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1357030736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:15.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 -- 172.21.15.93:0/2703148971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 msgr2=0x7f18d8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 --2- 172.21.15.93:0/2703148971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d8106d20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b0a0 tx=0x7f18d000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 -- 172.21.15.93:0/2703148971 shutdown_connections 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 --2- 172.21.15.93:0/2703148971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d8106d20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 --2- 172.21.15.93:0/2703148971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80fe8e0 0x7f18d80fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 --2- 172.21.15.93:0/2703148971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80ffad0 0x7f18d80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 -- 172.21.15.93:0/2703148971 >> 172.21.15.93:0/2703148971 conn(0x7f18d80f9ee0 msgr2=0x7f18d80fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:15.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 -- 172.21.15.93:0/2703148971 shutdown_connections 2023-07-21T22:16:15.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.680+0000 7f18dd99e700 1 -- 172.21.15.93:0/2703148971 wait complete. 2023-07-21T22:16:15.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 Processor -- start 2023-07-21T22:16:15.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 -- start start 2023-07-21T22:16:15.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80fe8e0 0x7f18d81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 0x7f18d8197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d819f350 con 0x7f18d8100460 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d819f490 con 0x7f18d80fe8e0 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd99e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d819f620 con 0x7f18d80ffad0 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43796/0 (socket says 172.21.15.93:43796) 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 -- 172.21.15.93:0/3425565034 learned_addr learned my addr 172.21.15.93:0/3425565034 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dc99c700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80fe8e0 0x7f18d81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18d7fff700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 0x7f18d8197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 -- 172.21.15.93:0/3425565034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 msgr2=0x7f18d8197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 0x7f18d8197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 -- 172.21.15.93:0/3425565034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80fe8e0 msgr2=0x7f18d81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.681+0000 7f18dd19d700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80fe8e0 0x7f18d81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18dd19d700 1 -- 172.21.15.93:0/3425565034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0009d00 con 0x7f18d8100460 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18d7fff700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 0x7f18d8197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18dd19d700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b780 tx=0x7f18d000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000e040 con 0x7f18d8100460 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18d0004e60 con 0x7f18d8100460 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d819f8a0 con 0x7f18d8100460 2023-07-21T22:16:15.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d001b740 con 0x7f18d8100460 2023-07-21T22:16:15.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.682+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d819fd20 con 0x7f18d8100460 2023-07-21T22:16:15.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.683+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d810a0c0 con 0x7f18d8100460 2023-07-21T22:16:15.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.684+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f18d001da40 con 0x7f18d8100460 2023-07-21T22:16:15.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.685+0000 7f18d5ffb700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 0x7f18c80a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:15.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.685+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f18d00cec40 con 0x7f18d8100460 2023-07-21T22:16:15.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.685+0000 7f18dc99c700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 0x7f18c80a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:15.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.685+0000 7f18dc99c700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 0x7f18c80a3790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ff930 tx=0x7f18cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:15.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.688+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f18d00982f0 con 0x7f18d8100460 2023-07-21T22:16:15.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:15 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2696727738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:15.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:15 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1357030736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:15.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.873+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18d8067e00 con 0x7f18c80a12e0 2023-07-21T22:16:15.926 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.883+0000 7f18d5ffb700 1 -- 172.21.15.93:0/3425565034 <== mgr.14152 v2:172.21.15.93:6800/2721150511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343490 (secure 0 0 0) 0x7f18d8067e00 con 0x7f18c80a12e0 2023-07-21T22:16:15.929 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:15.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 msgr2=0x7f18c80a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 0x7f18c80a3790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ff930 tx=0x7f18cc009260 comp rx=0 tx=0).stop 2023-07-21T22:16:15.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 msgr2=0x7f18d819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:15.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b780 tx=0x7f18d000bd00 comp rx=0 tx=0).stop 2023-07-21T22:16:15.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 shutdown_connections 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d8100460 0x7f18d819ed80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f18c80a12e0 0x7f18c80a3790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d80fe8e0 0x7f18d81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 --2- 172.21.15.93:0/3425565034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18d80ffad0 0x7f18d8197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 >> 172.21.15.93:0/3425565034 conn(0x7f18d80f9ee0 msgr2=0x7f18d80676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 shutdown_connections 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:15.886+0000 7f18dd99e700 1 -- 172.21.15.93:0/3425565034 wait complete. 2023-07-21T22:16:15.934 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-07-21T22:16:15.942 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-07-21T22:16:15.228965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654660,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":935,"num_read_kb":592,"num_write":418,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336324,"kb_used_data":12900,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605500,"statfs":{"total":1151940427776,"available":1151596032000,"internally_reserved":0,"allocated":13209600,"data_stored":7243053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043777+0000","last_change":"2023-07-21T22:14:51.250530+0000","last_active":"2023-07-21T22:15:53.043777+0000","last_peered":"2023-07-21T22:15:53.043777+0000","last_clean":"2023-07-21T22:15:53.043777+0000","last_became_active":"2023-07-21T22:14:51.250001+0000","last_became_peered":"2023-07-21T22:14:51.250001+0000","last_unstale":"2023-07-21T22:15:53.043777+0000","last_undegraded":"2023-07-21T22:15:53.043777+0000","last_fullsized":"2023-07-21T22:15:53.043777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:19:32.645830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152152+0000","last_change":"2023-07-21T22:14:51.252260+0000","last_active":"2023-07-21T22:15:53.152152+0000","last_peered":"2023-07-21T22:15:53.152152+0000","last_clean":"2023-07-21T22:15:53.152152+0000","last_became_active":"2023-07-21T22:14:51.251913+0000","last_became_peered":"2023-07-21T22:14:51.251913+0000","last_unstale":"2023-07-21T22:15:53.152152+0000","last_undegraded":"2023-07-21T22:15:53.152152+0000","last_fullsized":"2023-07-21T22:15:53.152152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:12.886060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152220+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.152220+0000","last_peered":"2023-07-21T22:15:53.152220+0000","last_clean":"2023-07-21T22:15:53.152220+0000","last_became_active":"2023-07-21T22:14:51.249538+0000","last_became_peered":"2023-07-21T22:14:51.249538+0000","last_unstale":"2023-07-21T22:15:53.152220+0000","last_undegraded":"2023-07-21T22:15:53.152220+0000","last_fullsized":"2023-07-21T22:15:53.152220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:27.406439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082127+0000","last_change":"2023-07-21T22:14:51.250783+0000","last_active":"2023-07-21T22:15:53.082127+0000","last_peered":"2023-07-21T22:15:53.082127+0000","last_clean":"2023-07-21T22:15:53.082127+0000","last_became_active":"2023-07-21T22:14:51.250380+0000","last_became_peered":"2023-07-21T22:14:51.250380+0000","last_unstale":"2023-07-21T22:15:53.082127+0000","last_undegraded":"2023-07-21T22:15:53.082127+0000","last_fullsized":"2023-07-21T22:15:53.082127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:48.099744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040768+0000","last_change":"2023-07-21T22:14:51.261378+0000","last_active":"2023-07-21T22:15:53.040768+0000","last_peered":"2023-07-21T22:15:53.040768+0000","last_clean":"2023-07-21T22:15:53.040768+0000","last_became_active":"2023-07-21T22:14:51.261017+0000","last_became_peered":"2023-07-21T22:14:51.261017+0000","last_unstale":"2023-07-21T22:15:53.040768+0000","last_undegraded":"2023-07-21T22:15:53.040768+0000","last_fullsized":"2023-07-21T22:15:53.040768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:03:07.972409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:51.253180+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:51.252883+0000","last_became_peered":"2023-07-21T22:14:51.252883+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:40:56.830770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035947+0000","last_change":"2023-07-21T22:14:51.252311+0000","last_active":"2023-07-21T22:15:53.035947+0000","last_peered":"2023-07-21T22:15:53.035947+0000","last_clean":"2023-07-21T22:15:53.035947+0000","last_became_active":"2023-07-21T22:14:51.252023+0000","last_became_peered":"2023-07-21T22:14:51.252023+0000","last_unstale":"2023-07-21T22:15:53.035947+0000","last_undegraded":"2023-07-21T22:15:53.035947+0000","last_fullsized":"2023-07-21T22:15:53.035947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:47.456515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040964+0000","last_change":"2023-07-21T22:14:51.248786+0000","last_active":"2023-07-21T22:15:53.040964+0000","last_peered":"2023-07-21T22:15:53.040964+0000","last_clean":"2023-07-21T22:15:53.040964+0000","last_became_active":"2023-07-21T22:14:51.248349+0000","last_became_peered":"2023-07-21T22:14:51.248349+0000","last_unstale":"2023-07-21T22:15:53.040964+0000","last_undegraded":"2023-07-21T22:15:53.040964+0000","last_fullsized":"2023-07-21T22:15:53.040964+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:30.005830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038418+0000","last_change":"2023-07-21T22:14:51.253331+0000","last_active":"2023-07-21T22:15:53.038418+0000","last_peered":"2023-07-21T22:15:53.038418+0000","last_clean":"2023-07-21T22:15:53.038418+0000","last_became_active":"2023-07-21T22:14:51.253042+0000","last_became_peered":"2023-07-21T22:14:51.253042+0000","last_unstale":"2023-07-21T22:15:53.038418+0000","last_undegraded":"2023-07-21T22:15:53.038418+0000","last_fullsized":"2023-07-21T22:15:53.038418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:58:57.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080966+0000","last_change":"2023-07-21T22:14:51.251846+0000","last_active":"2023-07-21T22:15:53.080966+0000","last_peered":"2023-07-21T22:15:53.080966+0000","last_clean":"2023-07-21T22:15:53.080966+0000","last_became_active":"2023-07-21T22:14:51.251470+0000","last_became_peered":"2023-07-21T22:14:51.251470+0000","last_unstale":"2023-07-21T22:15:53.080966+0000","last_undegraded":"2023-07-21T22:15:53.080966+0000","last_fullsized":"2023-07-21T22:15:53.080966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:50.746825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036344+0000","last_change":"2023-07-21T22:14:51.249070+0000","last_active":"2023-07-21T22:15:53.036344+0000","last_peered":"2023-07-21T22:15:53.036344+0000","last_clean":"2023-07-21T22:15:53.036344+0000","last_became_active":"2023-07-21T22:14:51.248770+0000","last_became_peered":"2023-07-21T22:14:51.248770+0000","last_unstale":"2023-07-21T22:15:53.036344+0000","last_undegraded":"2023-07-21T22:15:53.036344+0000","last_fullsized":"2023-07-21T22:15:53.036344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:24:07.779545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082003+0000","last_change":"2023-07-21T22:14:51.249799+0000","last_active":"2023-07-21T22:15:53.082003+0000","last_peered":"2023-07-21T22:15:53.082003+0000","last_clean":"2023-07-21T22:15:53.082003+0000","last_became_active":"2023-07-21T22:14:51.249487+0000","last_became_peered":"2023-07-21T22:14:51.249487+0000","last_unstale":"2023-07-21T22:15:53.082003+0000","last_undegraded":"2023-07-21T22:15:53.082003+0000","last_fullsized":"2023-07-21T22:15:53.082003+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:40.403683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.082836+0000","last_change":"2023-07-21T22:15:53.082836+0000","last_active":"2023-07-21T22:15:53.082836+0000","last_peered":"2023-07-21T22:15:53.082836+0000","last_clean":"2023-07-21T22:15:53.082836+0000","last_became_active":"2023-07-21T22:15:53.082492+0000","last_became_peered":"2023-07-21T22:15:53.082492+0000","last_unstale":"2023-07-21T22:15:53.082836+0000","last_undegraded":"2023-07-21T22:15:53.082836+0000","last_fullsized":"2023-07-21T22:15:53.082836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:37.252281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152025+0000","last_change":"2023-07-21T22:14:51.252188+0000","last_active":"2023-07-21T22:15:53.152025+0000","last_peered":"2023-07-21T22:15:53.152025+0000","last_clean":"2023-07-21T22:15:53.152025+0000","last_became_active":"2023-07-21T22:14:51.251871+0000","last_became_peered":"2023-07-21T22:14:51.251871+0000","last_unstale":"2023-07-21T22:15:53.152025+0000","last_undegraded":"2023-07-21T22:15:53.152025+0000","last_fullsized":"2023-07-21T22:15:53.152025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:00:20.418659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.151988+0000","last_change":"2023-07-21T22:14:51.251598+0000","last_active":"2023-07-21T22:15:53.151988+0000","last_peered":"2023-07-21T22:15:53.151988+0000","last_clean":"2023-07-21T22:15:53.151988+0000","last_became_active":"2023-07-21T22:14:51.251250+0000","last_became_peered":"2023-07-21T22:14:51.251250+0000","last_unstale":"2023-07-21T22:15:53.151988+0000","last_undegraded":"2023-07-21T22:15:53.151988+0000","last_fullsized":"2023-07-21T22:15:53.151988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:30:09.436268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037879+0000","last_change":"2023-07-21T22:14:51.249644+0000","last_active":"2023-07-21T22:15:53.037879+0000","last_peered":"2023-07-21T22:15:53.037879+0000","last_clean":"2023-07-21T22:15:53.037879+0000","last_became_active":"2023-07-21T22:14:51.249325+0000","last_became_peered":"2023-07-21T22:14:51.249325+0000","last_unstale":"2023-07-21T22:15:53.037879+0000","last_undegraded":"2023-07-21T22:15:53.037879+0000","last_fullsized":"2023-07-21T22:15:53.037879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:12.939463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043953+0000","last_change":"2023-07-21T22:14:51.250146+0000","last_active":"2023-07-21T22:15:53.043953+0000","last_peered":"2023-07-21T22:15:53.043953+0000","last_clean":"2023-07-21T22:15:53.043953+0000","last_became_active":"2023-07-21T22:14:51.248897+0000","last_became_peered":"2023-07-21T22:14:51.248897+0000","last_unstale":"2023-07-21T22:15:53.043953+0000","last_undegraded":"2023-07-21T22:15:53.043953+0000","last_fullsized":"2023-07-21T22:15:53.043953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:23.203665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039040+0000","last_change":"2023-07-21T22:14:51.248549+0000","last_active":"2023-07-21T22:15:53.039040+0000","last_peered":"2023-07-21T22:15:53.039040+0000","last_clean":"2023-07-21T22:15:53.039040+0000","last_became_active":"2023-07-21T22:14:51.248221+0000","last_became_peered":"2023-07-21T22:14:51.248221+0000","last_unstale":"2023-07-21T22:15:53.039040+0000","last_undegraded":"2023-07-21T22:15:53.039040+0000","last_fullsized":"2023-07-21T22:15:53.039040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:32.580202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044944+0000","last_change":"2023-07-21T22:14:51.256084+0000","last_active":"2023-07-21T22:15:53.044944+0000","last_peered":"2023-07-21T22:15:53.044944+0000","last_clean":"2023-07-21T22:15:53.044944+0000","last_became_active":"2023-07-21T22:14:51.255794+0000","last_became_peered":"2023-07-21T22:14:51.255794+0000","last_unstale":"2023-07-21T22:15:53.044944+0000","last_undegraded":"2023-07-21T22:15:53.044944+0000","last_fullsized":"2023-07-21T22:15:53.044944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:34.324418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.116122+0000","last_change":"2023-07-21T22:14:50.233432+0000","last_active":"2023-07-21T22:16:11.116122+0000","last_peered":"2023-07-21T22:16:11.116122+0000","last_clean":"2023-07-21T22:16:11.116122+0000","last_became_active":"2023-07-21T22:14:50.233125+0000","last_became_peered":"2023-07-21T22:14:50.233125+0000","last_unstale":"2023-07-21T22:16:11.116122+0000","last_undegraded":"2023-07-21T22:16:11.116122+0000","last_fullsized":"2023-07-21T22:16:11.116122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:37.453186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041881+0000","last_change":"2023-07-21T22:14:51.253979+0000","last_active":"2023-07-21T22:15:53.041881+0000","last_peered":"2023-07-21T22:15:53.041881+0000","last_clean":"2023-07-21T22:15:53.041881+0000","last_became_active":"2023-07-21T22:14:51.250957+0000","last_became_peered":"2023-07-21T22:14:51.250957+0000","last_unstale":"2023-07-21T22:15:53.041881+0000","last_undegraded":"2023-07-21T22:15:53.041881+0000","last_fullsized":"2023-07-21T22:15:53.041881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:08:06.854091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042034+0000","last_change":"2023-07-21T22:14:50.231754+0000","last_active":"2023-07-21T22:15:53.042034+0000","last_peered":"2023-07-21T22:15:53.042034+0000","last_clean":"2023-07-21T22:15:53.042034+0000","last_became_active":"2023-07-21T22:14:50.230105+0000","last_became_peered":"2023-07-21T22:14:50.230105+0000","last_unstale":"2023-07-21T22:15:53.042034+0000","last_undegraded":"2023-07-21T22:15:53.042034+0000","last_fullsized":"2023-07-21T22:15:53.042034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:50.856377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152334+0000","last_change":"2023-07-21T22:14:51.252225+0000","last_active":"2023-07-21T22:15:53.152334+0000","last_peered":"2023-07-21T22:15:53.152334+0000","last_clean":"2023-07-21T22:15:53.152334+0000","last_became_active":"2023-07-21T22:14:51.251680+0000","last_became_peered":"2023-07-21T22:14:51.251680+0000","last_unstale":"2023-07-21T22:15:53.152334+0000","last_undegraded":"2023-07-21T22:15:53.152334+0000","last_fullsized":"2023-07-21T22:15:53.152334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:48:22.931628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041052+0000","last_change":"2023-07-21T22:14:50.229208+0000","last_active":"2023-07-21T22:15:53.041052+0000","last_peered":"2023-07-21T22:15:53.041052+0000","last_clean":"2023-07-21T22:15:53.041052+0000","last_became_active":"2023-07-21T22:14:50.228923+0000","last_became_peered":"2023-07-21T22:14:50.228923+0000","last_unstale":"2023-07-21T22:15:53.041052+0000","last_undegraded":"2023-07-21T22:15:53.041052+0000","last_fullsized":"2023-07-21T22:15:53.041052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:54:35.987213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038150+0000","last_change":"2023-07-21T22:14:51.250350+0000","last_active":"2023-07-21T22:15:53.038150+0000","last_peered":"2023-07-21T22:15:53.038150+0000","last_clean":"2023-07-21T22:15:53.038150+0000","last_became_active":"2023-07-21T22:14:51.250068+0000","last_became_peered":"2023-07-21T22:14:51.250068+0000","last_unstale":"2023-07-21T22:15:53.038150+0000","last_undegraded":"2023-07-21T22:15:53.038150+0000","last_fullsized":"2023-07-21T22:15:53.038150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:50.632781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040901+0000","last_change":"2023-07-21T22:14:50.227282+0000","last_active":"2023-07-21T22:15:53.040901+0000","last_peered":"2023-07-21T22:15:53.040901+0000","last_clean":"2023-07-21T22:15:53.040901+0000","last_became_active":"2023-07-21T22:14:50.226956+0000","last_became_peered":"2023-07-21T22:14:50.226956+0000","last_unstale":"2023-07-21T22:15:53.040901+0000","last_undegraded":"2023-07-21T22:15:53.040901+0000","last_fullsized":"2023-07-21T22:15:53.040901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:50.098672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039265+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.039265+0000","last_peered":"2023-07-21T22:15:53.039265+0000","last_clean":"2023-07-21T22:15:53.039265+0000","last_became_active":"2023-07-21T22:14:51.254765+0000","last_became_peered":"2023-07-21T22:14:51.254765+0000","last_unstale":"2023-07-21T22:15:53.039265+0000","last_undegraded":"2023-07-21T22:15:53.039265+0000","last_fullsized":"2023-07-21T22:15:53.039265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:29:43.599400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.105103+0000","last_change":"2023-07-21T22:14:50.232617+0000","last_active":"2023-07-21T22:16:10.105103+0000","last_peered":"2023-07-21T22:16:10.105103+0000","last_clean":"2023-07-21T22:16:10.105103+0000","last_became_active":"2023-07-21T22:14:50.232226+0000","last_became_peered":"2023-07-21T22:14:50.232226+0000","last_unstale":"2023-07-21T22:16:10.105103+0000","last_undegraded":"2023-07-21T22:16:10.105103+0000","last_fullsized":"2023-07-21T22:16:10.105103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:01:43.880287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040613+0000","last_change":"2023-07-21T22:14:51.253378+0000","last_active":"2023-07-21T22:15:53.040613+0000","last_peered":"2023-07-21T22:15:53.040613+0000","last_clean":"2023-07-21T22:15:53.040613+0000","last_became_active":"2023-07-21T22:14:51.253048+0000","last_became_peered":"2023-07-21T22:14:51.253048+0000","last_unstale":"2023-07-21T22:15:53.040613+0000","last_undegraded":"2023-07-21T22:15:53.040613+0000","last_fullsized":"2023-07-21T22:15:53.040613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:22.887613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080548+0000","last_change":"2023-07-21T22:14:50.229996+0000","last_active":"2023-07-21T22:15:53.080548+0000","last_peered":"2023-07-21T22:15:53.080548+0000","last_clean":"2023-07-21T22:15:53.080548+0000","last_became_active":"2023-07-21T22:14:50.229710+0000","last_became_peered":"2023-07-21T22:14:50.229710+0000","last_unstale":"2023-07-21T22:15:53.080548+0000","last_undegraded":"2023-07-21T22:15:53.080548+0000","last_fullsized":"2023-07-21T22:15:53.080548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:15:40.848961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039177+0000","last_change":"2023-07-21T22:14:51.249717+0000","last_active":"2023-07-21T22:15:53.039177+0000","last_peered":"2023-07-21T22:15:53.039177+0000","last_clean":"2023-07-21T22:15:53.039177+0000","last_became_active":"2023-07-21T22:14:51.249456+0000","last_became_peered":"2023-07-21T22:14:51.249456+0000","last_unstale":"2023-07-21T22:15:53.039177+0000","last_undegraded":"2023-07-21T22:15:53.039177+0000","last_fullsized":"2023-07-21T22:15:53.039177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:07.499178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038020+0000","last_change":"2023-07-21T22:14:50.232063+0000","last_active":"2023-07-21T22:15:53.038020+0000","last_peered":"2023-07-21T22:15:53.038020+0000","last_clean":"2023-07-21T22:15:53.038020+0000","last_became_active":"2023-07-21T22:14:50.231669+0000","last_became_peered":"2023-07-21T22:14:50.231669+0000","last_unstale":"2023-07-21T22:15:53.038020+0000","last_undegraded":"2023-07-21T22:15:53.038020+0000","last_fullsized":"2023-07-21T22:15:53.038020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:26.375743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043928+0000","last_change":"2023-07-21T22:15:53.043434+0000","last_active":"2023-07-21T22:15:53.043928+0000","last_peered":"2023-07-21T22:15:53.043928+0000","last_clean":"2023-07-21T22:15:53.043928+0000","last_became_active":"2023-07-21T22:15:53.043161+0000","last_became_peered":"2023-07-21T22:15:53.043161+0000","last_unstale":"2023-07-21T22:15:53.043928+0000","last_undegraded":"2023-07-21T22:15:53.043928+0000","last_fullsized":"2023-07-21T22:15:53.043928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:15.187751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038994+0000","last_change":"2023-07-21T22:14:50.231813+0000","last_active":"2023-07-21T22:15:53.038994+0000","last_peered":"2023-07-21T22:15:53.038994+0000","last_clean":"2023-07-21T22:15:53.038994+0000","last_became_active":"2023-07-21T22:14:50.231510+0000","last_became_peered":"2023-07-21T22:14:50.231510+0000","last_unstale":"2023-07-21T22:15:53.038994+0000","last_undegraded":"2023-07-21T22:15:53.038994+0000","last_fullsized":"2023-07-21T22:15:53.038994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:48:54.003434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038084+0000","last_change":"2023-07-21T22:14:51.250259+0000","last_active":"2023-07-21T22:15:53.038084+0000","last_peered":"2023-07-21T22:15:53.038084+0000","last_clean":"2023-07-21T22:15:53.038084+0000","last_became_active":"2023-07-21T22:14:51.248161+0000","last_became_peered":"2023-07-21T22:14:51.248161+0000","last_unstale":"2023-07-21T22:15:53.038084+0000","last_undegraded":"2023-07-21T22:15:53.038084+0000","last_fullsized":"2023-07-21T22:15:53.038084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:24:58.235810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":174,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.106811+0000","last_change":"2023-07-21T22:14:50.228121+0000","last_active":"2023-07-21T22:16:12.106811+0000","last_peered":"2023-07-21T22:16:12.106811+0000","last_clean":"2023-07-21T22:16:12.106811+0000","last_became_active":"2023-07-21T22:14:50.227794+0000","last_became_peered":"2023-07-21T22:14:50.227794+0000","last_unstale":"2023-07-21T22:16:12.106811+0000","last_undegraded":"2023-07-21T22:16:12.106811+0000","last_fullsized":"2023-07-21T22:16:12.106811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:55.908002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038324+0000","last_change":"2023-07-21T22:14:51.253189+0000","last_active":"2023-07-21T22:15:53.038324+0000","last_peered":"2023-07-21T22:15:53.038324+0000","last_clean":"2023-07-21T22:15:53.038324+0000","last_became_active":"2023-07-21T22:14:51.252687+0000","last_became_peered":"2023-07-21T22:14:51.252687+0000","last_unstale":"2023-07-21T22:15:53.038324+0000","last_undegraded":"2023-07-21T22:15:53.038324+0000","last_fullsized":"2023-07-21T22:15:53.038324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:48:35.468546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036139+0000","last_change":"2023-07-21T22:14:50.231958+0000","last_active":"2023-07-21T22:15:53.036139+0000","last_peered":"2023-07-21T22:15:53.036139+0000","last_clean":"2023-07-21T22:15:53.036139+0000","last_became_active":"2023-07-21T22:14:50.231624+0000","last_became_peered":"2023-07-21T22:14:50.231624+0000","last_unstale":"2023-07-21T22:15:53.036139+0000","last_undegraded":"2023-07-21T22:15:53.036139+0000","last_fullsized":"2023-07-21T22:15:53.036139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:44.826615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152319+0000","last_change":"2023-07-21T22:14:51.257261+0000","last_active":"2023-07-21T22:15:53.152319+0000","last_peered":"2023-07-21T22:15:53.152319+0000","last_clean":"2023-07-21T22:15:53.152319+0000","last_became_active":"2023-07-21T22:14:51.256980+0000","last_became_peered":"2023-07-21T22:14:51.256980+0000","last_unstale":"2023-07-21T22:15:53.152319+0000","last_undegraded":"2023-07-21T22:15:53.152319+0000","last_fullsized":"2023-07-21T22:15:53.152319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:50:28.765938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039401+0000","last_change":"2023-07-21T22:14:50.228831+0000","last_active":"2023-07-21T22:15:53.039401+0000","last_peered":"2023-07-21T22:15:53.039401+0000","last_clean":"2023-07-21T22:15:53.039401+0000","last_became_active":"2023-07-21T22:14:50.228522+0000","last_became_peered":"2023-07-21T22:14:50.228522+0000","last_unstale":"2023-07-21T22:15:53.039401+0000","last_undegraded":"2023-07-21T22:15:53.039401+0000","last_fullsized":"2023-07-21T22:15:53.039401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:13.870585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041812+0000","last_change":"2023-07-21T22:15:53.041812+0000","last_active":"2023-07-21T22:15:53.041812+0000","last_peered":"2023-07-21T22:15:53.041812+0000","last_clean":"2023-07-21T22:15:53.041812+0000","last_became_active":"2023-07-21T22:15:53.041531+0000","last_became_peered":"2023-07-21T22:15:53.041531+0000","last_unstale":"2023-07-21T22:15:53.041812+0000","last_undegraded":"2023-07-21T22:15:53.041812+0000","last_fullsized":"2023-07-21T22:15:53.041812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:42.585690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:12.107270+0000","last_change":"2023-07-21T22:14:50.230914+0000","last_active":"2023-07-21T22:16:12.107270+0000","last_peered":"2023-07-21T22:16:12.107270+0000","last_clean":"2023-07-21T22:16:12.107270+0000","last_became_active":"2023-07-21T22:14:50.230591+0000","last_became_peered":"2023-07-21T22:14:50.230591+0000","last_unstale":"2023-07-21T22:16:12.107270+0000","last_undegraded":"2023-07-21T22:16:12.107270+0000","last_fullsized":"2023-07-21T22:16:12.107270+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:11.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57816,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":161,"num_read_kb":138,"num_write":35,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040868+0000","last_change":"2023-07-21T22:14:51.254154+0000","last_active":"2023-07-21T22:15:53.040868+0000","last_peered":"2023-07-21T22:15:53.040868+0000","last_clean":"2023-07-21T22:15:53.040868+0000","last_became_active":"2023-07-21T22:14:51.253836+0000","last_became_peered":"2023-07-21T22:14:51.253836+0000","last_unstale":"2023-07-21T22:15:53.040868+0000","last_undegraded":"2023-07-21T22:15:53.040868+0000","last_fullsized":"2023-07-21T22:15:53.040868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:47.935173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044645+0000","last_change":"2023-07-21T22:14:50.230686+0000","last_active":"2023-07-21T22:15:53.044645+0000","last_peered":"2023-07-21T22:15:53.044645+0000","last_clean":"2023-07-21T22:15:53.044645+0000","last_became_active":"2023-07-21T22:14:50.230385+0000","last_became_peered":"2023-07-21T22:14:50.230385+0000","last_unstale":"2023-07-21T22:15:53.044645+0000","last_undegraded":"2023-07-21T22:15:53.044645+0000","last_fullsized":"2023-07-21T22:15:53.044645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:22.972635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038296+0000","last_change":"2023-07-21T22:14:51.255282+0000","last_active":"2023-07-21T22:15:53.038296+0000","last_peered":"2023-07-21T22:15:53.038296+0000","last_clean":"2023-07-21T22:15:53.038296+0000","last_became_active":"2023-07-21T22:14:51.254386+0000","last_became_peered":"2023-07-21T22:14:51.254386+0000","last_unstale":"2023-07-21T22:15:53.038296+0000","last_undegraded":"2023-07-21T22:15:53.038296+0000","last_fullsized":"2023-07-21T22:15:53.038296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:13.685376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039022+0000","last_change":"2023-07-21T22:14:50.232226+0000","last_active":"2023-07-21T22:15:53.039022+0000","last_peered":"2023-07-21T22:15:53.039022+0000","last_clean":"2023-07-21T22:15:53.039022+0000","last_became_active":"2023-07-21T22:14:50.231921+0000","last_became_peered":"2023-07-21T22:14:50.231921+0000","last_unstale":"2023-07-21T22:15:53.039022+0000","last_undegraded":"2023-07-21T22:15:53.039022+0000","last_fullsized":"2023-07-21T22:15:53.039022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:59.623892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040566+0000","last_change":"2023-07-21T22:15:53.040566+0000","last_active":"2023-07-21T22:15:53.040566+0000","last_peered":"2023-07-21T22:15:53.040566+0000","last_clean":"2023-07-21T22:15:53.040566+0000","last_became_active":"2023-07-21T22:15:53.040142+0000","last_became_peered":"2023-07-21T22:15:53.040142+0000","last_unstale":"2023-07-21T22:15:53.040566+0000","last_undegraded":"2023-07-21T22:15:53.040566+0000","last_fullsized":"2023-07-21T22:15:53.040566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:49.679368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:10.104617+0000","last_change":"2023-07-21T22:14:50.230176+0000","last_active":"2023-07-21T22:16:10.104617+0000","last_peered":"2023-07-21T22:16:10.104617+0000","last_clean":"2023-07-21T22:16:10.104617+0000","last_became_active":"2023-07-21T22:14:50.229262+0000","last_became_peered":"2023-07-21T22:14:50.229262+0000","last_unstale":"2023-07-21T22:16:10.104617+0000","last_undegraded":"2023-07-21T22:16:10.104617+0000","last_fullsized":"2023-07-21T22:16:10.104617+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:41.232090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036126+0000","last_change":"2023-07-21T22:14:51.257033+0000","last_active":"2023-07-21T22:15:53.036126+0000","last_peered":"2023-07-21T22:15:53.036126+0000","last_clean":"2023-07-21T22:15:53.036126+0000","last_became_active":"2023-07-21T22:14:51.256767+0000","last_became_peered":"2023-07-21T22:14:51.256767+0000","last_unstale":"2023-07-21T22:15:53.036126+0000","last_undegraded":"2023-07-21T22:15:53.036126+0000","last_fullsized":"2023-07-21T22:15:53.036126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:00.043258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040503+0000","last_change":"2023-07-21T22:14:50.225558+0000","last_active":"2023-07-21T22:15:53.040503+0000","last_peered":"2023-07-21T22:15:53.040503+0000","last_clean":"2023-07-21T22:15:53.040503+0000","last_became_active":"2023-07-21T22:14:50.225059+0000","last_became_peered":"2023-07-21T22:14:50.225059+0000","last_unstale":"2023-07-21T22:15:53.040503+0000","last_undegraded":"2023-07-21T22:15:53.040503+0000","last_fullsized":"2023-07-21T22:15:53.040503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:16:27.945731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.042070+0000","last_change":"2023-07-21T22:15:53.042070+0000","last_active":"2023-07-21T22:15:53.042070+0000","last_peered":"2023-07-21T22:15:53.042070+0000","last_clean":"2023-07-21T22:15:53.042070+0000","last_became_active":"2023-07-21T22:15:53.041704+0000","last_became_peered":"2023-07-21T22:15:53.041704+0000","last_unstale":"2023-07-21T22:15:53.042070+0000","last_undegraded":"2023-07-21T22:15:53.042070+0000","last_fullsized":"2023-07-21T22:15:53.042070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:09.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041217+0000","last_change":"2023-07-21T22:14:50.231788+0000","last_active":"2023-07-21T22:15:53.041217+0000","last_peered":"2023-07-21T22:15:53.041217+0000","last_clean":"2023-07-21T22:15:53.041217+0000","last_became_active":"2023-07-21T22:14:50.231482+0000","last_became_peered":"2023-07-21T22:14:50.231482+0000","last_unstale":"2023-07-21T22:15:53.041217+0000","last_undegraded":"2023-07-21T22:15:53.041217+0000","last_fullsized":"2023-07-21T22:15:53.041217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:07.204911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043616+0000","last_change":"2023-07-21T22:14:51.245475+0000","last_active":"2023-07-21T22:15:53.043616+0000","last_peered":"2023-07-21T22:15:53.043616+0000","last_clean":"2023-07-21T22:15:53.043616+0000","last_became_active":"2023-07-21T22:14:51.245146+0000","last_became_peered":"2023-07-21T22:14:51.245146+0000","last_unstale":"2023-07-21T22:15:53.043616+0000","last_undegraded":"2023-07-21T22:15:53.043616+0000","last_fullsized":"2023-07-21T22:15:53.043616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.352059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038528+0000","last_change":"2023-07-21T22:14:50.234450+0000","last_active":"2023-07-21T22:15:53.038528+0000","last_peered":"2023-07-21T22:15:53.038528+0000","last_clean":"2023-07-21T22:15:53.038528+0000","last_became_active":"2023-07-21T22:14:50.233866+0000","last_became_peered":"2023-07-21T22:14:50.233866+0000","last_unstale":"2023-07-21T22:15:53.038528+0000","last_undegraded":"2023-07-21T22:15:53.038528+0000","last_fullsized":"2023-07-21T22:15:53.038528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:31.287806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080667+0000","last_change":"2023-07-21T22:14:51.256546+0000","last_active":"2023-07-21T22:15:53.080667+0000","last_peered":"2023-07-21T22:15:53.080667+0000","last_clean":"2023-07-21T22:15:53.080667+0000","last_became_active":"2023-07-21T22:14:51.256248+0000","last_became_peered":"2023-07-21T22:14:51.256248+0000","last_unstale":"2023-07-21T22:15:53.080667+0000","last_undegraded":"2023-07-21T22:15:53.080667+0000","last_fullsized":"2023-07-21T22:15:53.080667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:37.888682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:13.118907+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:16:13.118907+0000","last_peered":"2023-07-21T22:16:13.118907+0000","last_clean":"2023-07-21T22:16:13.118907+0000","last_became_active":"2023-07-21T22:14:50.225759+0000","last_became_peered":"2023-07-21T22:14:50.225759+0000","last_unstale":"2023-07-21T22:16:13.118907+0000","last_undegraded":"2023-07-21T22:16:13.118907+0000","last_fullsized":"2023-07-21T22:16:13.118907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:03:12.172610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.080778+0000","last_change":"2023-07-21T22:14:51.248581+0000","last_active":"2023-07-21T22:15:53.080778+0000","last_peered":"2023-07-21T22:15:53.080778+0000","last_clean":"2023-07-21T22:15:53.080778+0000","last_became_active":"2023-07-21T22:14:51.248251+0000","last_became_peered":"2023-07-21T22:14:51.248251+0000","last_unstale":"2023-07-21T22:15:53.080778+0000","last_undegraded":"2023-07-21T22:15:53.080778+0000","last_fullsized":"2023-07-21T22:15:53.080778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:14:50.140355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039406+0000","last_change":"2023-07-21T22:14:51.255946+0000","last_active":"2023-07-21T22:15:53.039406+0000","last_peered":"2023-07-21T22:15:53.039406+0000","last_clean":"2023-07-21T22:15:53.039406+0000","last_became_active":"2023-07-21T22:14:51.255193+0000","last_became_peered":"2023-07-21T22:14:51.255193+0000","last_unstale":"2023-07-21T22:15:53.039406+0000","last_undegraded":"2023-07-21T22:15:53.039406+0000","last_fullsized":"2023-07-21T22:15:53.039406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:49:49.868555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:13.108088+0000","last_change":"2023-07-21T22:14:50.237124+0000","last_active":"2023-07-21T22:16:13.108088+0000","last_peered":"2023-07-21T22:16:13.108088+0000","last_clean":"2023-07-21T22:16:13.108088+0000","last_became_active":"2023-07-21T22:14:50.236712+0000","last_became_peered":"2023-07-21T22:14:50.236712+0000","last_unstale":"2023-07-21T22:16:13.108088+0000","last_undegraded":"2023-07-21T22:16:13.108088+0000","last_fullsized":"2023-07-21T22:16:13.108088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:13:07.058472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041428+0000","last_change":"2023-07-21T22:14:51.255159+0000","last_active":"2023-07-21T22:15:53.041428+0000","last_peered":"2023-07-21T22:15:53.041428+0000","last_clean":"2023-07-21T22:15:53.041428+0000","last_became_active":"2023-07-21T22:14:51.254849+0000","last_became_peered":"2023-07-21T22:14:51.254849+0000","last_unstale":"2023-07-21T22:15:53.041428+0000","last_undegraded":"2023-07-21T22:15:53.041428+0000","last_fullsized":"2023-07-21T22:15:53.041428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:51.070606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081766+0000","last_change":"2023-07-21T22:14:50.226305+0000","last_active":"2023-07-21T22:15:53.081766+0000","last_peered":"2023-07-21T22:15:53.081766+0000","last_clean":"2023-07-21T22:15:53.081766+0000","last_became_active":"2023-07-21T22:14:50.225925+0000","last_became_peered":"2023-07-21T22:14:50.225925+0000","last_unstale":"2023-07-21T22:15:53.081766+0000","last_undegraded":"2023-07-21T22:15:53.081766+0000","last_fullsized":"2023-07-21T22:15:53.081766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:22:41.442254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035963+0000","last_change":"2023-07-21T22:13:50.999483+0000","last_active":"2023-07-21T22:15:53.035963+0000","last_peered":"2023-07-21T22:15:53.035963+0000","last_clean":"2023-07-21T22:15:53.035963+0000","last_became_active":"2023-07-21T22:13:50.998911+0000","last_became_peered":"2023-07-21T22:13:50.998911+0000","last_unstale":"2023-07-21T22:15:53.035963+0000","last_undegraded":"2023-07-21T22:15:53.035963+0000","last_fullsized":"2023-07-21T22:15:53.035963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:10:57.850242+0000","last_clean_scrub_stamp":"2023-07-21T22:10:57.850242+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:40:40.251027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037702+0000","last_change":"2023-07-21T22:14:51.250837+0000","last_active":"2023-07-21T22:15:53.037702+0000","last_peered":"2023-07-21T22:15:53.037702+0000","last_clean":"2023-07-21T22:15:53.037702+0000","last_became_active":"2023-07-21T22:14:51.250035+0000","last_became_peered":"2023-07-21T22:14:51.250035+0000","last_unstale":"2023-07-21T22:15:53.037702+0000","last_undegraded":"2023-07-21T22:15:53.037702+0000","last_fullsized":"2023-07-21T22:15:53.037702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:44:27.102398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.035899+0000","last_change":"2023-07-21T22:14:50.228637+0000","last_active":"2023-07-21T22:15:53.035899+0000","last_peered":"2023-07-21T22:15:53.035899+0000","last_clean":"2023-07-21T22:15:53.035899+0000","last_became_active":"2023-07-21T22:14:50.228331+0000","last_became_peered":"2023-07-21T22:14:50.228331+0000","last_unstale":"2023-07-21T22:15:53.035899+0000","last_undegraded":"2023-07-21T22:15:53.035899+0000","last_fullsized":"2023-07-21T22:15:53.035899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:21.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040944+0000","last_change":"2023-07-21T22:14:50.230744+0000","last_active":"2023-07-21T22:15:53.040944+0000","last_peered":"2023-07-21T22:15:53.040944+0000","last_clean":"2023-07-21T22:15:53.040944+0000","last_became_active":"2023-07-21T22:14:50.230441+0000","last_became_peered":"2023-07-21T22:14:50.230441+0000","last_unstale":"2023-07-21T22:15:53.040944+0000","last_undegraded":"2023-07-21T22:15:53.040944+0000","last_fullsized":"2023-07-21T22:15:53.040944+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:00:57.736308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038301+0000","last_change":"2023-07-21T22:14:51.250642+0000","last_active":"2023-07-21T22:15:53.038301+0000","last_peered":"2023-07-21T22:15:53.038301+0000","last_clean":"2023-07-21T22:15:53.038301+0000","last_became_active":"2023-07-21T22:14:51.250334+0000","last_became_peered":"2023-07-21T22:14:51.250334+0000","last_unstale":"2023-07-21T22:15:53.038301+0000","last_undegraded":"2023-07-21T22:15:53.038301+0000","last_fullsized":"2023-07-21T22:15:53.038301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:15:20.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.049655+0000","last_change":"2023-07-21T22:15:53.049655+0000","last_active":"2023-07-21T22:15:53.049655+0000","last_peered":"2023-07-21T22:15:53.049655+0000","last_clean":"2023-07-21T22:15:53.049655+0000","last_became_active":"2023-07-21T22:15:53.044176+0000","last_became_peered":"2023-07-21T22:15:53.044176+0000","last_unstale":"2023-07-21T22:15:53.049655+0000","last_undegraded":"2023-07-21T22:15:53.049655+0000","last_fullsized":"2023-07-21T22:15:53.049655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:18.559755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041952+0000","last_change":"2023-07-21T22:14:51.256247+0000","last_active":"2023-07-21T22:15:53.041952+0000","last_peered":"2023-07-21T22:15:53.041952+0000","last_clean":"2023-07-21T22:15:53.041952+0000","last_became_active":"2023-07-21T22:14:51.255984+0000","last_became_peered":"2023-07-21T22:14:51.255984+0000","last_unstale":"2023-07-21T22:15:53.041952+0000","last_undegraded":"2023-07-21T22:15:53.041952+0000","last_fullsized":"2023-07-21T22:15:53.041952+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:53.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.081167+0000","last_change":"2023-07-21T22:14:50.229424+0000","last_active":"2023-07-21T22:15:53.081167+0000","last_peered":"2023-07-21T22:15:53.081167+0000","last_clean":"2023-07-21T22:15:53.081167+0000","last_became_active":"2023-07-21T22:14:50.229115+0000","last_became_peered":"2023-07-21T22:14:50.229115+0000","last_unstale":"2023-07-21T22:15:53.081167+0000","last_undegraded":"2023-07-21T22:15:53.081167+0000","last_fullsized":"2023-07-21T22:15:53.081167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:32:22.375441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152195+0000","last_change":"2023-07-21T22:14:51.259211+0000","last_active":"2023-07-21T22:15:53.152195+0000","last_peered":"2023-07-21T22:15:53.152195+0000","last_clean":"2023-07-21T22:15:53.152195+0000","last_became_active":"2023-07-21T22:14:51.258857+0000","last_became_peered":"2023-07-21T22:14:51.258857+0000","last_unstale":"2023-07-21T22:15:53.152195+0000","last_undegraded":"2023-07-21T22:15:53.152195+0000","last_fullsized":"2023-07-21T22:15:53.152195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:33:50.917618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038042+0000","last_change":"2023-07-21T22:14:50.236108+0000","last_active":"2023-07-21T22:15:53.038042+0000","last_peered":"2023-07-21T22:15:53.038042+0000","last_clean":"2023-07-21T22:15:53.038042+0000","last_became_active":"2023-07-21T22:14:50.235843+0000","last_became_peered":"2023-07-21T22:14:50.235843+0000","last_unstale":"2023-07-21T22:15:53.038042+0000","last_undegraded":"2023-07-21T22:15:53.038042+0000","last_fullsized":"2023-07-21T22:15:53.038042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:25.720248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040485+0000","last_change":"2023-07-21T22:14:51.254137+0000","last_active":"2023-07-21T22:15:53.040485+0000","last_peered":"2023-07-21T22:15:53.040485+0000","last_clean":"2023-07-21T22:15:53.040485+0000","last_became_active":"2023-07-21T22:14:51.253202+0000","last_became_peered":"2023-07-21T22:14:51.253202+0000","last_unstale":"2023-07-21T22:15:53.040485+0000","last_undegraded":"2023-07-21T22:15:53.040485+0000","last_fullsized":"2023-07-21T22:15:53.040485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:09.802817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041150+0000","last_change":"2023-07-21T22:14:50.224094+0000","last_active":"2023-07-21T22:15:53.041150+0000","last_peered":"2023-07-21T22:15:53.041150+0000","last_clean":"2023-07-21T22:15:53.041150+0000","last_became_active":"2023-07-21T22:14:50.223742+0000","last_became_peered":"2023-07-21T22:14:50.223742+0000","last_unstale":"2023-07-21T22:15:53.041150+0000","last_undegraded":"2023-07-21T22:15:53.041150+0000","last_fullsized":"2023-07-21T22:15:53.041150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:00:32.552673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043438+0000","last_change":"2023-07-21T22:14:51.250535+0000","last_active":"2023-07-21T22:15:53.043438+0000","last_peered":"2023-07-21T22:15:53.043438+0000","last_clean":"2023-07-21T22:15:53.043438+0000","last_became_active":"2023-07-21T22:14:51.249843+0000","last_became_peered":"2023-07-21T22:14:51.249843+0000","last_unstale":"2023-07-21T22:15:53.043438+0000","last_undegraded":"2023-07-21T22:15:53.043438+0000","last_fullsized":"2023-07-21T22:15:53.043438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:10.540110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.053218+0000","last_change":"2023-07-21T22:15:53.052828+0000","last_active":"2023-07-21T22:15:53.053218+0000","last_peered":"2023-07-21T22:15:53.053218+0000","last_clean":"2023-07-21T22:15:53.053218+0000","last_became_active":"2023-07-21T22:15:53.044310+0000","last_became_peered":"2023-07-21T22:15:53.044310+0000","last_unstale":"2023-07-21T22:15:53.053218+0000","last_undegraded":"2023-07-21T22:15:53.053218+0000","last_fullsized":"2023-07-21T22:15:53.053218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:06.298156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041487+0000","last_change":"2023-07-21T22:14:51.262317+0000","last_active":"2023-07-21T22:15:53.041487+0000","last_peered":"2023-07-21T22:15:53.041487+0000","last_clean":"2023-07-21T22:15:53.041487+0000","last_became_active":"2023-07-21T22:14:51.255660+0000","last_became_peered":"2023-07-21T22:14:51.255660+0000","last_unstale":"2023-07-21T22:15:53.041487+0000","last_undegraded":"2023-07-21T22:15:53.041487+0000","last_fullsized":"2023-07-21T22:15:53.041487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:20:04.518114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.040652+0000","last_change":"2023-07-21T22:14:50.227809+0000","last_active":"2023-07-21T22:15:53.040652+0000","last_peered":"2023-07-21T22:15:53.040652+0000","last_clean":"2023-07-21T22:15:53.040652+0000","last_became_active":"2023-07-21T22:14:50.227499+0000","last_became_peered":"2023-07-21T22:14:50.227499+0000","last_unstale":"2023-07-21T22:15:53.040652+0000","last_undegraded":"2023-07-21T22:15:53.040652+0000","last_fullsized":"2023-07-21T22:15:53.040652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:39.235625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036234+0000","last_change":"2023-07-21T22:14:51.258427+0000","last_active":"2023-07-21T22:15:53.036234+0000","last_peered":"2023-07-21T22:15:53.036234+0000","last_clean":"2023-07-21T22:15:53.036234+0000","last_became_active":"2023-07-21T22:14:51.258107+0000","last_became_peered":"2023-07-21T22:14:51.258107+0000","last_unstale":"2023-07-21T22:15:53.036234+0000","last_undegraded":"2023-07-21T22:15:53.036234+0000","last_fullsized":"2023-07-21T22:15:53.036234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:46:13.244517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:16:11.106020+0000","last_change":"2023-07-21T22:14:50.231606+0000","last_active":"2023-07-21T22:16:11.106020+0000","last_peered":"2023-07-21T22:16:11.106020+0000","last_clean":"2023-07-21T22:16:11.106020+0000","last_became_active":"2023-07-21T22:14:50.231243+0000","last_became_peered":"2023-07-21T22:14:50.231243+0000","last_unstale":"2023-07-21T22:16:11.106020+0000","last_undegraded":"2023-07-21T22:16:11.106020+0000","last_fullsized":"2023-07-21T22:16:11.106020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:48.941886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036358+0000","last_change":"2023-07-21T22:14:51.258919+0000","last_active":"2023-07-21T22:15:53.036358+0000","last_peered":"2023-07-21T22:15:53.036358+0000","last_clean":"2023-07-21T22:15:53.036358+0000","last_became_active":"2023-07-21T22:14:51.258576+0000","last_became_peered":"2023-07-21T22:14:51.258576+0000","last_unstale":"2023-07-21T22:15:53.036358+0000","last_undegraded":"2023-07-21T22:15:53.036358+0000","last_fullsized":"2023-07-21T22:15:53.036358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:07.030261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039125+0000","last_change":"2023-07-21T22:14:50.235598+0000","last_active":"2023-07-21T22:15:53.039125+0000","last_peered":"2023-07-21T22:15:53.039125+0000","last_clean":"2023-07-21T22:15:53.039125+0000","last_became_active":"2023-07-21T22:14:50.235414+0000","last_became_peered":"2023-07-21T22:14:50.235414+0000","last_unstale":"2023-07-21T22:15:53.039125+0000","last_undegraded":"2023-07-21T22:15:53.039125+0000","last_fullsized":"2023-07-21T22:15:53.039125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:55:02.807962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036062+0000","last_change":"2023-07-21T22:14:51.263142+0000","last_active":"2023-07-21T22:15:53.036062+0000","last_peered":"2023-07-21T22:15:53.036062+0000","last_clean":"2023-07-21T22:15:53.036062+0000","last_became_active":"2023-07-21T22:14:51.262893+0000","last_became_peered":"2023-07-21T22:14:51.262893+0000","last_unstale":"2023-07-21T22:15:53.036062+0000","last_undegraded":"2023-07-21T22:15:53.036062+0000","last_fullsized":"2023-07-21T22:15:53.036062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:04:20.299792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038109+0000","last_change":"2023-07-21T22:14:50.235760+0000","last_active":"2023-07-21T22:15:53.038109+0000","last_peered":"2023-07-21T22:15:53.038109+0000","last_clean":"2023-07-21T22:15:53.038109+0000","last_became_active":"2023-07-21T22:14:50.235442+0000","last_became_peered":"2023-07-21T22:14:50.235442+0000","last_unstale":"2023-07-21T22:15:53.038109+0000","last_undegraded":"2023-07-21T22:15:53.038109+0000","last_fullsized":"2023-07-21T22:15:53.038109+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:49.207784+0000","last_clean_scrub_stamp":"2023-07-21T22:14:49.207784+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:28:40.999682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.036311+0000","last_change":"2023-07-21T22:14:51.262175+0000","last_active":"2023-07-21T22:15:53.036311+0000","last_peered":"2023-07-21T22:15:53.036311+0000","last_clean":"2023-07-21T22:15:53.036311+0000","last_became_active":"2023-07-21T22:14:51.261843+0000","last_became_peered":"2023-07-21T22:14:51.261843+0000","last_unstale":"2023-07-21T22:15:53.036311+0000","last_undegraded":"2023-07-21T22:15:53.036311+0000","last_fullsized":"2023-07-21T22:15:53.036311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:28:05.342503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043455+0000","last_change":"2023-07-21T22:14:51.250661+0000","last_active":"2023-07-21T22:15:53.043455+0000","last_peered":"2023-07-21T22:15:53.043455+0000","last_clean":"2023-07-21T22:15:53.043455+0000","last_became_active":"2023-07-21T22:14:51.250376+0000","last_became_peered":"2023-07-21T22:14:51.250376+0000","last_unstale":"2023-07-21T22:15:53.043455+0000","last_undegraded":"2023-07-21T22:15:53.043455+0000","last_fullsized":"2023-07-21T22:15:53.043455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:56:12.123011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044352+0000","last_change":"2023-07-21T22:14:51.252638+0000","last_active":"2023-07-21T22:15:53.044352+0000","last_peered":"2023-07-21T22:15:53.044352+0000","last_clean":"2023-07-21T22:15:53.044352+0000","last_became_active":"2023-07-21T22:14:51.252337+0000","last_became_peered":"2023-07-21T22:14:51.252337+0000","last_unstale":"2023-07-21T22:15:53.044352+0000","last_undegraded":"2023-07-21T22:15:53.044352+0000","last_fullsized":"2023-07-21T22:15:53.044352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:18:56.552168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.044446+0000","last_change":"2023-07-21T22:14:51.255364+0000","last_active":"2023-07-21T22:15:53.044446+0000","last_peered":"2023-07-21T22:15:53.044446+0000","last_clean":"2023-07-21T22:15:53.044446+0000","last_became_active":"2023-07-21T22:14:51.255069+0000","last_became_peered":"2023-07-21T22:14:51.255069+0000","last_unstale":"2023-07-21T22:15:53.044446+0000","last_undegraded":"2023-07-21T22:15:53.044446+0000","last_fullsized":"2023-07-21T22:15:53.044446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:01.263877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.152181+0000","last_change":"2023-07-21T22:14:51.251374+0000","last_active":"2023-07-21T22:15:53.152181+0000","last_peered":"2023-07-21T22:15:53.152181+0000","last_clean":"2023-07-21T22:15:53.152181+0000","last_became_active":"2023-07-21T22:14:51.251038+0000","last_became_peered":"2023-07-21T22:14:51.251038+0000","last_unstale":"2023-07-21T22:15:53.152181+0000","last_undegraded":"2023-07-21T22:15:53.152181+0000","last_fullsized":"2023-07-21T22:15:53.152181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:42.437133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037860+0000","last_change":"2023-07-21T22:14:51.249747+0000","last_active":"2023-07-21T22:15:53.037860+0000","last_peered":"2023-07-21T22:15:53.037860+0000","last_clean":"2023-07-21T22:15:53.037860+0000","last_became_active":"2023-07-21T22:14:51.249439+0000","last_became_peered":"2023-07-21T22:14:51.249439+0000","last_unstale":"2023-07-21T22:15:53.037860+0000","last_undegraded":"2023-07-21T22:15:53.037860+0000","last_fullsized":"2023-07-21T22:15:53.037860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:46:12.184590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043738+0000","last_change":"2023-07-21T22:14:51.245834+0000","last_active":"2023-07-21T22:15:53.043738+0000","last_peered":"2023-07-21T22:15:53.043738+0000","last_clean":"2023-07-21T22:15:53.043738+0000","last_became_active":"2023-07-21T22:14:51.245508+0000","last_became_peered":"2023-07-21T22:14:51.245508+0000","last_unstale":"2023-07-21T22:15:53.043738+0000","last_undegraded":"2023-07-21T22:15:53.043738+0000","last_fullsized":"2023-07-21T22:15:53.043738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:17.735466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.045030+0000","last_change":"2023-07-21T22:14:51.250493+0000","last_active":"2023-07-21T22:15:53.045030+0000","last_peered":"2023-07-21T22:15:53.045030+0000","last_clean":"2023-07-21T22:15:53.045030+0000","last_became_active":"2023-07-21T22:14:51.250142+0000","last_became_peered":"2023-07-21T22:14:51.250142+0000","last_unstale":"2023-07-21T22:15:53.045030+0000","last_undegraded":"2023-07-21T22:15:53.045030+0000","last_fullsized":"2023-07-21T22:15:53.045030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:35:20.503504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041943+0000","last_change":"2023-07-21T22:14:51.255018+0000","last_active":"2023-07-21T22:15:53.041943+0000","last_peered":"2023-07-21T22:15:53.041943+0000","last_clean":"2023-07-21T22:15:53.041943+0000","last_became_active":"2023-07-21T22:14:51.254727+0000","last_became_peered":"2023-07-21T22:14:51.254727+0000","last_unstale":"2023-07-21T22:15:53.041943+0000","last_undegraded":"2023-07-21T22:15:53.041943+0000","last_fullsized":"2023-07-21T22:15:53.041943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:03.680020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.039144+0000","last_change":"2023-07-21T22:14:51.257698+0000","last_active":"2023-07-21T22:15:53.039144+0000","last_peered":"2023-07-21T22:15:53.039144+0000","last_clean":"2023-07-21T22:15:53.039144+0000","last_became_active":"2023-07-21T22:14:51.257343+0000","last_became_peered":"2023-07-21T22:14:51.257343+0000","last_unstale":"2023-07-21T22:15:53.039144+0000","last_undegraded":"2023-07-21T22:15:53.039144+0000","last_fullsized":"2023-07-21T22:15:53.039144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:52:43.313944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.041119+0000","last_change":"2023-07-21T22:14:51.256656+0000","last_active":"2023-07-21T22:15:53.041119+0000","last_peered":"2023-07-21T22:15:53.041119+0000","last_clean":"2023-07-21T22:15:53.041119+0000","last_became_active":"2023-07-21T22:14:51.255498+0000","last_became_peered":"2023-07-21T22:14:51.255498+0000","last_unstale":"2023-07-21T22:15:53.041119+0000","last_undegraded":"2023-07-21T22:15:53.041119+0000","last_fullsized":"2023-07-21T22:15:53.041119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:12.043605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.037985+0000","last_change":"2023-07-21T22:14:51.255663+0000","last_active":"2023-07-21T22:15:53.037985+0000","last_peered":"2023-07-21T22:15:53.037985+0000","last_clean":"2023-07-21T22:15:53.037985+0000","last_became_active":"2023-07-21T22:14:51.255358+0000","last_became_peered":"2023-07-21T22:14:51.255358+0000","last_unstale":"2023-07-21T22:15:53.037985+0000","last_undegraded":"2023-07-21T22:15:53.037985+0000","last_fullsized":"2023-07-21T22:15:53.037985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:49.251492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.038404+0000","last_change":"2023-07-21T22:14:51.257786+0000","last_active":"2023-07-21T22:15:53.038404+0000","last_peered":"2023-07-21T22:15:53.038404+0000","last_clean":"2023-07-21T22:15:53.038404+0000","last_became_active":"2023-07-21T22:14:51.257434+0000","last_became_peered":"2023-07-21T22:14:51.257434+0000","last_unstale":"2023-07-21T22:15:53.038404+0000","last_undegraded":"2023-07-21T22:15:53.038404+0000","last_fullsized":"2023-07-21T22:15:53.038404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:28:33.815947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:15:53.043592+0000","last_change":"2023-07-21T22:14:51.249986+0000","last_active":"2023-07-21T22:15:53.043592+0000","last_peered":"2023-07-21T22:15:53.043592+0000","last_clean":"2023-07-21T22:15:53.043592+0000","last_became_active":"2023-07-21T22:14:51.249691+0000","last_became_peered":"2023-07-21T22:14:51.249691+0000","last_unstale":"2023-07-21T22:15:53.043592+0000","last_undegraded":"2023-07-21T22:15:53.043592+0000","last_fullsized":"2023-07-21T22:15:53.043592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:50.212384+0000","last_clean_scrub_stamp":"2023-07-21T22:14:50.212384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:36:28.567637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64046,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":799,"num_read_kb":473,"num_write":235,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":441176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.88},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.626}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.89000000000000001}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.68600000000000005,"5min":0.75,"15min":0.75},"last":0.499},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.497,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.59599999999999997,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.68100000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.51000000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82999999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70799999999999996,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85199999999999998,"5min":1.01,"15min":1.01},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.54200000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.879,"5min":1.034,"15min":1.034},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.625,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.61299999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.91300000000000003,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80800000000000005,"15min":0.81399999999999995},"min":{"1min":0.57099999999999995,"5min":0.503,"15min":0.503},"max":{"1min":1.04,"5min":1.248,"15min":1.248},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.80900000000000005,"15min":0.81899999999999995},"min":{"1min":0.57799999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.91800000000000004,"5min":1.125,"15min":1.125},"last":0.68600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.871},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.82299999999999995}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":441154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.47299999999999998,"15min":0.46000000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.48399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.29199999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.499}]},{"osd":2,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66600000000000004,"15min":0.67300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.309,"5min":0.308,"15min":0.308},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.311}]},{"osd":4,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.496},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.375},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.48999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79700000000000004,"15min":0.80800000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70499999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82899999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.52300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":5.9480000000000004,"5min":5.9480000000000004,"15min":5.9480000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.58099999999999996,"15min":0.55300000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.2400000000000002,"5min":4.2400000000000002,"15min":4.2400000000000002},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.45400000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.1820000000000004,"5min":6.1820000000000004,"15min":6.1820000000000004},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.41999999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.58299999999999996,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.69299999999999995,"15min":0.63400000000000001},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.4550000000000001,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.3630000000000004,"5min":6.3630000000000004,"15min":6.3630000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.67900000000000005,"15min":0.65400000000000003},"min":{"1min":0.45900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.74299999999999999,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":0.745}]},{"osd":9,"last update":"Fri Jul 21 22:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.75900000000000001,"15min":0.73099999999999998},"min":{"1min":0.63600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.71399999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.081,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96399999999999997,"15min":0.97099999999999997},"min":{"1min":0.79800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0680000000000001,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.80500000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.84399999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1033190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.8660000000000001,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54600000000000004,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.45500000000000002},"min":{"1min":0.309,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.46600000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.60599999999999998},"min":{"1min":0.52800000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.755,"5min":1.024,"15min":1.024},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.46600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77400000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.628}]},{"osd":4,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.41499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72799999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.34799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.66700000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64000000000000001,"15min":0.624},"min":{"1min":0.57599999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.45100000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68200000000000005,"15min":0.67600000000000005},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84499999999999997,"5min":1.143,"15min":1.143},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86499999999999999,"5min":1.036,"15min":1.036},"last":0.71899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.66700000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.50700000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89600000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93200000000000005,"5min":1.173,"15min":1.173},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.78800000000000003,"15min":0.78600000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94699999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76900000000000002,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89000000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.61499999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.97599999999999998,"5min":1.036,"15min":1.036},"last":0.79300000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.878}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.49399999999999999,"15min":0.40999999999999998},"min":{"1min":0.27500000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":13.989000000000001,"5min":13.989000000000001,"15min":13.989000000000001},"last":0.308},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38700000000000001,"15min":0.379},"min":{"1min":0.28399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.57099999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":0.70799999999999996,"15min":0.622},"min":{"1min":0.437,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.2800000000000002,"5min":7.2800000000000002,"15min":7.2800000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.68600000000000005,"15min":0.59399999999999997},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":13.789999999999999,"5min":13.789999999999999,"15min":13.789999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.56899999999999995,"15min":0.48399999999999999},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":14.023,"5min":14.023,"15min":14.023},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.64500000000000002,"15min":0.59999999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":7.1189999999999998,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":0.75800000000000001,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":13.956,"5min":13.956,"15min":13.956},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77300000000000002,"5min":0.878,"15min":0.878},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.65600000000000003,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.70099999999999996,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":7.1529999999999996,"5min":7.1529999999999996,"15min":7.1529999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.204,"5min":0.81799999999999995,"15min":0.749},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":13.885999999999999,"5min":13.885999999999999,"15min":13.885999999999999},"last":0.49199999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71599999999999997,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.748,"15min":0.745},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.74299999999999999,"5min":7.2489999999999997,"15min":7.2489999999999997},"last":0.65600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.67100000000000004,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.0609999999999999,"5min":7.0609999999999999,"15min":7.0609999999999999},"last":0.503},{"interface":"front","average":{"1min":1.137,"5min":0.78700000000000003,"15min":0.72899999999999998},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":14.058999999999999,"5min":14.058999999999999,"15min":14.058999999999999},"last":0.64700000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.274,"5min":0.878,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":13.920999999999999,"5min":13.920999999999999,"15min":13.920999999999999},"last":0.66700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":499234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.50700000000000001},"min":{"1min":0.40000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78700000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.51700000000000002},"min":{"1min":0.439,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.61899999999999999,"15min":0.57099999999999995},"min":{"1min":0.52300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.44700000000000001,"15min":0.38700000000000001},"min":{"1min":0.36099999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.46200000000000002,"15min":0.40799999999999997},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76400000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.372}]},{"osd":5,"last update":"Fri Jul 21 22:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.61199999999999999,"15min":0.54600000000000004},"min":{"1min":0.47599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91900000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.60999999999999999,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.59799999999999998,"15min":0.55400000000000005},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59599999999999997,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.748},{"interface":"front","average":{"1min":0.75,"5min":0.629,"15min":0.60199999999999998},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.64100000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.74199999999999999},"min":{"1min":0.53200000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90800000000000003,"5min":0.995,"15min":0.995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53800000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95899999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.59099999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.75800000000000001,"15min":0.748},"min":{"1min":0.628,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84199999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75600000000000001,"5min":0.871,"15min":0.871},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53200000000000003,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.71699999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55800000000000005,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84399999999999997,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70299999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.44800000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.874,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67500000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59199999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.5},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.53100000000000003,"15min":0.54300000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65000000000000002,"5min":0.877,"15min":0.877},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74099999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:15:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.59299999999999997,"15min":0.61399999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.70699999999999996,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.747,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 22:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76200000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70699999999999996,"15min":0.71699999999999997},"min":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76900000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":11,"last update":"Fri Jul 21 22:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.86199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":499724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.75,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63200000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91800000000000004,"5min":2.4489999999999998,"15min":2.4489999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.58599999999999997,"15min":0.53200000000000003},"min":{"1min":0.48099999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78700000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.504,"15min":0.45100000000000001},"min":{"1min":0.36099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.95499999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47999999999999998,"15min":0.45200000000000001},"min":{"1min":0.32900000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45600000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.627,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98799999999999999,"5min":1.012,"15min":1.012},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71399999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97499999999999998,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1200000000000001,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.749}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35099999999999998,"15min":0.32300000000000001},"min":{"1min":0.255,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.438},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.378,"15min":0.33600000000000002},"min":{"1min":0.28000000000000003,"5min":0.19,"15min":0.19},"max":{"1min":1.232,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76200000000000001,"5min":1.512,"15min":1.512},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.034,"5min":0.63300000000000001,"15min":0.55500000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.545,"5min":10.545,"15min":10.545},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.434,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0329999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54500000000000004},"min":{"1min":0.42499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67300000000000004,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.62}]},{"osd":5,"last update":"Fri Jul 21 22:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":0.59699999999999998},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":10.446999999999999,"15min":10.446999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56899999999999995,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0409999999999999,"5min":1.653,"15min":1.653},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.1749999999999998,"5min":3.1749999999999998,"15min":3.1749999999999998},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.69899999999999995,"15min":0.64500000000000002},"min":{"1min":0.46999999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.579000000000001,"5min":10.579000000000001,"15min":10.579000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70299999999999996,"15min":0.71899999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97599999999999998,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.501}]},{"osd":9,"last update":"Fri Jul 21 22:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79400000000000004,"5min":0.998,"15min":0.998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.68100000000000005,"15min":0.63200000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":10.869999999999999,"5min":10.869999999999999,"15min":10.869999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.67900000000000005,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.76500000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:15:50 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60899999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:14:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.88100000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.38100000000000001},"min":{"1min":0.34399999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57499999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.34300000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.307}]},{"osd":2,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.628,"15min":0.59799999999999998},"min":{"1min":0.51400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86499999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78800000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.55600000000000005},"min":{"1min":0.44600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":1.22,"15min":1.22},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.016,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65100000000000002,"15min":0.59599999999999997},"min":{"1min":0.57199999999999995,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89400000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.624}]},{"osd":6,"last update":"Fri Jul 21 22:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.435},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.625},"min":{"1min":0.55500000000000005,"5min":0.372,"15min":0.372},"max":{"1min":0.95099999999999996,"5min":1.105,"15min":1.105},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59599999999999997,"15min":0.56299999999999994},"min":{"1min":0.436,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34499999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:15:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.5,"5min":0.437,"15min":0.437},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.61199999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0169999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66200000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:15:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.83199999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80600000000000005,"15min":0.81000000000000005},"min":{"1min":0.59499999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99399999999999999,"5min":1.286,"15min":1.286},"last":0.48399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.92100000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68600000000000005,"15min":0.69099999999999995},"min":{"1min":0.52400000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.84199999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51300000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67400000000000004}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1033295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.505},"min":{"1min":0.40000000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83199999999999996,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38900000000000001,"15min":0.39100000000000001},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.5,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.376},"min":{"1min":0.307,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.45600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56799999999999995,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82399999999999995,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82699999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.442},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82399999999999995,"5min":1.917,"15min":1.917},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.55000000000000004,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.948,"15min":1.948},"last":0.874}]},{"osd":8,"last update":"Fri Jul 21 22:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.379,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59099999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.67100000000000004},"min":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.59299999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88800000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.58599999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.64500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97399999999999998,"5min":1.98,"15min":1.98},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:16:15.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:16.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 -- 172.21.15.93:0/88503676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 msgr2=0x7f2070107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 --2- 172.21.15.93:0/88503676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f2070107f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f206800b0a0 tx=0x7f206800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 -- 172.21.15.93:0/88503676 shutdown_connections 2023-07-21T22:16:16.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 --2- 172.21.15.93:0/88503676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f2070107f40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 --2- 172.21.15.93:0/88503676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f20700fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 --2- 172.21.15.93:0/88503676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 -- 172.21.15.93:0/88503676 >> 172.21.15.93:0/88503676 conn(0x7f20700fb100 msgr2=0x7f20700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 -- 172.21.15.93:0/88503676 shutdown_connections 2023-07-21T22:16:16.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.060+0000 7f207691b700 1 -- 172.21.15.93:0/88503676 wait complete. 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 Processor -- start 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 -- start start 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f2070195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019d410 con 0x7f2070101680 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019d550 con 0x7f2070100cf0 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f207691b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019d6e0 con 0x7f20700ffb00 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43810/0 (socket says 172.21.15.93:43810) 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 -- 172.21.15.93:0/2379871846 learned_addr learned my addr 172.21.15.93:0/2379871846 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f206f7fe700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f206ffff700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f2070195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 -- 172.21.15.93:0/2379871846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 msgr2=0x7f2070195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f2070195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 -- 172.21.15.93:0/2379871846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 msgr2=0x7f2070195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.061+0000 7f2074eb8700 1 -- 172.21.15.93:0/2379871846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068009d00 con 0x7f2070101680 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f206ffff700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f2070195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f206f7fe700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:16.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f2074eb8700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f206800b780 tx=0x7f206800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800e040 con 0x7f2070101680 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2068004e60 con 0x7f2070101680 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207019d960 con 0x7f2070101680 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206801b870 con 0x7f2070101680 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.062+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207019dde0 con 0x7f2070101680 2023-07-21T22:16:16.110 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.063+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2070106f50 con 0x7f2070101680 2023-07-21T22:16:16.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.064+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f206801b9d0 con 0x7f2070101680 2023-07-21T22:16:16.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.064+0000 7f206d7fa700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 0x7f205c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.064+0000 7f206ffff700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 0x7f205c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.064+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20680ce970 con 0x7f2070101680 2023-07-21T22:16:16.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.065+0000 7f206ffff700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 0x7f205c0a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2070100b50 tx=0x7f2058005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.068+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2068098020 con 0x7f2070101680 2023-07-21T22:16:16.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.259+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f207002c940 con 0x7f2070101680 2023-07-21T22:16:16.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.260+0000 7f206d7fa700 1 -- 172.21.15.93:0/2379871846 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7f206801bc80 con 0x7f2070101680 2023-07-21T22:16:16.304 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:16.305 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:16.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.262+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 msgr2=0x7f205c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 0x7f205c0a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2070100b50 tx=0x7f2058005f90 comp rx=0 tx=0).stop 2023-07-21T22:16:16.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 msgr2=0x7f207019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f206800b780 tx=0x7f206800bdb0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 shutdown_connections 2023-07-21T22:16:16.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2070101680 0x7f207019ce40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20700ffb00 0x7f2070195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f205c0a1220 0x7f205c0a36d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 --2- 172.21.15.93:0/2379871846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2070100cf0 0x7f2070195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 >> 172.21.15.93:0/2379871846 conn(0x7f20700fb100 msgr2=0x7f2070104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 shutdown_connections 2023-07-21T22:16:16.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.263+0000 7f207691b700 1 -- 172.21.15.93:0/2379871846 wait complete. 2023-07-21T22:16:16.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:16:16.319 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:16:16.444 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 -- 172.21.15.93:0/1456681027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0ff380 msgr2=0x7f967c0ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 --2- 172.21.15.93:0/1456681027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0ff380 0x7f967c0ff7d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9670009990 tx=0x7f9670009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 -- 172.21.15.93:0/1456681027 shutdown_connections 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 --2- 172.21.15.93:0/1456681027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f967c0ffd10 0x7f967c1065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 --2- 172.21.15.93:0/1456681027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0fe190 0x7f967c0fe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 --2- 172.21.15.93:0/1456681027 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0ff380 0x7f967c0ff7d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.445 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 -- 172.21.15.93:0/1456681027 >> 172.21.15.93:0/1456681027 conn(0x7f967c0f97b0 msgr2=0x7f967c0fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 -- 172.21.15.93:0/1456681027 shutdown_connections 2023-07-21T22:16:16.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.400+0000 7f9681759700 1 -- 172.21.15.93:0/1456681027 wait complete. 2023-07-21T22:16:16.446 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 Processor -- start 2023-07-21T22:16:16.447 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 -- start start 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f967c0ff380 0x7f967c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0ffd10 0x7f967c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19d390 con 0x7f967c0fe190 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19d500 con 0x7f967c0ffd10 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f9681759700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19d6c0 con 0x7f967c0ff380 2023-07-21T22:16:16.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f967affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f967affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43818/0 (socket says 172.21.15.93:43818) 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.401+0000 7f967affd700 1 -- 172.21.15.93:0/2546214827 learned_addr learned my addr 172.21.15.93:0/2546214827 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967b7fe700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0ffd10 0x7f967c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 -- 172.21.15.93:0/2546214827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f967c0ff380 msgr2=0x7f967c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f967c0ff380 0x7f967c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 -- 172.21.15.93:0/2546214827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0ffd10 msgr2=0x7f967c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0ffd10 0x7f967c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 -- 172.21.15.93:0/2546214827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700095f0 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f967affd700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f966c00c970 tx=0x7f966c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c004220 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f966c007d60 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c19d9a0 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c003d00 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.402+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c19df30 con 0x7f967c0fe190 2023-07-21T22:16:16.450 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.403+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967c105d00 con 0x7f967c0fe190 2023-07-21T22:16:16.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.404+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f966c004380 con 0x7f967c0fe190 2023-07-21T22:16:16.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.404+0000 7f9663fff700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 0x7f96640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.405+0000 7f967a7fc700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 0x7f96640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.405+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f966c0cca10 con 0x7f967c0fe190 2023-07-21T22:16:16.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.405+0000 7f967a7fc700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 0x7f96640a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f967000b5c0 tx=0x7f9670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.408+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f966c096030 con 0x7f967c0fe190 2023-07-21T22:16:16.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:16 smithi134 ceph-mon[97873]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:16 smithi134 ceph-mon[97873]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:16.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:16 smithi134 ceph-mon[97873]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:16 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2379871846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:16.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.594+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f967c02c9e0 con 0x7f967c0fe190 2023-07-21T22:16:16.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.594+0000 7f9663fff700 1 -- 172.21.15.93:0/2546214827 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f966c099650 con 0x7f967c0fe190 2023-07-21T22:16:16.638 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:16:16.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 msgr2=0x7f96640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 0x7f96640a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f967000b5c0 tx=0x7f9670005fb0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 msgr2=0x7f967c0fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f966c00c970 tx=0x7f966c00cd30 comp rx=0 tx=0).stop 2023-07-21T22:16:16.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 shutdown_connections 2023-07-21T22:16:16.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f96640a11e0 0x7f96640a3690 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0ffd10 0x7f967c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f967c0fe190 0x7f967c0fbbf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 --2- 172.21.15.93:0/2546214827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f967c0ff380 0x7f967c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 >> 172.21.15.93:0/2546214827 conn(0x7f967c0f97b0 msgr2=0x7f967c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 shutdown_connections 2023-07-21T22:16:16.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.596+0000 7f9681759700 1 -- 172.21.15.93:0/2546214827 wait complete. 2023-07-21T22:16:16.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:16 smithi093 ceph-mon[96579]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:16 smithi093 ceph-mon[96579]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:16.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:16 smithi093 ceph-mon[96579]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.652 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:16 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2379871846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:16.653 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:16:16.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 -- 172.21.15.93:0/1596027566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 msgr2=0x7f38e8106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/1596027566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e8106e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b0a0 tx=0x7f38e000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 -- 172.21.15.93:0/1596027566 shutdown_connections 2023-07-21T22:16:16.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/1596027566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e8106e20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/1596027566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/1596027566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 0x7f38e8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 -- 172.21.15.93:0/1596027566 >> 172.21.15.93:0/1596027566 conn(0x7f38e80fa000 msgr2=0x7f38e80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 -- 172.21.15.93:0/1596027566 shutdown_connections 2023-07-21T22:16:16.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.733+0000 7f38ed7a0700 1 -- 172.21.15.93:0/1596027566 wait complete. 2023-07-21T22:16:16.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.734+0000 7f38ed7a0700 1 Processor -- start 2023-07-21T22:16:16.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.734+0000 7f38ed7a0700 1 -- start start 2023-07-21T22:16:16.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.734+0000 7f38ed7a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e8194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.734+0000 7f38ed7a0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 0x7f38e8194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e819c410 con 0x7f38e8100560 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e819c550 con 0x7f38e80ffbd0 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e819c6e0 con 0x7f38e80fe9e0 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43836/0 (socket says 172.21.15.93:43836) 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 -- 172.21.15.93:0/3711654948 learned_addr learned my addr 172.21.15.93:0/3711654948 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e67fc700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 0x7f38e8194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e6ffd700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e8194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 -- 172.21.15.93:0/3711654948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 msgr2=0x7f38e8194890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e8194890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 -- 172.21.15.93:0/3711654948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 msgr2=0x7f38e8194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 0x7f38e8194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 -- 172.21.15.93:0/3711654948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0009d00 con 0x7f38e8100560 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e6ffd700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e8194890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38e77fe700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b830 tx=0x7f38e0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000e040 con 0x7f38e8100560 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e00094d0 con 0x7f38e8100560 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e819c960 con 0x7f38e8100560 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e001bd20 con 0x7f38e8100560 2023-07-21T22:16:16.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.735+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e819ce30 con 0x7f38e8100560 2023-07-21T22:16:16.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.737+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8105870 con 0x7f38e8100560 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.737+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f38e001b450 con 0x7f38e8100560 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.738+0000 7f38cffff700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 0x7f38d00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.738+0000 7f38e6ffd700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 0x7f38d00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.738+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f38e00ce9d0 con 0x7f38e8100560 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.738+0000 7f38e6ffd700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 0x7f38d00a3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ffa30 tx=0x7f38d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:16.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.742+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f38e0098080 con 0x7f38e8100560 2023-07-21T22:16:16.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:16 smithi194 ceph-mon[98288]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:16 smithi194 ceph-mon[98288]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:16.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:16 smithi194 ceph-mon[98288]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:16.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:16 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2379871846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:16.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.929+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38e802ca00 con 0x7f38e8100560 2023-07-21T22:16:16.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.930+0000 7f38cffff700 1 -- 172.21.15.93:0/3711654948 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38e009b6a0 con 0x7f38e8100560 2023-07-21T22:16:16.974 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:16:16.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 msgr2=0x7f38d00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 0x7f38d00a3710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ffa30 tx=0x7f38d8008040 comp rx=0 tx=0).stop 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 msgr2=0x7f38e819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b830 tx=0x7f38e0004d00 comp rx=0 tx=0).stop 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 shutdown_connections 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e8100560 0x7f38e819be40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38e80fe9e0 0x7f38e8194890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f38d00a1260 0x7f38d00a3710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 --2- 172.21.15.93:0/3711654948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e80ffbd0 0x7f38e8194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:16.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 >> 172.21.15.93:0/3711654948 conn(0x7f38e80fa000 msgr2=0x7f38e80fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:16.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 shutdown_connections 2023-07-21T22:16:16.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:16.932+0000 7f38ed7a0700 1 -- 172.21.15.93:0/3711654948 wait complete. 2023-07-21T22:16:16.988 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:16:17.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.070+0000 7f3ac08e4700 1 -- 172.21.15.93:0/952617572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc100d30 msgr2=0x7f3abc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.070+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/952617572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc100d30 0x7f3abc101180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009990 tx=0x7f3ab0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 -- 172.21.15.93:0/952617572 shutdown_connections 2023-07-21T22:16:17.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/952617572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc1016c0 0x7f3abc107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/952617572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc0ffb40 0x7f3abc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/952617572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc100d30 0x7f3abc101180 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 -- 172.21.15.93:0/952617572 >> 172.21.15.93:0/952617572 conn(0x7f3abc0fb160 msgr2=0x7f3abc0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:17.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 -- 172.21.15.93:0/952617572 shutdown_connections 2023-07-21T22:16:17.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.071+0000 7f3ac08e4700 1 -- 172.21.15.93:0/952617572 wait complete. 2023-07-21T22:16:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 Processor -- start 2023-07-21T22:16:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 -- start start 2023-07-21T22:16:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc0ffb40 0x7f3abc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 0x7f3abc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19d380 con 0x7f3abc1016c0 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19d4c0 con 0x7f3abc100d30 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ac08e4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19d650 con 0x7f3abc0ffb40 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43850/0 (socket says 172.21.15.93:43850) 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 -- 172.21.15.93:0/1281706454 learned_addr learned my addr 172.21.15.93:0/1281706454 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3aba59c700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc0ffb40 0x7f3abc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ab9d9b700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 0x7f3abc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 -- 172.21.15.93:0/1281706454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc0ffb40 msgr2=0x7f3abc195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc0ffb40 0x7f3abc195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 -- 172.21.15.93:0/1281706454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 msgr2=0x7f3abc195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 0x7f3abc195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 -- 172.21.15.93:0/1281706454 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab00095f0 con 0x7f3abc1016c0 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3ab9d9b700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 0x7f3abc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:17.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.072+0000 7f3abad9d700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007ee0 tx=0x7f3ab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.073+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400ddd0 con 0x7f3abc1016c0 2023-07-21T22:16:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.073+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc19d930 con 0x7f3abc1016c0 2023-07-21T22:16:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.073+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab400f040 con 0x7f3abc1016c0 2023-07-21T22:16:17.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.073+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40048e0 con 0x7f3abc1016c0 2023-07-21T22:16:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.073+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc19ddf0 con 0x7f3abc1016c0 2023-07-21T22:16:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.074+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc10b2a0 con 0x7f3abc1016c0 2023-07-21T22:16:17.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.075+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ab4003a00 con 0x7f3abc1016c0 2023-07-21T22:16:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.075+0000 7f3aab7fe700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 0x7f3aa40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.075+0000 7f3aba59c700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 0x7f3aa40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.075+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3ab40cc3c0 con 0x7f3abc1016c0 2023-07-21T22:16:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.076+0000 7f3aba59c700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 0x7f3aa40a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3abc100b90 tx=0x7f3aac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.079+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3ab404e770 con 0x7f3abc1016c0 2023-07-21T22:16:17.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.269+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3abc106a70 con 0x7f3abc1016c0 2023-07-21T22:16:17.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.270+0000 7f3aab7fe700 1 -- 172.21.15.93:0/1281706454 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ab40959e0 con 0x7f3abc1016c0 2023-07-21T22:16:17.314 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:16:17.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 msgr2=0x7f3aa40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 0x7f3aa40a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3abc100b90 tx=0x7f3aac00b410 comp rx=0 tx=0).stop 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 msgr2=0x7f3abc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4007ee0 tx=0x7f3ab400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 shutdown_connections 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3abc1016c0 0x7f3abc19cdb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3abc0ffb40 0x7f3abc195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3aa40a11e0 0x7f3aa40a3690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 --2- 172.21.15.93:0/1281706454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3abc100d30 0x7f3abc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 >> 172.21.15.93:0/1281706454 conn(0x7f3abc0fb160 msgr2=0x7f3abc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:17.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 shutdown_connections 2023-07-21T22:16:17.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.272+0000 7f3ac08e4700 1 -- 172.21.15.93:0/1281706454 wait complete. 2023-07-21T22:16:17.328 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:16:17.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 -- 172.21.15.93:0/4014497890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00ff5e0 msgr2=0x7f31f0106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 --2- 172.21.15.93:0/4014497890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00ff5e0 0x7f31f0106e90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f31e800b0a0 tx=0x7f31e800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 -- 172.21.15.93:0/4014497890 shutdown_connections 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 --2- 172.21.15.93:0/4014497890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00ff5e0 0x7f31f0106e90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 --2- 172.21.15.93:0/4014497890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00fffe0 0x7f31f01003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 --2- 172.21.15.93:0/4014497890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fec30 0x7f31f00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 -- 172.21.15.93:0/4014497890 >> 172.21.15.93:0/4014497890 conn(0x7f31f00f9770 msgr2=0x7f31f00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 -- 172.21.15.93:0/4014497890 shutdown_connections 2023-07-21T22:16:17.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.407+0000 7f31f73f3700 1 -- 172.21.15.93:0/4014497890 wait complete. 2023-07-21T22:16:17.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 Processor -- start 2023-07-21T22:16:17.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 -- start start 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00ff5e0 0x7f31f0197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 0x7f31f019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019f630 con 0x7f31f00fec30 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019f770 con 0x7f31f00fffe0 2023-07-21T22:16:17.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f73f3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f019f900 con 0x7f31f00ff5e0 2023-07-21T22:16:17.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f518f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f518f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43860/0 (socket says 172.21.15.93:43860) 2023-07-21T22:16:17.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f518f700 1 -- 172.21.15.93:0/3344635330 learned_addr learned my addr 172.21.15.93:0/3344635330 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:17.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f5990700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 0x7f31f019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.408+0000 7f31f498e700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00ff5e0 0x7f31f0197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 -- 172.21.15.93:0/3344635330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00ff5e0 msgr2=0x7f31f0197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00ff5e0 0x7f31f0197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 -- 172.21.15.93:0/3344635330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 msgr2=0x7f31f019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 0x7f31f019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 -- 172.21.15.93:0/3344635330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8009d00 con 0x7f31f00fec30 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f5990700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 0x7f31f019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f518f700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f31e000b500 tx=0x7f31e000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0011830 con 0x7f31f00fec30 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31e0011e50 con 0x7f31f00fec30 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000f2e0 con 0x7f31f00fec30 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f019fbe0 con 0x7f31f00fec30 2023-07-21T22:16:17.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.409+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f01a00f0 con 0x7f31f00fec30 2023-07-21T22:16:17.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.410+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f010a240 con 0x7f31f00fec30 2023-07-21T22:16:17.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.411+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f31e0010580 con 0x7f31f00fec30 2023-07-21T22:16:17.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.412+0000 7f31e67fc700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 0x7f31dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.412+0000 7f31f498e700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 0x7f31dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.412+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31e00cc3d0 con 0x7f31f00fec30 2023-07-21T22:16:17.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.412+0000 7f31f498e700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 0x7f31dc0a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005ea0 tx=0x7f31ec005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.416+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f31e0095b00 con 0x7f31f00fec30 2023-07-21T22:16:17.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:17 smithi134 ceph-mon[97873]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:17.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:17 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2546214827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:17 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3711654948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:17 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1281706454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.646+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31f01a0700 con 0x7f31f00fec30 2023-07-21T22:16:17.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.646+0000 7f31e67fc700 1 -- 172.21.15.93:0/3344635330 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f31e0099120 con 0x7f31f00fec30 2023-07-21T22:16:17.690 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:17.691 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:16:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:17 smithi093 ceph-mon[96579]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:17 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2546214827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:17 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3711654948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:17 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1281706454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 msgr2=0x7f31dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 0x7f31dc0a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f31ec005ea0 tx=0x7f31ec005e30 comp rx=0 tx=0).stop 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 msgr2=0x7f31f0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f31e000b500 tx=0x7f31e000b8c0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 shutdown_connections 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31dc0a12a0 0x7f31dc0a3750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31f00fffe0 0x7f31f019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31f00fec30 0x7f31f0197ab0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 --2- 172.21.15.93:0/3344635330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31f00ff5e0 0x7f31f0197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 >> 172.21.15.93:0/3344635330 conn(0x7f31f00f9770 msgr2=0x7f31f00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:17.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 shutdown_connections 2023-07-21T22:16:17.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.649+0000 7f31f73f3700 1 -- 172.21.15.93:0/3344635330 wait complete. 2023-07-21T22:16:17.694 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:17.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:16:17.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 -- 172.21.15.93:0/204123912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c100560 msgr2=0x7f318c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 --2- 172.21.15.93:0/204123912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c100560 0x7f318c106e20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f318400b0a0 tx=0x7f318400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 -- 172.21.15.93:0/204123912 shutdown_connections 2023-07-21T22:16:17.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 --2- 172.21.15.93:0/204123912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c100560 0x7f318c106e20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 --2- 172.21.15.93:0/204123912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0fe9e0 0x7f318c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 --2- 172.21.15.93:0/204123912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c0ffbd0 0x7f318c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 -- 172.21.15.93:0/204123912 >> 172.21.15.93:0/204123912 conn(0x7f318c0fa000 msgr2=0x7f318c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 -- 172.21.15.93:0/204123912 shutdown_connections 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 -- 172.21.15.93:0/204123912 wait complete. 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.782+0000 7f319142a700 1 Processor -- start 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 -- start start 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0ffbd0 0x7f318c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 0x7f318c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c19d3b0 con 0x7f318c0fe9e0 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c19d4f0 con 0x7f318c0ffbd0 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f319142a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c19d680 con 0x7f318c100560 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43874/0 (socket says 172.21.15.93:43874) 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 -- 172.21.15.93:0/1157204397 learned_addr learned my addr 172.21.15.93:0/1157204397 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:17.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318a7fc700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0ffbd0 0x7f318c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318b7fe700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 0x7f318c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 -- 172.21.15.93:0/1157204397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 msgr2=0x7f318c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 0x7f318c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 -- 172.21.15.93:0/1157204397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0ffbd0 msgr2=0x7f318c195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0ffbd0 0x7f318c195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 -- 172.21.15.93:0/1157204397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184009d00 con 0x7f318c0fe9e0 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318b7fe700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 0x7f318c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:17.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f318affd700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c990 tx=0x7f317c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.783+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c007b00 con 0x7f318c0fe9e0 2023-07-21T22:16:17.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.784+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f317c007c60 con 0x7f318c0fe9e0 2023-07-21T22:16:17.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.784+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c19d960 con 0x7f318c0fe9e0 2023-07-21T22:16:17.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.784+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c018660 con 0x7f318c0fe9e0 2023-07-21T22:16:17.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.784+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c19de70 con 0x7f318c0fe9e0 2023-07-21T22:16:17.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.784+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c10a1c0 con 0x7f318c0fe9e0 2023-07-21T22:16:17.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.786+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f317c004020 con 0x7f318c0fe9e0 2023-07-21T22:16:17.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.787+0000 7f3173fff700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 0x7f31740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:17.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.787+0000 7f318a7fc700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 0x7f31740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:17.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.787+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f317c0ccc30 con 0x7f318c0fe9e0 2023-07-21T22:16:17.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.787+0000 7f318a7fc700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 0x7f31740a36d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3180005950 tx=0x7f31800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:17.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.790+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f317c096280 con 0x7f318c0fe9e0 2023-07-21T22:16:17.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:17 smithi194 ceph-mon[98288]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:16:17.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:17 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2546214827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:17 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3711654948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:16:17.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:17 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1281706454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:16:18.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.979+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f318c10a3e0 con 0x7f318c0fe9e0 2023-07-21T22:16:18.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.980+0000 7f3173fff700 1 -- 172.21.15.93:0/1157204397 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22415 (secure 0 0 0) 0x7f317c0998a0 con 0x7f318c0fe9e0 2023-07-21T22:16:18.024 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:18.025 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:15:53.025400+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:16:18.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.982+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 msgr2=0x7f31740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.982+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 0x7f31740a36d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3180005950 tx=0x7f31800058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.982+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 msgr2=0x7f318c195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.982+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c990 tx=0x7f317c00cd50 comp rx=0 tx=0).stop 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 shutdown_connections 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f318c100560 0x7f318c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f318c0fe9e0 0x7f318c195830 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f31740a1220 0x7f31740a36d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 --2- 172.21.15.93:0/1157204397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f318c0ffbd0 0x7f318c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 >> 172.21.15.93:0/1157204397 conn(0x7f318c0fa000 msgr2=0x7f318c0689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 shutdown_connections 2023-07-21T22:16:18.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:17.983+0000 7f319142a700 1 -- 172.21.15.93:0/1157204397 wait complete. 2023-07-21T22:16:18.039 DEBUG:tasks.fwd_scrub:Yielding 2023-07-21T22:16:18.040 INFO:teuthology.run_tasks:Running task exec... 2023-07-21T22:16:18.051 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T22:16:18.051 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:16:18.052 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-21T22:16:18.052 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-21T22:16:18.053 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:16:18.053 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:16:18.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 -- 172.21.15.93:0/1674777462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580fea00 msgr2=0x7fc7580fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 --2- 172.21.15.93:0/1674777462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580fea00 0x7fc7580fee10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc754009990 tx=0x7fc754009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 -- 172.21.15.93:0/1674777462 shutdown_connections 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 --2- 172.21.15.93:0/1674777462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc758100640 0x7fc758106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 --2- 172.21.15.93:0/1674777462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580fea00 0x7fc7580fee10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 --2- 172.21.15.93:0/1674777462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7580ffc90 0x7fc758100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 -- 172.21.15.93:0/1674777462 >> 172.21.15.93:0/1674777462 conn(0x7fc7580fa000 msgr2=0x7fc7580fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 -- 172.21.15.93:0/1674777462 shutdown_connections 2023-07-21T22:16:18.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.134+0000 7fc7604ca700 1 -- 172.21.15.93:0/1674777462 wait complete. 2023-07-21T22:16:18.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 Processor -- start 2023-07-21T22:16:18.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 -- start start 2023-07-21T22:16:18.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580fea00 0x7fc7581985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100640 0x7fc75819d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819d640 con 0x7fc7580ffc90 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819d780 con 0x7fc758100640 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc7604ca700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819d940 con 0x7fc7580fea00 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc75da65700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc75da65700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43896/0 (socket says 172.21.15.93:43896) 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc75da65700 1 -- 172.21.15.93:0/3103994736 learned_addr learned my addr 172.21.15.93:0/3103994736 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc75ea67700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100640 0x7fc75819d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.135+0000 7fc75e266700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580fea00 0x7fc7581985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 -- 172.21.15.93:0/3103994736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580fea00 msgr2=0x7fc7581985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580fea00 0x7fc7581985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 -- 172.21.15.93:0/3103994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100640 msgr2=0x7fc75819d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100640 0x7fc75819d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 -- 172.21.15.93:0/3103994736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540095f0 con 0x7fc7580ffc90 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc75da65700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc74800b530 tx=0x7fc74800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011830 con 0x7fc7580ffc90 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc748011e50 con 0x7fc7580ffc90 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75819dc20 con 0x7fc7580ffc90 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800f2e0 con 0x7fc7580ffc90 2023-07-21T22:16:18.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75819e180 con 0x7fc7580ffc90 2023-07-21T22:16:18.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 -- 172.21.15.93:0/322518265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840fe9e0 msgr2=0x7faa840fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 --2- 172.21.15.93:0/322518265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840fe9e0 0x7faa840fedf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faa80009990 tx=0x7faa80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 -- 172.21.15.93:0/322518265 shutdown_connections 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 --2- 172.21.15.93:0/322518265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa84106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 --2- 172.21.15.93:0/322518265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840fe9e0 0x7faa840fedf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 --2- 172.21.15.93:0/322518265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840ffbd0 0x7faa84100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.136+0000 7faa8c105700 1 -- 172.21.15.93:0/322518265 >> 172.21.15.93:0/322518265 conn(0x7faa840fa000 msgr2=0x7faa840fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- 172.21.15.93:0/322518265 shutdown_connections 2023-07-21T22:16:18.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- 172.21.15.93:0/322518265 wait complete. 2023-07-21T22:16:18.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 Processor -- start 2023-07-21T22:16:18.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- start start 2023-07-21T22:16:18.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75804e2f0 con 0x7fc7580ffc90 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 0x7faa84068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa8419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419f5d0 con 0x7faa840ffbd0 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419f740 con 0x7faa84100560 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.137+0000 7faa8c105700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8419f900 con 0x7faa840fe9e0 2023-07-21T22:16:18.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43902/0 (socket says 172.21.15.93:43902) 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 -- 172.21.15.93:0/1682559434 learned_addr learned my addr 172.21.15.93:0/1682559434 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa8419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 0x7faa84068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 -- 172.21.15.93:0/1682559434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 msgr2=0x7faa84068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 0x7faa84068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 -- 172.21.15.93:0/1682559434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 msgr2=0x7faa8419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa8419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 -- 172.21.15.93:0/1682559434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800095f0 con 0x7faa840ffbd0 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa8419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 0x7faa84068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa896a0700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faa7400d980 tx=0x7faa7400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa7affd700 1 -- 172.21.15.93:0/1682559434 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400bb60 con 0x7faa840ffbd0 2023-07-21T22:16:18.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8419fbe0 con 0x7faa840ffbd0 2023-07-21T22:16:18.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa7affd700 1 -- 172.21.15.93:0/1682559434 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa74007640 con 0x7faa840ffbd0 2023-07-21T22:16:18.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa7affd700 1 -- 172.21.15.93:0/1682559434 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74004e90 con 0x7faa840ffbd0 2023-07-21T22:16:18.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa841a00f0 con 0x7faa840ffbd0 2023-07-21T22:16:18.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.138+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc748010380 con 0x7fc7580ffc90 2023-07-21T22:16:18.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.139+0000 7fc74f7fe700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 0x7fc7440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.139+0000 7fc75e266700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 0x7fc7440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.139+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc7480cc410 con 0x7fc7580ffc90 2023-07-21T22:16:18.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.139+0000 7fc75e266700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 0x7fc7440a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc75400b5c0 tx=0x7fc754005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.141+0000 7faa7affd700 1 -- 172.21.15.93:0/1682559434 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faa7400bcc0 con 0x7faa840ffbd0 2023-07-21T22:16:18.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.141+0000 7faa7affd700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 0x7faa700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.141+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 0x7faa700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 0x7faa700a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa8000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 Processor -- start 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 -- start start 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f322804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32280a90 0x558f32280f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32185d30 con 0x558f322818b0 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f321d8ac0 con 0x558f32282220 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa9828ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f321d1f50 con 0x558f32280a90 2023-07-21T22:16:18.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43910/0 (socket says 172.21.15.93:43910) 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.142+0000 7faa89ea1700 1 -- 172.21.15.93:0/1226483406 learned_addr learned my addr 172.21.15.93:0/1226483406 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa896a0700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f322804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32280a90 0x558f32280f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 -- 172.21.15.93:0/1226483406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32280a90 msgr2=0x558f32280f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32280a90 0x558f32280f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 -- 172.21.15.93:0/1226483406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 msgr2=0x558f322804b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f322804b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 -- 172.21.15.93:0/1226483406 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f31fffe60 con 0x558f322818b0 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa896a0700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f322804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.143+0000 7faa89ea1700 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faa80005480 tx=0x7faa8001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bba39eb6f78f0ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.144+0000 7faa6ffff700 1 -- 172.21.15.93:0/1226483406 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8002b070 con 0x558f322818b0 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.144+0000 7faa7affd700 1 -- 172.21.15.93:0/1682559434 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa74004750 con 0x7faa840ffbd0 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.144+0000 7faa6ffff700 1 -- 172.21.15.93:0/1226483406 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa8001e040 con 0x558f322818b0 2023-07-21T22:16:18.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.144+0000 7faa6ffff700 1 -- 172.21.15.93:0/1226483406 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80004d80 con 0x558f322818b0 2023-07-21T22:16:18.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.144+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc748095b40 con 0x7fc7580ffc90 2023-07-21T22:16:18.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- 172.21.15.93:0/1226483406 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 msgr2=0x558f32281cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faa80005480 tx=0x7faa8001f000 comp rx=0 tx=0).stop 2023-07-21T22:16:18.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- 172.21.15.93:0/1226483406 shutdown_connections 2023-07-21T22:16:18.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32280a90 0x558f32280f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f322818b0 0x558f32281cc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 --2- 172.21.15.93:0/1226483406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f322804b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- 172.21.15.93:0/1226483406 >> 172.21.15.93:0/1226483406 conn(0x558f32275f70 msgr2=0x558f3227b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- 172.21.15.93:0/1226483406 shutdown_connections 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- 172.21.15.93:0/1226483406 wait complete. 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 -- start start 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.145+0000 7faa9828ab80 1 Processor -- start 2023-07-21T22:16:18.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f3228a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32171b40 con 0x558f32282220 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32171840 con 0x558f32280a90 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f320559a0 con 0x558f322818b0 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 10 client.0 _ll_register_callbacks cb 0x558f3216f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:18.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43920/0 (socket says 172.21.15.93:43920) 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 -- 172.21.15.93:0/2613739447 learned_addr learned my addr 172.21.15.93:0/2613739447 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa89ea1700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa896a0700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f3228a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 -- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 msgr2=0x558f3228a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f3228a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 -- 172.21.15.93:0/2613739447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32280a90 msgr2=0x558f3228a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 -- 172.21.15.93:0/2613739447 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa84107750 con 0x558f32282220 2023-07-21T22:16:18.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa896a0700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f3228a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00aaf0 tx=0x7faa7c00abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356654b2cd1bff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:18.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0079c0 con 0x558f32282220 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa7c006e40 con 0x558f32282220 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.146+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f3212b570 con 0x558f32282220 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa7c007b20 con 0x558f32282220 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 10 client.15411 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558f31fe8ce0 con 0x558f32282220 2023-07-21T22:16:18.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f3212b570 con 0x558f32282220 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faa7c0143a0 con 0x558f32282220 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 10 client.15411 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 20 client.15411 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f32295f70 0x558f32298380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 4 client.15411 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:18.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f321caf30 con 0x558f32295f70 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.147+0000 7faa89ea1700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f32295f70 0x558f32298380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa6d7fa700 10 client.15411 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa89ea1700 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f32295f70 0x558f32298380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68dfea31e592fc46 server_cookie=6930a9334ea7b5e2 in_seq=0 out_seq=0 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa6d7fa700 1 -- 172.21.15.93:0/2613739447 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f321caf30 con 0x558f32295f70 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa6d7fa700 10 client.15411 handle_command_reply: tid=0 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa9828ab80 1 client.15411 shutdown 2023-07-21T22:16:18.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 msgr2=0x558f32291610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.148+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00aaf0 tx=0x7faa7c00abd0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 shutdown_connections 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f32295f70 0x558f32298380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32282220 0x558f32291610 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 --2- 172.21.15.93:0/2613739447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f3228a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa6d7fa700 0 client.15411 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 >> 172.21.15.93:0/2613739447 conn(0x558f32275f70 msgr2=0x558f3227aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 shutdown_connections 2023-07-21T22:16:18.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 1 -- 172.21.15.93:0/2613739447 wait complete. 2023-07-21T22:16:18.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.149+0000 7faa9828ab80 20 client.15411 trim_cache size 0 max 16384 2023-07-21T22:16:18.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 Processor -- start 2023-07-21T22:16:18.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 -- start start 2023-07-21T22:16:18.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f32291530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f3228a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f321caf30 con 0x558f32280a90 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3212b570 con 0x558f32282220 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa9828ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32289540 con 0x558f322818b0 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43924/0 (socket says 172.21.15.93:43924) 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 -- 172.21.15.93:0/3018993641 learned_addr learned my addr 172.21.15.93:0/3018993641 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f3228a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 -- 172.21.15.93:0/3018993641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 msgr2=0x558f32291530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f32291530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 -- 172.21.15.93:0/3018993641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 msgr2=0x558f3228a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f3228a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa89ea1700 1 -- 172.21.15.93:0/3018993641 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f320262d0 con 0x558f32280a90 2023-07-21T22:16:18.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.166+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f3228a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa89ea1700 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faa80005f50 tx=0x7faa80009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9146fc4d537426 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa78ff9700 1 -- 172.21.15.93:0/3018993641 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8002b040 con 0x558f32280a90 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa78ff9700 1 -- 172.21.15.93:0/3018993641 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa80038e30 con 0x558f32280a90 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa78ff9700 1 -- 172.21.15.93:0/3018993641 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80004a00 con 0x558f32280a90 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 -- 172.21.15.93:0/3018993641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 msgr2=0x558f3228a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faa80005f50 tx=0x7faa80009310 comp rx=0 tx=0).stop 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 -- 172.21.15.93:0/3018993641 shutdown_connections 2023-07-21T22:16:18.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f32282220 0x558f3228a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228a090 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 --2- 172.21.15.93:0/3018993641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f322818b0 0x558f32291530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 -- 172.21.15.93:0/3018993641 >> 172.21.15.93:0/3018993641 conn(0x558f32275f70 msgr2=0x558f32284b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 -- 172.21.15.93:0/3018993641 shutdown_connections 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.167+0000 7faa9828ab80 1 -- 172.21.15.93:0/3018993641 wait complete. 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 -- start start 2023-07-21T22:16:18.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 Processor -- start 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f322818b0 0x558f3228f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32282220 0x558f322902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32290800 con 0x558f32280a90 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32290970 con 0x558f322818b0 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32290b30 con 0x558f32282220 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa9828ab80 10 client.0 _ll_register_callbacks cb 0x7faa7c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:18.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa89ea1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43930/0 (socket says 172.21.15.93:43930) 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa89ea1700 1 -- 172.21.15.93:0/4006145343 learned_addr learned my addr 172.21.15.93:0/4006145343 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.168+0000 7faa896a0700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f322818b0 0x558f3228f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa8a6a2700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32282220 0x558f322902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 -- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32282220 msgr2=0x558f322902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32282220 0x558f322902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 -- 172.21.15.93:0/4006145343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f322818b0 msgr2=0x558f3228f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f322818b0 0x558f3228f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 -- 172.21.15.93:0/4006145343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f31fffe60 con 0x558f32280a90 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa89ea1700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa80031040 tx=0x7faa800385c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2450878bb6a8f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8002b040 con 0x558f32280a90 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa8001e040 con 0x558f32280a90 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7faa8000b970 con 0x558f32280a90 2023-07-21T22:16:18.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa8000bb30 con 0x558f32280a90 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa9828ab80 10 client.15417 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558f31ffe330 con 0x558f32280a90 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.169+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa80051020 con 0x558f32280a90 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faa80003bc0 con 0x558f32280a90 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 10 client.15417 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 20 client.15417 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:18.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f322ec210 0x558f322ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 4 client.15417 mds_command: new command op to 24484 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558f3218d0d0 con 0x558f322ec210 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa896a0700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f322ec210 0x558f322ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa6d7fa700 10 client.15417 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.170+0000 7faa896a0700 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f322ec210 0x558f322ee620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e164dea7c51bcc40 server_cookie=b0a950dab553f10f in_seq=0 out_seq=0 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa6d7fa700 1 -- 172.21.15.93:0/4006145343 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f3218d0d0 con 0x558f322ec210 2023-07-21T22:16:18.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa6d7fa700 10 client.15417 handle_command_reply: tid=0 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 client.15417 shutdown 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 msgr2=0x558f3228ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa80031040 tx=0x7faa800385c0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 shutdown_connections 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558f322ec210 0x558f322ee620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f32282220 0x558f322902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f32280a90 0x558f3228ed70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 --2- 172.21.15.93:0/4006145343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558f322818b0 0x558f3228f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 >> 172.21.15.93:0/4006145343 conn(0x558f32275f70 msgr2=0x558f32284b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa6d7fa700 0 client.15417 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 shutdown_connections 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 1 -- 172.21.15.93:0/4006145343 wait complete. 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa9828ab80 20 client.15417 trim_cache size 0 max 16384 2023-07-21T22:16:18.227 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:18.228 INFO:teuthology.orchestra.run.smithi093.stdout: "return_code": 0, 2023-07-21T22:16:18.228 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_tag": "4916a248-c305-4506-8e54-79475ac18fa8", 2023-07-21T22:16:18.228 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": "asynchronous" 2023-07-21T22:16:18.228 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:18.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 msgr2=0x7faa700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.172+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 0x7faa700a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa8000b540 comp rx=0 tx=0).stop 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 msgr2=0x7faa8419c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faa7400d980 tx=0x7faa7400dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 shutdown_connections 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7faa700a11e0 0x7faa700a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa84100560 0x7faa8419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa840fe9e0 0x7faa84068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 --2- 172.21.15.93:0/1682559434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa840ffbd0 0x7faa8419c9a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 >> 172.21.15.93:0/1682559434 conn(0x7faa840fa000 msgr2=0x7faa840fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 shutdown_connections 2023-07-21T22:16:18.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.173+0000 7faa8c105700 1 -- 172.21.15.93:0/1682559434 wait complete. 2023-07-21T22:16:18.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:16:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3777112200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 msgr2=0x7fb7c80fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3777112200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c80fedf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009990 tx=0x7fb7bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3777112200 shutdown_connections 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3777112200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3777112200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c80fedf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3777112200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 0x7fb7c8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3777112200 >> 172.21.15.93:0/3777112200 conn(0x7fb7c80fa000 msgr2=0x7fb7c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3777112200 shutdown_connections 2023-07-21T22:16:18.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.316+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3777112200 wait complete. 2023-07-21T22:16:18.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 Processor -- start 2023-07-21T22:16:18.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 -- start start 2023-07-21T22:16:18.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 0x7fb7c8194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c819c2d0 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c819c410 con 0x7fb7c8100560 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cedb2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c819c5a0 con 0x7fb7c80ffbd0 2023-07-21T22:16:18.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7ccb4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7ccb4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43946/0 (socket says 172.21.15.93:43946) 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/3998890340 learned_addr learned my addr 172.21.15.93:0/3998890340 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.317+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 0x7fb7c8194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/3998890340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 msgr2=0x7fb7c8194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 0x7fb7c8194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/3998890340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 msgr2=0x7fb7c819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/3998890340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7bc0095f0 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7c5ffb700 1 -- 172.21.15.93:0/3998890340 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc01d070 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7c5ffb700 1 -- 172.21.15.93:0/3998890340 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7bc004180 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c819c820 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7c5ffb700 1 -- 172.21.15.93:0/3998890340 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc022e00 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.318+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c819cd20 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.321+0000 7fb7c5ffb700 1 -- 172.21.15.93:0/3998890340 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb7bc022500 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.367 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:18 smithi093 ceph-mon[96579]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:18.368 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:18 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3344635330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:18.368 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:18 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1157204397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:18.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.322+0000 7fb7c5ffb700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 0x7fb7b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.322+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 0x7fb7b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.322+0000 7fb7c5ffb700 1 -- 172.21.15.93:0/3998890340 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb7bc0cdc90 con 0x7fb7c80fe9e0 2023-07-21T22:16:18.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.322+0000 7fb7daf37b80 1 Processor -- start 2023-07-21T22:16:18.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.322+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 0x7fb7b80a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4005950 tx=0x7fb7b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 -- start start 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d90b3cb0 con 0x5611d91afbe0 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d9106a40 con 0x5611d91ae670 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d90ffed0 con 0x5611d91aefc0 2023-07-21T22:16:18.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43958/0 (socket says 172.21.15.93:43958) 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2705084284 learned_addr learned my addr 172.21.15.93:0/2705084284 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2705084284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 msgr2=0x5611d91af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2705084284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 msgr2=0x5611d91aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2705084284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d8f2dde0 con 0x5611d91afbe0 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.325+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4bc45c336421b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7b2ffd700 1 -- 172.21.15.93:0/2705084284 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0014610 con 0x5611d91afbe0 2023-07-21T22:16:18.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7b2ffd700 1 -- 172.21.15.93:0/2705084284 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c000f050 con 0x5611d91afbe0 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7b2ffd700 1 -- 172.21.15.93:0/2705084284 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0013ab0 con 0x5611d91afbe0 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2705084284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 msgr2=0x5611d91b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c00077e0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2705084284 shutdown_connections 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91b0050 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2705084284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2705084284 >> 172.21.15.93:0/2705084284 conn(0x5611d91a3ef0 msgr2=0x5611d91a91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2705084284 shutdown_connections 2023-07-21T22:16:18.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.326+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2705084284 wait complete. 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 -- start start 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 Processor -- start 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91afbe0 0x5611d91bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d909fac0 con 0x5611d91ae670 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d909f7c0 con 0x5611d91afbe0 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d8f83920 con 0x5611d91aefc0 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7daf37b80 10 client.0 _ll_register_callbacks cb 0x5611d909d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7ccb4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7ccb4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43960/0 (socket says 172.21.15.93:43960) 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/1524122402 learned_addr learned my addr 172.21.15.93:0/1524122402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.327+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91afbe0 0x5611d91bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 msgr2=0x5611d91b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/1524122402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91afbe0 msgr2=0x5611d91bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91afbe0 0x5611d91bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 -- 172.21.15.93:0/1524122402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c0009a90 con 0x5611d91ae670 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:18.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc097a70 tx=0x7fb7bc005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7089dbd21357bc62 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b2070 con 0x5611d91ae670 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb7bc09f030 con 0x5611d91ae670 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611d90594f0 con 0x5611d91ae670 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7bc0a33c0 con 0x5611d91ae670 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 10 client.15429 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:18.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5611d8f16c60 con 0x5611d91ae670 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7bc0c9080 con 0x5611d91ae670 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7bc096df0 con 0x5611d91ae670 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 20 client.15429 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d91c3940 0x5611d91c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 4 client.15429 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.328+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611d90f8eb0 con 0x5611d91c3940 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.329+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d91c3940 0x5611d91c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.329+0000 7fb7a7fff700 10 client.15429 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.329+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d91c3940 0x5611d91c5d50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34414b298456af65 server_cookie=66507e28c3d772f in_seq=0 out_seq=0 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7a7fff700 1 -- 172.21.15.93:0/1524122402 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5611d90f8eb0 con 0x5611d91c3940 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7a7fff700 10 client.15429 handle_command_reply: tid=0 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 client.15429 shutdown 2023-07-21T22:16:18.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 msgr2=0x5611d91b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc097a70 tx=0x7fb7bc005ad0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 shutdown_connections 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d91c3940 0x5611d91c5d50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91afbe0 0x5611d91bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91ae670 0x5611d91b80b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/1524122402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91aefc0 0x5611d91b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 >> 172.21.15.93:0/1524122402 conn(0x5611d91a3ef0 msgr2=0x5611d91a8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7a7fff700 0 client.15429 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 shutdown_connections 2023-07-21T22:16:18.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 1 -- 172.21.15.93:0/1524122402 wait complete. 2023-07-21T22:16:18.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.330+0000 7fb7daf37b80 20 client.15429 trim_cache size 0 max 16384 2023-07-21T22:16:18.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 Processor -- start 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 -- start start 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 0x5611d91b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91aefc0 0x5611d91bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d91b7560 con 0x5611d91afbe0 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d91aa5e0 con 0x5611d91aefc0 2023-07-21T22:16:18.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d91af390 con 0x5611d91ae670 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43972/0 (socket says 172.21.15.93:43972) 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2398024798 learned_addr learned my addr 172.21.15.93:0/2398024798 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 0x5611d91b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91aefc0 0x5611d91bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2398024798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 msgr2=0x5611d91b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 0x5611d91b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2398024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91aefc0 msgr2=0x5611d91bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91aefc0 0x5611d91bf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 -- 172.21.15.93:0/2398024798 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611d8f54250 con 0x5611d91afbe0 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 0x5611d91b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.360+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580f77afdeb909cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7b37fe700 1 -- 172.21.15.93:0/2398024798 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0023070 con 0x5611d91afbe0 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7b37fe700 1 -- 172.21.15.93:0/2398024798 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c00133e0 con 0x5611d91afbe0 2023-07-21T22:16:18.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7b37fe700 1 -- 172.21.15.93:0/2398024798 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0005040 con 0x5611d91afbe0 2023-07-21T22:16:18.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2398024798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 msgr2=0x5611d91bff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0006000 comp rx=0 tx=0).stop 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2398024798 shutdown_connections 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91afbe0 0x5611d91bff20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91ae670 0x5611d91b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/2398024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91aefc0 0x5611d91bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2398024798 >> 172.21.15.93:0/2398024798 conn(0x5611d91a3ef0 msgr2=0x5611d91ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2398024798 shutdown_connections 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.361+0000 7fb7daf37b80 1 -- 172.21.15.93:0/2398024798 wait complete. 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 -- start start 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 Processor -- start 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 0x5611d91bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d91bccd0 con 0x5611d91aefc0 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d92177f0 con 0x5611d91ae670 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611d92179b0 con 0x5611d91afbe0 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7daf37b80 10 client.0 _ll_register_callbacks cb 0x7fb7bc09aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43976/0 (socket says 172.21.15.93:43976) 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7ccb4e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7c7fff700 1 -- 172.21.15.93:0/3368148254 learned_addr learned my addr 172.21.15.93:0/3368148254 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:18.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.362+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 0x5611d91bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 -- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 msgr2=0x5611d91bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 0x5611d91bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 -- 172.21.15.93:0/3368148254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 msgr2=0x5611d91bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 -- 172.21.15.93:0/3368148254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611d8f2dde0 con 0x5611d91aefc0 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7cd34f700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 0x5611d91bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:18.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7c7fff700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5611d91c6630 tx=0x7fb7b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf2102bc87680 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40264e0 con 0x5611d91aefc0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb7b40208a0 con 0x5611d91aefc0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c000f050 con 0x5611d91aefc0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b4020a60 con 0x5611d91aefc0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7daf37b80 10 client.15435 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5611d8f2c2b0 con 0x5611d91aefc0 2023-07-21T22:16:18.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c000f050 con 0x5611d91aefc0 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.363+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7b402f470 con 0x5611d91aefc0 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 20 client.15435 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d9219390 0x5611d921b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 4 client.15435 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611d90bb050 con 0x5611d9219390 2023-07-21T22:16:18.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d9219390 0x5611d921b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7a7fff700 10 client.15435 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7ccb4e700 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d9219390 0x5611d921b7a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b8cfb039b56d95 server_cookie=d50a26624696eafd in_seq=0 out_seq=0 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7a7fff700 1 -- 172.21.15.93:0/3368148254 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5611d90bb050 con 0x5611d9219390 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7a7fff700 10 client.15435 handle_command_reply: tid=0 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.364+0000 7fb7daf37b80 1 client.15435 shutdown 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 msgr2=0x5611d91bc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5611d91c6630 tx=0x7fb7b40047c0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 shutdown_connections 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5611d9219390 0x5611d921b7a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5611d91afbe0 0x5611d91bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7a7fff700 0 client.15435 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611d91ae670 0x5611d91bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 --2- 172.21.15.93:0/3368148254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5611d91aefc0 0x5611d91bc080 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 >> 172.21.15.93:0/3368148254 conn(0x5611d91a3ef0 msgr2=0x5611d91bff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 shutdown_connections 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 1 -- 172.21.15.93:0/3368148254 wait complete. 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7daf37b80 20 client.15435 trim_cache size 0 max 16384 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:16:18.419 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": { 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: "4916a248-c305-4506-8e54-79475ac18fa8": { 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/", 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: "tag": "4916a248-c305-4506-8e54-79475ac18fa8", 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: "options": "recursive,force" 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:16:18.420 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:18.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 msgr2=0x7fb7b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 0x7fb7b80a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4005950 tx=0x7fb7b40058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 msgr2=0x7fb7c8194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.365+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc009f90 comp rx=0 tx=0).stop 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 shutdown_connections 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fb7b80a1260 0x7fb7b80a3710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c8100560 0x7fb7c819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9e0 0x7fb7c8194750 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 --2- 172.21.15.93:0/3998890340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ffbd0 0x7fb7c8194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:18.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 >> 172.21.15.93:0/3998890340 conn(0x7fb7c80fa000 msgr2=0x7fb7c80fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:18.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 shutdown_connections 2023-07-21T22:16:18.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.366+0000 7fb7cedb2700 1 -- 172.21.15.93:0/3998890340 wait complete. 2023-07-21T22:16:18.423 INFO:tasks.cephfs.filesystem:scrub status for tag:4916a248-c305-4506-8e54-79475ac18fa8 - {'path': '/', 'tag': '4916a248-c305-4506-8e54-79475ac18fa8', 'options': 'recursive,force'} 2023-07-21T22:16:18.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:18.424+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc7581059a0 con 0x7fc7580ffc90 2023-07-21T22:16:18.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:18 smithi134 ceph-mon[97873]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:18.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:18 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3344635330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:18.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:18 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1157204397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:18.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:18 smithi194 ceph-mon[98288]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-21T22:16:18.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:18 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3344635330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:18.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:18 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1157204397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:16:19.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.341+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7fc748010660 con 0x7fc7580ffc90 2023-07-21T22:16:19.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.342+0000 7fc74f7fe700 1 -- 172.21.15.93:0/3103994736 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc74804c6c0 con 0x7fc7580ffc90 2023-07-21T22:16:19.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 msgr2=0x7fc7440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 0x7fc7440a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc75400b5c0 tx=0x7fc754005fd0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 msgr2=0x7fc75819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc74800b530 tx=0x7fc74800b840 comp rx=0 tx=0).stop 2023-07-21T22:16:19.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 shutdown_connections 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7fc7440a1260 0x7fc7440a3710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc758100640 0x7fc75819d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7580fea00 0x7fc7581985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 --2- 172.21.15.93:0/3103994736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7580ffc90 0x7fc75819ab20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 >> 172.21.15.93:0/3103994736 conn(0x7fc7580fa000 msgr2=0x7fc7580fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 shutdown_connections 2023-07-21T22:16:19.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.344+0000 7fc7604ca700 1 -- 172.21.15.93:0/3103994736 wait complete. 2023-07-21T22:16:19.401 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-21T22:16:19.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- 172.21.15.93:0/2941195568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c0fea80 msgr2=0x7f3f7c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/2941195568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c0fee90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- 172.21.15.93:0/2941195568 shutdown_connections 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/2941195568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c100600 0x7f3f7c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/2941195568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c0fee90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/2941195568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 0x7f3f7c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- 172.21.15.93:0/2941195568 >> 172.21.15.93:0/2941195568 conn(0x7f3f7c0fa0c0 msgr2=0x7f3f7c0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:19.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- 172.21.15.93:0/2941195568 shutdown_connections 2023-07-21T22:16:19.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- 172.21.15.93:0/2941195568 wait complete. 2023-07-21T22:16:19.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 Processor -- start 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- start start 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 0x7f3f7c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c197360 con 0x7f3f7c100600 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1974d0 con 0x7f3f7c0ffc70 2023-07-21T22:16:19.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.501+0000 7f3f82ca2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c197690 con 0x7f3f7c0fea80 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44018/0 (socket says 172.21.15.93:44018) 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 -- 172.21.15.93:0/3286568666 learned_addr learned my addr 172.21.15.93:0/3286568666 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f7bfff700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 0x7f3f7c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f80a3e700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c0fea80 msgr2=0x7f3f7c195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 msgr2=0x7f3f7c195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 0x7f3f7c195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 -- 172.21.15.93:0/3286568666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700095f0 con 0x7f3f7c100600 2023-07-21T22:16:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f8123f700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007ee0 tx=0x7f3f7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400dd20 con 0x7f3f7c100600 2023-07-21T22:16:19.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c197970 con 0x7f3f7c100600 2023-07-21T22:16:19.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7400f040 con 0x7f3f7c100600 2023-07-21T22:16:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.502+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740048a0 con 0x7f3f7c100600 2023-07-21T22:16:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.503+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c19e130 con 0x7f3f7c100600 2023-07-21T22:16:19.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.504+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c04e2f0 con 0x7f3f7c100600 2023-07-21T22:16:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.505+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f3f74003a00 con 0x7f3f7c100600 2023-07-21T22:16:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.505+0000 7f3f79ffb700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 0x7f3f6c0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.505+0000 7f3f80a3e700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 msgr2=0x7f3f6c0a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.506+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3f740cc2e0 con 0x7f3f7c100600 2023-07-21T22:16:19.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.506+0000 7f3f80a3e700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 0x7f3f6c0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:16:19.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.510+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3f7404e770 con 0x7f3f7c100600 2023-07-21T22:16:19.628 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mon[98288]: scrub queued for path: / 2023-07-21T22:16:19.628 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.628 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:16:19.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.629 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:16:19.629 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: ignoring --setuser ceph since I am not root 2023-07-21T22:16:19.629 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: ignoring --setgroup ceph since I am not root 2023-07-21T22:16:19.629 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-mgr[100406]: -- 172.21.15.194:0/2298631150 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610afc451e0 con 0x5610afc1ac00 2023-07-21T22:16:19.630 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:19.505+0000 7f225b99c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:16:19.630 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:19.584+0000 7f225b99c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:19.630 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: ignoring --setuser ceph since I am not root 2023-07-21T22:16:19.630 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: ignoring --setgroup ceph since I am not root 2023-07-21T22:16:19.630 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mgr[99345]: -- 172.21.15.134:0/3704567262 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a94a1411e0 con 0x55a94a116c00 2023-07-21T22:16:19.631 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:19.508+0000 7fafc1a78200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:16:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mon[97873]: scrub queued for path: / 2023-07-21T22:16:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:16:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:16:19.649 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-mon[96579]: scrub queued for path: / 2023-07-21T22:16:19.649 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.649 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setuser ceph since I am not root 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: ignoring --setgroup ceph since I am not root 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:19.523+0000 7fb708eac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:16:19.650 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:19.605+0000 7fb708eac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:19.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.705+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3f7c19e2c0 con 0x7f3f7c100600 2023-07-21T22:16:19.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.707+0000 7f3f80a3e700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 msgr2=0x7f3f6c0a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:19.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.707+0000 7f3f80a3e700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 0x7f3f6c0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:16:19.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.712+0000 7f3f79ffb700 1 -- 172.21.15.93:0/3286568666 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f74095900 con 0x7f3f7c100600 2023-07-21T22:16:19.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 msgr2=0x7f3f6c0a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:16:19.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 0x7f3f6c0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 msgr2=0x7f3f7c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3f74007ee0 tx=0x7f3f7400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 shutdown_connections 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.714+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f7c100600 0x7f3f7c196d00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f7c0fea80 0x7f3f7c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f3f6c0a1230 0x7f3f6c0a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 --2- 172.21.15.93:0/3286568666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f7c0ffc70 0x7f3f7c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 >> 172.21.15.93:0/3286568666 conn(0x7f3f7c0fa0c0 msgr2=0x7f3f7c068870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:19.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 shutdown_connections 2023-07-21T22:16:19.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.715+0000 7f3f82ca2700 1 -- 172.21.15.93:0/3286568666 wait complete. 2023-07-21T22:16:19.771 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-21T22:16:19.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- 172.21.15.93:0/3122337098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 msgr2=0x7f77e00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 --2- 172.21.15.93:0/3122337098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e00fff40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009990 tx=0x7f77d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- 172.21.15.93:0/3122337098 shutdown_connections 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 --2- 172.21.15.93:0/3122337098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e01016b0 0x7f77e0107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 --2- 172.21.15.93:0/3122337098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e00fff40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 --2- 172.21.15.93:0/3122337098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e0100d20 0x7f77e0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- 172.21.15.93:0/3122337098 >> 172.21.15.93:0/3122337098 conn(0x7f77e00fb130 msgr2=0x7f77e00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- 172.21.15.93:0/3122337098 shutdown_connections 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- 172.21.15.93:0/3122337098 wait complete. 2023-07-21T22:16:19.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 Processor -- start 2023-07-21T22:16:19.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.854+0000 7f77e6644700 1 -- start start 2023-07-21T22:16:19.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e0100d20 0x7f77e0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e01016b0 0x7f77e0073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019f910 con 0x7f77e00ffb30 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019fa50 con 0x7f77e01016b0 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e6644700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019fb90 con 0x7f77e0100d20 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44052/0 (socket says 172.21.15.93:44052) 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 -- 172.21.15.93:0/3439657214 learned_addr learned my addr 172.21.15.93:0/3439657214 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77df7fe700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e0100d20 0x7f77e0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77e4be1700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e01016b0 0x7f77e0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e0100d20 msgr2=0x7f77e0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e0100d20 0x7f77e0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e01016b0 msgr2=0x7f77e0073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:19.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e01016b0 0x7f77e0073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 -- 172.21.15.93:0/3439657214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00095f0 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dffff700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f77d0004730 tx=0x7f77d0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.855+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d001d070 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.856+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e019fdc0 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.856+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d0021410 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.856+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0017650 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.856+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e01a0190 con 0x7f77e00ffb30 2023-07-21T22:16:19.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.857+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e004e360 con 0x7f77e00ffb30 2023-07-21T22:16:19.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.858+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f77d0021580 con 0x7f77e00ffb30 2023-07-21T22:16:19.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.858+0000 7f77dd7fa700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 0x7f77c80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:19.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.858+0000 7f77df7fe700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 msgr2=0x7f77c80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:19.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.859+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f77d00cdeb0 con 0x7f77e00ffb30 2023-07-21T22:16:19.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.859+0000 7f77df7fe700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 0x7f77c80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:16:19.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:19.863+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f77d0097560 con 0x7f77e00ffb30 2023-07-21T22:16:19.941 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:19.696+0000 7fb708eac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:16:19.941 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:19.798+0000 7fb708eac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:16:19.979 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:19.672+0000 7f225b99c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:16:19.979 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:19.766+0000 7f225b99c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:16:19.981 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:19.588+0000 7fafc1a78200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:19.982 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:19.676+0000 7fafc1a78200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:16:19.982 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:19.769+0000 7fafc1a78200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:16:20.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.060+0000 7f77df7fe700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 msgr2=0x7f77c80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:20.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.060+0000 7f77df7fe700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 0x7f77c80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:16:20.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.062+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f77e0106a30 con 0x7f77e00ffb30 2023-07-21T22:16:20.111 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.068+0000 7f77dd7fa700 1 -- 172.21.15.93:0/3439657214 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f77d009ab80 con 0x7f77e00ffb30 2023-07-21T22:16:20.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 msgr2=0x7f77c80a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:16:20.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 0x7f77c80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 msgr2=0x7f77e0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:20.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f77d0004730 tx=0x7f77d0004760 comp rx=0 tx=0).stop 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 shutdown_connections 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f77c80a1230 0x7f77c80a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e01016b0 0x7f77e0073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77e00ffb30 0x7f77e0072900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 --2- 172.21.15.93:0/3439657214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77e0100d20 0x7f77e0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 >> 172.21.15.93:0/3439657214 conn(0x7f77e00fb130 msgr2=0x7f77e00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:20.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 shutdown_connections 2023-07-21T22:16:20.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.070+0000 7f77e6644700 1 -- 172.21.15.93:0/3439657214 wait complete. 2023-07-21T22:16:20.126 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-21T22:16:20.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 -- 172.21.15.93:0/1522681333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00ff4d0 msgr2=0x7f85d00ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 --2- 172.21.15.93:0/1522681333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00ff4d0 0x7f85d00ffd60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b0a0 tx=0x7f85c800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 -- 172.21.15.93:0/1522681333 shutdown_connections 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 --2- 172.21.15.93:0/1522681333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00ff4d0 0x7f85d00ffd60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 --2- 172.21.15.93:0/1522681333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d00fddc0 0x7f85d00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 --2- 172.21.15.93:0/1522681333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85d00feb20 0x7f85d00fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 -- 172.21.15.93:0/1522681333 >> 172.21.15.93:0/1522681333 conn(0x7f85d00f9780 msgr2=0x7f85d00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 -- 172.21.15.93:0/1522681333 shutdown_connections 2023-07-21T22:16:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.207+0000 7f85d68f5700 1 -- 172.21.15.93:0/1522681333 wait complete. 2023-07-21T22:16:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 Processor -- start 2023-07-21T22:16:20.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 -- start start 2023-07-21T22:16:20.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d00fddc0 0x7f85d0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:20.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 0x7f85d0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:20.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85d00ff4d0 0x7f85d0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:20.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0199630 con 0x7f85d00feb20 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01997a0 con 0x7f85d00fddc0 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d68f5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0199960 con 0x7f85d00ff4d0 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 0x7f85d0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d00fddc0 0x7f85d0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 0x7f85d0198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44094/0 (socket says 172.21.15.93:44094) 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 -- 172.21.15.93:0/4198355700 learned_addr learned my addr 172.21.15.93:0/4198355700 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85d4e92700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85d00ff4d0 0x7f85d0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:20.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85d00ff4d0 msgr2=0x7f85d0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85d00ff4d0 0x7f85d0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.208+0000 7f85cf7fe700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d00fddc0 msgr2=0x7f85d0197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cf7fe700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d00fddc0 0x7f85d0197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cf7fe700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8009d00 con 0x7f85d00feb20 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cf7fe700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 0x7f85d0198150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9a0 tx=0x7f85c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4009790 con 0x7f85d00feb20 2023-07-21T22:16:20.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85c4010430 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d0199c40 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4018680 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.209+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d01a0400 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.210+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d0109960 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.211+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f85c40105a0 con 0x7f85d00feb20 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.211+0000 7f85cd7fa700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.211+0000 7f85cffff700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:20.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.212+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f85c4014070 con 0x7f85d00feb20 2023-07-21T22:16:20.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.212+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:16:20.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.216+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f85c4051770 con 0x7f85d00feb20 2023-07-21T22:16:20.286 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:19.935+0000 7f225b99c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:16:20.290 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:19.938+0000 7fafc1a78200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:16:20.340 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:19.975+0000 7fb708eac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:16:20.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.413+0000 7f85cffff700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:20.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.413+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:16:20.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.438+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d0060950 con 0x7f85b80a1230 2023-07-21T22:16:20.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:16:20.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-mon[98288]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:16:20.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3286568666' entity='client.admin' 2023-07-21T22:16:20.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3439657214' entity='client.admin' 2023-07-21T22:16:20.587 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:20.242+0000 7f225b99c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:16:20.587 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:20.332+0000 7f225b99c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:16:20.588 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:20.461+0000 7f225b99c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:16:20.602 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:20.247+0000 7fafc1a78200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:16:20.603 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:20.337+0000 7fafc1a78200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:16:20.603 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:20.466+0000 7fafc1a78200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:16:20.603 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:16:20.603 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-mon[97873]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:16:20.603 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3286568666' entity='client.admin' 2023-07-21T22:16:20.604 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3439657214' entity='client.admin' 2023-07-21T22:16:20.656 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:20.296+0000 7fb708eac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:16:20.657 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:20.389+0000 7fb708eac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:16:20.657 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:20.518+0000 7fb708eac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:16:20.657 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3103994736' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:16:20.657 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-mon[96579]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:16:20.658 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3286568666' entity='client.admin' 2023-07-21T22:16:20.658 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3439657214' entity='client.admin' 2023-07-21T22:16:20.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.814+0000 7f85cffff700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:20.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:20.814+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:16:20.878 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:20.559+0000 7fafc1a78200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:16:21.017 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:20.555+0000 7f225b99c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:16:21.336 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:20.973+0000 7f225b99c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:16:21.378 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:20.976+0000 7fafc1a78200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:16:21.440 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:20.612+0000 7fb708eac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:16:21.441 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:21.030+0000 7fb708eac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:16:21.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:21.615+0000 7f85cffff700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:21.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:21.615+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:16:21.836 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:21.533+0000 7f225b99c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:16:21.837 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:21.704+0000 7f225b99c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:16:21.878 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:21.538+0000 7fafc1a78200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:16:21.879 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:21.708+0000 7fafc1a78200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:16:21.941 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:21.587+0000 7fb708eac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:16:21.941 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:21.756+0000 7fb708eac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:16:22.772 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:22.468+0000 7f225b99c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:22.772 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:22.558+0000 7f225b99c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:16:22.773 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:22.470+0000 7fafc1a78200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:22.773 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:22.561+0000 7fafc1a78200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:16:22.819 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:22.514+0000 7fb708eac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:22.820 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:22.605+0000 7fb708eac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:16:23.048 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:22.727+0000 7f225b99c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:23.049 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:22.730+0000 7fafc1a78200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:23.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:23.216+0000 7f85cffff700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2721150511 2023-07-21T22:16:23.260 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:23.216+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:16:23.336 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:23.004+0000 7f225b99c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:16:23.337 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:23.232+0000 7f225b99c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:16:23.378 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:23.005+0000 7fafc1a78200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:16:23.379 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:23.233+0000 7fafc1a78200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:16:23.441 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:22.776+0000 7fb708eac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:16:23.441 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:23.054+0000 7fb708eac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:16:23.441 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:23.279+0000 7fb708eac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:16:23.681 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:23.379+0000 7f225b99c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:16:23.685 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:23.381+0000 7fafc1a78200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:16:23.736 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:23.428+0000 7fb708eac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:16:24.086 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:23.637+0000 7f225b99c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:16:24.128 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:23.642+0000 7fafc1a78200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:16:24.191 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:23.692+0000 7fb708eac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:16:24.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:24.394+0000 7fb708eac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:16:24.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:24.525+0000 7fb708eac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:16:24.696 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:24.334+0000 7f225b99c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:16:24.696 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:24.463+0000 7f225b99c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:16:24.702 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:24.340+0000 7fafc1a78200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:16:24.702 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:24.470+0000 7fafc1a78200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:16:25.022 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:24.652+0000 7f225b99c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:16:25.022 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:24.766+0000 7f225b99c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:25.030 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:24.659+0000 7fafc1a78200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:16:25.030 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:24.772+0000 7fafc1a78200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:25.099 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:24.718+0000 7fb708eac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:16:25.099 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:24.835+0000 7fb708eac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:16:25.099 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:25.055+0000 7fb708eac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:16:25.336 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:24.978+0000 7f225b99c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:16:25.378 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:24.987+0000 7fafc1a78200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:16:25.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:25.904+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 19) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f85c4010880 con 0x7f85d00feb20 2023-07-21T22:16:26.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:25.958+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mgrmap(e 20) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f85c4096680 con 0x7f85d00feb20 2023-07-21T22:16:26.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:25.958+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 msgr2=0x7f85b80a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:16:26.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:25.958+0000 7f85cd7fa700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/2721150511,v1:172.21.15.93:6801/2721150511] conn(0x7f85b80a1230 0x7f85b80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:26.173 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y[99321]: 2023-07-21T22:16:25.842+0000 7fafc1a78200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:16:26.173 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z[100307]: 2023-07-21T22:16:25.830+0000 7f225b99c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: Standby manager daemon z restarted 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: Standby manager daemon z started 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:16:26.174 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: Standby manager daemon y restarted 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: Standby manager daemon y started 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:16:26.175 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:25 smithi194 ceph-mon[98288]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: Standby manager daemon z restarted 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: Standby manager daemon z started 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:16:26.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: Standby manager daemon y restarted 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: Standby manager daemon y started 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:16:26.177 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:25 smithi134 ceph-mon[97873]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.221 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: Standby manager daemon z restarted 2023-07-21T22:16:26.221 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: Standby manager daemon z started 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.194:0/364337395' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: Standby manager daemon y restarted 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: Standby manager daemon y started 2023-07-21T22:16:26.222 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-mon[96579]: from='mgr.? 172.21.15.134:0/4118378050' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:25.931+0000 7fb708eac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb66830c700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb66830c700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.223 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb66830c700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.224 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb66830c700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.224 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb66830c700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb665306700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb665306700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb665306700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb665306700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:26.691 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:16:26.177+0000 7fb665306700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:16:27.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:26.961+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mon.0 v2:172.21.15.93:3300/0 9 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f85c40ca8c0 con 0x7f85d00feb20 2023-07-21T22:16:27.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:26.962+0000 7f85cd7fa700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f85b8027be0 0x7f85b8027fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:26.962+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d0060950 con 0x7f85b8027be0 2023-07-21T22:16:27.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:26.964+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f85b8027be0 0x7f85b8027fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:26.967+0000 7f85cffff700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f85b8027be0 0x7f85b8027fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009880 tx=0x7f85c0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:27.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.028+0000 7f85cd7fa700 1 -- 172.21.15.93:0/4198355700 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f85d0060950 con 0x7f85b8027be0 2023-07-21T22:16:27.072 INFO:teuthology.orchestra.run.smithi093.stdout:Schedule set for path / 2023-07-21T22:16:27.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.030+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f85b8027be0 msgr2=0x7f85b8027fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.030+0000 7f85d68f5700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f85b8027be0 0x7f85b8027fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009880 tx=0x7f85c0009260 comp rx=0 tx=0).stop 2023-07-21T22:16:27.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.030+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 msgr2=0x7f85d0198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.030+0000 7f85d68f5700 1 --2- 172.21.15.93:0/4198355700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85d00feb20 0x7f85d0198150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f85c400e9a0 tx=0x7f85c400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.030+0000 7f85d4e92700 1 -- 172.21.15.93:0/4198355700 reap_dead start 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.031+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 shutdown_connections 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.031+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 >> 172.21.15.93:0/4198355700 conn(0x7f85d00f9780 msgr2=0x7f85d00fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.031+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 shutdown_connections 2023-07-21T22:16:27.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.031+0000 7f85d68f5700 1 -- 172.21.15.93:0/4198355700 wait complete. 2023-07-21T22:16:27.089 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-21T22:16:27.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 -- 172.21.15.93:0/262813697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c105840 msgr2=0x7fe32c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 --2- 172.21.15.93:0/262813697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c105840 0x7fe32c107ca0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe32400b0a0 tx=0x7fe32400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 -- 172.21.15.93:0/262813697 shutdown_connections 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 --2- 172.21.15.93:0/262813697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c105840 0x7fe32c107ca0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 --2- 172.21.15.93:0/262813697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c100600 0x7fe32c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 --2- 172.21.15.93:0/262813697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c102f20 0x7fe32c105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 -- 172.21.15.93:0/262813697 >> 172.21.15.93:0/262813697 conn(0x7fe32c0fa0c0 msgr2=0x7fe32c0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 -- 172.21.15.93:0/262813697 shutdown_connections 2023-07-21T22:16:27.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.183+0000 7fe3318e8700 1 -- 172.21.15.93:0/262813697 wait complete. 2023-07-21T22:16:27.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 Processor -- start 2023-07-21T22:16:27.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 -- start start 2023-07-21T22:16:27.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c102f20 0x7fe32c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c105840 0x7fe32c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c197450 con 0x7fe32c100600 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1975c0 con 0x7fe32c105840 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe3318e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c197780 con 0x7fe32c102f20 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe32affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe32affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44334/0 (socket says 172.21.15.93:44334) 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe32affd700 1 -- 172.21.15.93:0/855782847 learned_addr learned my addr 172.21.15.93:0/855782847 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:27.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.184+0000 7fe32a7fc700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c102f20 0x7fe32c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32b7fe700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c105840 0x7fe32c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 -- 172.21.15.93:0/855782847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c102f20 msgr2=0x7fe32c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c102f20 0x7fe32c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 -- 172.21.15.93:0/855782847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c105840 msgr2=0x7fe32c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c105840 0x7fe32c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 -- 172.21.15.93:0/855782847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324009d00 con 0x7fe32c100600 2023-07-21T22:16:27.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe32affd700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00e9a0 tx=0x7fe31c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c009790 con 0x7fe32c100600 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe31c00cbd0 con 0x7fe32c100600 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c197a60 con 0x7fe32c100600 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c018680 con 0x7fe32c100600 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.185+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c19e220 con 0x7fe32c100600 2023-07-21T22:16:27.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.186+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0fbc30 con 0x7fe32c100600 2023-07-21T22:16:27.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.187+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fe31c00cd40 con 0x7fe32c100600 2023-07-21T22:16:27.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.188+0000 7fe3308e6700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 0x7fe3140a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.188+0000 7fe32a7fc700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 0x7fe3140a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.188+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe31c014070 con 0x7fe32c100600 2023-07-21T22:16:27.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.188+0000 7fe32a7fc700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 0x7fe3140a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe320009c60 tx=0x7fe320009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:27.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.192+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe31c051770 con 0x7fe32c100600 2023-07-21T22:16:27.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-21T22:16:27.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: Active manager daemon x restarted 2023-07-21T22:16:27.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: Activating manager daemon x 2023-07-21T22:16:27.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: mgrmap e20: x(active, starting, since 0.0134634s), standbys: z, y 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:16:27.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:16:27.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:16:27.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:16:27.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: Manager daemon x is now available 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.342 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:16:27.343 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:26 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:16:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-21T22:16:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: Active manager daemon x restarted 2023-07-21T22:16:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: Activating manager daemon x 2023-07-21T22:16:27.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: mgrmap e20: x(active, starting, since 0.0134634s), standbys: z, y 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:16:27.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:16:27.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:16:27.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:27.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: Manager daemon x is now available 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.385 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:16:27.385 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:26 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:16:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-21T22:16:27.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: Active manager daemon x restarted 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: Activating manager daemon x 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: mgrmap e20: x(active, starting, since 0.0134634s), standbys: z, y 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:16:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:16:27.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:16:27.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:27.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:16:27.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: Manager daemon x is now available 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:16:27.447 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:26 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:16:27.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.446+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe32c060950 con 0x7fe3140a0f50 2023-07-21T22:16:27.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.500+0000 7fe3308e6700 1 -- 172.21.15.93:0/855782847 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe32c060950 con 0x7fe3140a0f50 2023-07-21T22:16:27.544 INFO:teuthology.orchestra.run.smithi093.stdout:Retention added to path / 2023-07-21T22:16:27.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 msgr2=0x7fe3140a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 0x7fe3140a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe320009c60 tx=0x7fe320009260 comp rx=0 tx=0).stop 2023-07-21T22:16:27.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 msgr2=0x7fe32c195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00e9a0 tx=0x7fe31c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 shutdown_connections 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe32c105840 0x7fe32c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe3140a0f50 0x7fe3140a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c100600 0x7fe32c195a30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 --2- 172.21.15.93:0/855782847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe32c102f20 0x7fe32c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 >> 172.21.15.93:0/855782847 conn(0x7fe32c0fa0c0 msgr2=0x7fe32c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:27.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 shutdown_connections 2023-07-21T22:16:27.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.503+0000 7fe3318e8700 1 -- 172.21.15.93:0/855782847 wait complete. 2023-07-21T22:16:27.563 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T22:16:27.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- 172.21.15.93:0/76203497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc964100640 msgr2=0x7fc964106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/76203497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc964100640 0x7fc964106ee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009ec0 tx=0x7fc95c00b330 comp rx=0 tx=0).stop 2023-07-21T22:16:27.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- 172.21.15.93:0/76203497 shutdown_connections 2023-07-21T22:16:27.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/76203497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc964100640 0x7fc964106ee0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/76203497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640fea00 0x7fc9640fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/76203497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9640ffc90 0x7fc964100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- 172.21.15.93:0/76203497 >> 172.21.15.93:0/76203497 conn(0x7fc9640fa000 msgr2=0x7fc9640fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- 172.21.15.93:0/76203497 shutdown_connections 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- 172.21.15.93:0/76203497 wait complete. 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 Processor -- start 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- start start 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640ffc90 0x7fc964198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc964100640 0x7fc964198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964199610 con 0x7fc9640fea00 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964199780 con 0x7fc964100640 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.649+0000 7fc96ad5d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964199940 con 0x7fc9640ffc90 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44354/0 (socket says 172.21.15.93:44354) 2023-07-21T22:16:27.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 -- 172.21.15.93:0/697070495 learned_addr learned my addr 172.21.15.93:0/697070495 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc963fff700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640ffc90 0x7fc964198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 -- 172.21.15.93:0/697070495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640ffc90 msgr2=0x7fc964198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640ffc90 0x7fc964198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 -- 172.21.15.93:0/697070495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc964100640 msgr2=0x7fc964198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc964100640 0x7fc964198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 -- 172.21.15.93:0/697070495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c005e70 con 0x7fc9640fea00 2023-07-21T22:16:27.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc968af9700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc95800d980 tx=0x7fc95800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:27.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800bb60 con 0x7fc9640fea00 2023-07-21T22:16:27.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964199c20 con 0x7fc9640fea00 2023-07-21T22:16:27.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc958007640 con 0x7fc9640fea00 2023-07-21T22:16:27.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958004e90 con 0x7fc9640fea00 2023-07-21T22:16:27.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.650+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641a03e0 con 0x7fc9640fea00 2023-07-21T22:16:27.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.651+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96404e2f0 con 0x7fc9640fea00 2023-07-21T22:16:27.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.652+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fc95800bcc0 con 0x7fc9640fea00 2023-07-21T22:16:27.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.653+0000 7fc961ffb700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 0x7fc9540a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:27.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.653+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fc9580cca80 con 0x7fc9640fea00 2023-07-21T22:16:27.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.653+0000 7fc963fff700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 0x7fc9540a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:27.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.653+0000 7fc963fff700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 0x7fc9540a33e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc95000dad0 tx=0x7fc95000d4b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:27.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.656+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc958099e50 con 0x7fc9640fea00 2023-07-21T22:16:27.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.878+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc964060950 con 0x7fc9540a0f30 2023-07-21T22:16:27.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.886+0000 7fc961ffb700 1 -- 172.21.15.93:0/697070495 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fc964060950 con 0x7fc9540a0f30 2023-07-21T22:16:27.930 INFO:teuthology.orchestra.run.smithi093.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T22:16:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-21T22:16:27.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 msgr2=0x7fc9540a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 0x7fc9540a33e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc95000dad0 tx=0x7fc95000d4b0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 msgr2=0x7fc964197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc95800d980 tx=0x7fc95800dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 shutdown_connections 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc964100640 0x7fc964198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc9540a0f30 0x7fc9540a33e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9640fea00 0x7fc964197bf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 --2- 172.21.15.93:0/697070495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9640ffc90 0x7fc964198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:27.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 >> 172.21.15.93:0/697070495 conn(0x7fc9640fa000 msgr2=0x7fc9640fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:27.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 shutdown_connections 2023-07-21T22:16:27.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:27.888+0000 7fc96ad5d700 1 -- 172.21.15.93:0/697070495 wait complete. 2023-07-21T22:16:27.946 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T22:16:28.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4094481168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c0ffc10 msgr2=0x7fdc5c100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4094481168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c100060 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009990 tx=0x7fdc50009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4094481168 shutdown_connections 2023-07-21T22:16:28.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4094481168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4094481168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4094481168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c100060 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4094481168 >> 172.21.15.93:0/4094481168 conn(0x7fdc5c0fa020 msgr2=0x7fdc5c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:28.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4094481168 shutdown_connections 2023-07-21T22:16:28.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.031+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4094481168 wait complete. 2023-07-21T22:16:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 Processor -- start 2023-07-21T22:16:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 -- start start 2023-07-21T22:16:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c196260 con 0x7fdc5c1005a0 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c1963d0 con 0x7fdc5c0ffc10 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc63b9b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c196590 con 0x7fdc5c0fea20 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc62138700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc62138700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44374/0 (socket says 172.21.15.93:44374) 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc62138700 1 -- 172.21.15.93:0/4034904276 learned_addr learned my addr 172.21.15.93:0/4034904276 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.032+0000 7fdc61136700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc61937700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 -- 172.21.15.93:0/4034904276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 msgr2=0x7fdc5c194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 -- 172.21.15.93:0/4034904276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0ffc10 msgr2=0x7fdc5c194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 -- 172.21.15.93:0/4034904276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500095f0 con 0x7fdc5c1005a0 2023-07-21T22:16:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc61937700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc62138700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400d980 tx=0x7fdc5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004a30 con 0x7fdc5c1005a0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5400b5c0 con 0x7fdc5c1005a0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5c196870 con 0x7fdc5c1005a0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54009b50 con 0x7fdc5c1005a0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.033+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c19d030 con 0x7fdc5c1005a0 2023-07-21T22:16:28.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.034+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5c105f90 con 0x7fdc5c1005a0 2023-07-21T22:16:28.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.035+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fdc54009cb0 con 0x7fdc5c1005a0 2023-07-21T22:16:28.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.036+0000 7fdc5affd700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 0x7fdc440a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.036+0000 7fdc61937700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 0x7fdc440a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.036+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdc540cc8f0 con 0x7fdc5c1005a0 2023-07-21T22:16:28.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.036+0000 7fdc61937700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 0x7fdc440a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0ffa70 tx=0x7fdc4c00b040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:28.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.039+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdc54096230 con 0x7fdc5c1005a0 2023-07-21T22:16:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.267+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdc5c02ccd0 con 0x7fdc440a0f50 2023-07-21T22:16:28.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.274+0000 7fdc5affd700 1 -- 172.21.15.93:0/4034904276 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fdc5c02ccd0 con 0x7fdc440a0f50 2023-07-21T22:16:28.318 INFO:teuthology.orchestra.run.smithi093.stdout:/ 1M 6M3h 2023-07-21T22:16:28.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 msgr2=0x7fdc440a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 0x7fdc440a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0ffa70 tx=0x7fdc4c00b040 comp rx=0 tx=0).stop 2023-07-21T22:16:28.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 msgr2=0x7fdc5c195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400d980 tx=0x7fdc5400dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:28.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 shutdown_connections 2023-07-21T22:16:28.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdc440a0f50 0x7fdc440a3400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc5c1005a0 0x7fdc5c195c00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc5c0fea20 0x7fdc5c194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 --2- 172.21.15.93:0/4034904276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc5c0ffc10 0x7fdc5c194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 >> 172.21.15.93:0/4034904276 conn(0x7fdc5c0fa020 msgr2=0x7fdc5c1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 shutdown_connections 2023-07-21T22:16:28.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:28.276+0000 7fdc63b9b700 1 -- 172.21.15.93:0/4034904276 wait complete. 2023-07-21T22:16:28.330 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:27 smithi093 ceph-mon[96579]: mgrmap e21: x(active, since 1.01744s), standbys: z, y 2023-07-21T22:16:28.331 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:27 smithi093 ceph-mon[96579]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:28.331 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:27 smithi093 ceph-mon[96579]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:28.332 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-21T22:16:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:27 smithi194 ceph-mon[98288]: mgrmap e21: x(active, since 1.01744s), standbys: z, y 2023-07-21T22:16:28.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:27 smithi194 ceph-mon[98288]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:28.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:27 smithi194 ceph-mon[98288]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:28.359 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-21T22:16:28.370 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-21T22:16:28.409 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:16:28.410 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-21T22:16:28.410 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:27 smithi134 ceph-mon[97873]: mgrmap e21: x(active, since 1.01744s), standbys: z, y 2023-07-21T22:16:28.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:27 smithi134 ceph-mon[97873]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:28.411 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:27 smithi134 ceph-mon[97873]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:28.413 INFO:teuthology.run_tasks:Running task workunit... 2023-07-21T22:16:28.424 INFO:tasks.workunit:Pulling workunits from ref bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:16:28.425 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-21T22:16:28.425 INFO:tasks.workunit:timeout=3h 2023-07-21T22:16:28.425 INFO:tasks.workunit:cleanup=True 2023-07-21T22:16:28.426 DEBUG:teuthology.orchestra.run.smithi093:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:28.446 INFO:teuthology.orchestra.run.smithi093.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:28.446 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-07-21 22:14:57.768648127 +0000 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-07-21 22:14:57.768648127 +0000 2023-07-21T22:16:28.447 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-07-21 22:16:11.385822196 +0000 2023-07-21T22:16:28.448 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-07-21 22:14:57.768648127 +0000 2023-07-21T22:16:28.448 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-21T22:16:28.448 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-21T22:16:28.565 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:16:28.582 INFO:tasks.workunit.client.0.smithi093.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-21T22:16:29.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: [21/Jul/2023:22:16:27] ENGINE Bus STARTING 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: [21/Jul/2023:22:16:27] ENGINE Serving on http://172.21.15.93:8765 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: [21/Jul/2023:22:16:27] ENGINE Serving on https://172.21.15.93:7150 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: [21/Jul/2023:22:16:27] ENGINE Bus STARTED 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:28 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:16:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: [21/Jul/2023:22:16:27] ENGINE Bus STARTING 2023-07-21T22:16:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: [21/Jul/2023:22:16:27] ENGINE Serving on http://172.21.15.93:8765 2023-07-21T22:16:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: [21/Jul/2023:22:16:27] ENGINE Serving on https://172.21.15.93:7150 2023-07-21T22:16:29.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: [21/Jul/2023:22:16:27] ENGINE Bus STARTED 2023-07-21T22:16:29.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:29.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:28 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:16:29.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: [21/Jul/2023:22:16:27] ENGINE Bus STARTING 2023-07-21T22:16:29.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: [21/Jul/2023:22:16:27] ENGINE Serving on http://172.21.15.93:8765 2023-07-21T22:16:29.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: [21/Jul/2023:22:16:27] ENGINE Serving on https://172.21.15.93:7150 2023-07-21T22:16:29.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: [21/Jul/2023:22:16:27] ENGINE Bus STARTED 2023-07-21T22:16:29.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:29.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:28 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:16:30.336 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:29 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-07-21T22:16:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:29 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-07-21T22:16:30.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:30.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:29 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:31.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.337 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.338 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.339 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:16:31.340 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.341 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:31 smithi194 ceph-mon[98288]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:31.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:16:31.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:16:31.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.384 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:31 smithi134 ceph-mon[97873]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:16:31.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi093:/etc/ceph/ceph.conf 2023-07-21T22:16:31.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-21T22:16:31.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi194:/etc/ceph/ceph.conf 2023-07-21T22:16:31.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:31.446 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:31 smithi093 ceph-mon[96579]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:16:32.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:32.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:32.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:32.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:32.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:16:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: Updating smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:32.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:32.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:32.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:32.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:16:33.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:33 smithi194 ceph-mon[98288]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 184 KiB/s wr, 76 op/s 2023-07-21T22:16:33.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:33 smithi134 ceph-mon[97873]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 184 KiB/s wr, 76 op/s 2023-07-21T22:16:33.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:33 smithi093 ceph-mon[96579]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 184 KiB/s wr, 76 op/s 2023-07-21T22:16:35.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:35 smithi194 ceph-mon[98288]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T22:16:35.628 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:35 smithi134 ceph-mon[97873]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T22:16:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:35 smithi093 ceph-mon[96579]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 132 KiB/s wr, 54 op/s 2023-07-21T22:16:37.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:37 smithi194 ceph-mon[98288]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T22:16:37.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:37 smithi134 ceph-mon[97873]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T22:16:37.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:37 smithi093 ceph-mon[96579]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 103 KiB/s wr, 42 op/s 2023-07-21T22:16:39.586 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:39 smithi194 ceph-mon[98288]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T22:16:39.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:39 smithi134 ceph-mon[97873]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T22:16:39.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:39 smithi093 ceph-mon[96579]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 84 KiB/s wr, 39 op/s 2023-07-21T22:16:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 -- 172.21.15.93:0/635219391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90100d30 msgr2=0x7fce90101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 --2- 172.21.15.93:0/635219391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90100d30 0x7fce90101100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fce84009990 tx=0x7fce84009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 -- 172.21.15.93:0/635219391 shutdown_connections 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 --2- 172.21.15.93:0/635219391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 0x7fce9010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 --2- 172.21.15.93:0/635219391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90100d30 0x7fce90101100 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 --2- 172.21.15.93:0/635219391 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90101640 0x7fce90109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 -- 172.21.15.93:0/635219391 >> 172.21.15.93:0/635219391 conn(0x7fce900fa500 msgr2=0x7fce900fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 -- 172.21.15.93:0/635219391 shutdown_connections 2023-07-21T22:16:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.532+0000 7fce95e49700 1 -- 172.21.15.93:0/635219391 wait complete. 2023-07-21T22:16:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 Processor -- start 2023-07-21T22:16:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 -- start start 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90100d30 0x7fce9019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 0x7fce90195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90195fe0 con 0x7fce90101640 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90196150 con 0x7fce9010a070 2023-07-21T22:16:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce95e49700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90196310 con 0x7fce90100d30 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44392/0 (socket says 172.21.15.93:44392) 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8effd700 1 -- 172.21.15.93:0/3963143432 learned_addr learned my addr 172.21.15.93:0/3963143432 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8ffff700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 0x7fce90195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90100d30 0x7fce9019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.533+0000 7fce8effd700 1 -- 172.21.15.93:0/3963143432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90100d30 msgr2=0x7fce9019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8effd700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90100d30 0x7fce9019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8effd700 1 -- 172.21.15.93:0/3963143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 msgr2=0x7fce90195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8effd700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 0x7fce90195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8effd700 1 -- 172.21.15.93:0/3963143432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840095f0 con 0x7fce90101640 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8effd700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c9b0 tx=0x7fce8000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8cff9700 1 -- 172.21.15.93:0/3963143432 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80007b00 con 0x7fce90101640 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8cff9700 1 -- 172.21.15.93:0/3963143432 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce80007c60 con 0x7fce90101640 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce901965f0 con 0x7fce90101640 2023-07-21T22:16:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce8cff9700 1 -- 172.21.15.93:0/3963143432 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80018660 con 0x7fce90101640 2023-07-21T22:16:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.534+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce901a2b00 con 0x7fce90101640 2023-07-21T22:16:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.536+0000 7fce8cff9700 1 -- 172.21.15.93:0/3963143432 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fce8000f450 con 0x7fce90101640 2023-07-21T22:16:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.536+0000 7fce8cff9700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 0x7fce780a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.537+0000 7fce8cff9700 1 -- 172.21.15.93:0/3963143432 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce800cce90 con 0x7fce90101640 2023-07-21T22:16:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.537+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 0x7fce780a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.537+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 0x7fce780a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce8400b540 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.537+0000 7fcea1fceb80 1 Processor -- start 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 -- start start 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd03cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dcfdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dc03c50 con 0x55fb6dd04660 2023-07-21T22:16:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dc569e0 con 0x55fb6dd04210 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dc4fe70 con 0x55fb6db5a6b0 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44402/0 (socket says 172.21.15.93:44402) 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 -- 172.21.15.93:0/3417988761 learned_addr learned my addr 172.21.15.93:0/3417988761 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd03cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8effd700 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dcfdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 -- 172.21.15.93:0/3417988761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 msgr2=0x55fb6dd03cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd03cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 -- 172.21.15.93:0/3417988761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 msgr2=0x55fb6dcfdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dcfdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 -- 172.21.15.93:0/3417988761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6da7dd80 con 0x55fb6dd04660 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.538+0000 7fce8ffff700 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce88003f40 tx=0x7fce88009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5bd2a6e556b6fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fce75ffb700 1 -- 172.21.15.93:0/3417988761 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88019070 con 0x55fb6dd04660 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fce75ffb700 1 -- 172.21.15.93:0/3417988761 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce8800f040 con 0x55fb6dd04660 2023-07-21T22:16:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fce75ffb700 1 -- 172.21.15.93:0/3417988761 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88007950 con 0x55fb6dd04660 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3417988761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 msgr2=0x55fb6dcfe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce88003f40 tx=0x7fce88009ba0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3417988761 shutdown_connections 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dcfe4a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd03cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3417988761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dcfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3417988761 >> 172.21.15.93:0/3417988761 conn(0x55fb6dcf3ae0 msgr2=0x55fb6dcf90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3417988761 shutdown_connections 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.539+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3417988761 wait complete. 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 -- start start 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 Processor -- start 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dbefa60 con 0x55fb6dd04660 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dbef760 con 0x55fb6dd04210 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dad38c0 con 0x55fb6db5a6b0 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fcea1fceb80 10 client.0 _ll_register_callbacks cb 0x55fb6dbed700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fce8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44408/0 (socket says 172.21.15.93:44408) 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fce8ffff700 1 -- 172.21.15.93:0/4229608308 learned_addr learned my addr 172.21.15.93:0/4229608308 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.540+0000 7fce8effd700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 -- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 msgr2=0x55fb6dd0a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 -- 172.21.15.93:0/4229608308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 msgr2=0x55fb6dd07760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd07760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 -- 172.21.15.93:0/4229608308 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce88003b70 con 0x55fb6dd04660 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce8ffff700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fce88003f10 tx=0x7fce88004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d909c2fabb9c3cac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88019040 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce88007910 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb6dba9490 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce88007ad0 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fcea1fceb80 10 client.15540 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fb6da66c00 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb6dba9490 con 0x55fb6dd04660 2023-07-21T22:16:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce8801f450 con 0x55fb6dd04660 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fcea1fceb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.541+0000 7fcea1fceb80 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fcea1fceb80 20 client.15540 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd179d0 0x55fb6dd19de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fcea1fceb80 4 client.15540 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb6dc48e50 con 0x55fb6dd179d0 2023-07-21T22:16:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd179d0 0x55fb6dd19de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fce6effd700 10 client.15540 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.542+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd179d0 0x55fb6dd19de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639049251d24c211 server_cookie=43aeb931794728d5 in_seq=0 out_seq=0 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fce6effd700 1 -- 172.21.15.93:0/4229608308 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb6dc48e50 con 0x55fb6dd179d0 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fce6effd700 10 client.15540 handle_command_reply: tid=0 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 client.15540 shutdown 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 msgr2=0x55fb6dd07d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fce88003f10 tx=0x7fce88004b90 comp rx=0 tx=0).stop 2023-07-21T22:16:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 shutdown_connections 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd179d0 0x55fb6dd19de0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04660 0x55fb6dd07d70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fce6effd700 0 client.15540 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/4229608308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd07760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 >> 172.21.15.93:0/4229608308 conn(0x55fb6dcf3ae0 msgr2=0x55fb6dcf83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 shutdown_connections 2023-07-21T22:16:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 1 -- 172.21.15.93:0/4229608308 wait complete. 2023-07-21T22:16:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.543+0000 7fcea1fceb80 20 client.15540 trim_cache size 0 max 16384 2023-07-21T22:16:41.599 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:41 smithi194 ceph-mon[98288]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 36 op/s 2023-07-21T22:16:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 Processor -- start 2023-07-21T22:16:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 -- start start 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dcfdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04660 0x55fb6dd05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dd14440 con 0x55fb6dd04210 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dcfef50 con 0x55fb6dd04660 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dcf9d80 con 0x55fb6db5a6b0 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44414/0 (socket says 172.21.15.93:44414) 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 -- 172.21.15.93:0/3940091704 learned_addr learned my addr 172.21.15.93:0/3940091704 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dcfdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 -- 172.21.15.93:0/3940091704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 msgr2=0x55fb6dcfdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8ffff700 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04660 0x55fb6dd05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dcfdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 -- 172.21.15.93:0/3940091704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04660 msgr2=0x55fb6dd05140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04660 0x55fb6dd05140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.558+0000 7fce8effd700 1 -- 172.21.15.93:0/3940091704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6daa41f0 con 0x55fb6dd04210 2023-07-21T22:16:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fce8effd700 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fce800ca620 tx=0x7fce8000a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2118d18344633e1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fce767fc700 1 -- 172.21.15.93:0/3940091704 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8009de40 con 0x55fb6dd04210 2023-07-21T22:16:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fce767fc700 1 -- 172.21.15.93:0/3940091704 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce800a2260 con 0x55fb6dd04210 2023-07-21T22:16:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fce767fc700 1 -- 172.21.15.93:0/3940091704 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8000f700 con 0x55fb6dd04210 2023-07-21T22:16:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3940091704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 msgr2=0x55fb6dd0eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fce800ca620 tx=0x7fce8000a690 comp rx=0 tx=0).stop 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3940091704 shutdown_connections 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04660 0x55fb6dd05140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6db5a6b0 0x55fb6dcfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/3940091704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6dd04210 0x55fb6dd0eb70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3940091704 >> 172.21.15.93:0/3940091704 conn(0x55fb6dcf3ae0 msgr2=0x55fb6dd04a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3940091704 shutdown_connections 2023-07-21T22:16:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.559+0000 7fcea1fceb80 1 -- 172.21.15.93:0/3940091704 wait complete. 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 -- start start 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 Processor -- start 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6dd04660 0x55fb6dd0b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dd0bb40 con 0x55fb6db5a6b0 2023-07-21T22:16:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dd0bcb0 con 0x55fb6dd04210 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6dd6b890 con 0x55fb6dd04660 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fcea1fceb80 10 client.0 _ll_register_callbacks cb 0x7fce800ca3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44418/0 (socket says 172.21.15.93:44418) 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 -- 172.21.15.93:0/2458477205 learned_addr learned my addr 172.21.15.93:0/2458477205 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8effd700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8ffff700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6dd04660 0x55fb6dd0b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 -- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6dd04660 msgr2=0x55fb6dd0b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6dd04660 0x55fb6dd0b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 -- 172.21.15.93:0/2458477205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 msgr2=0x55fb6dd0af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.560+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd0af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce8f7fe700 1 -- 172.21.15.93:0/2458477205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce901a2b00 con 0x55fb6db5a6b0 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce8effd700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce8f7fe700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fce84009990 tx=0x7fce84009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5beaec73c91b5d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8402b040 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce8400f920 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce8800f040 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce8400fae0 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 10 client.15546 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fb6da7c250 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce8800f040 con 0x55fb6db5a6b0 2023-07-21T22:16:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce84024610 con 0x55fb6db5a6b0 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 20 client.15546 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd6ce90 0x55fb6dd6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 4 client.15546 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.561+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb6dc0aff0 con 0x55fb6dd6ce90 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.562+0000 7fce8effd700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd6ce90 0x55fb6dd6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.562+0000 7fce6effd700 10 client.15546 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.562+0000 7fce8effd700 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd6ce90 0x55fb6dd6f2a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92c642d7e1bace8 server_cookie=aa7e40bb63a164c6 in_seq=0 out_seq=0 2023-07-21T22:16:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fce6effd700 1 -- 172.21.15.93:0/2458477205 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb6dc0aff0 con 0x55fb6dd6ce90 2023-07-21T22:16:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fce6effd700 10 client.15546 handle_command_reply: tid=0 2023-07-21T22:16:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 client.15546 shutdown 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 msgr2=0x55fb6dd0eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fce84009990 tx=0x7fce84009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 shutdown_connections 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fb6dd6ce90 0x55fb6dd6f2a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fb6dd04660 0x55fb6dd0b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb6db5a6b0 0x55fb6dd0eaf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 --2- 172.21.15.93:0/2458477205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb6dd04210 0x55fb6dd0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fce6effd700 0 client.15546 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 >> 172.21.15.93:0/2458477205 conn(0x55fb6dcf3ae0 msgr2=0x55fb6dd052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 shutdown_connections 2023-07-21T22:16:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 1 -- 172.21.15.93:0/2458477205 wait complete. 2023-07-21T22:16:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.574+0000 7fcea1fceb80 20 client.15546 trim_cache size 0 max 16384 2023-07-21T22:16:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:16:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:16:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:16:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 msgr2=0x7fce780a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 0x7fce780a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce8400b540 comp rx=0 tx=0).stop 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 msgr2=0x7fce9019c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce8000c9b0 tx=0x7fce8000ccc0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 shutdown_connections 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce9010a070 0x7fce90195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce90100d30 0x7fce9019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce780a1350 0x7fce780a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 --2- 172.21.15.93:0/3963143432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce90101640 0x7fce9019c050 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 >> 172.21.15.93:0/3963143432 conn(0x7fce900fa500 msgr2=0x7fce90101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 shutdown_connections 2023-07-21T22:16:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:41.575+0000 7fce95e49700 1 -- 172.21.15.93:0/3963143432 wait complete. 2023-07-21T22:16:41.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:41 smithi134 ceph-mon[97873]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 36 op/s 2023-07-21T22:16:41.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:41 smithi093 ceph-mon[96579]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 36 op/s 2023-07-21T22:16:43.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:43 smithi194 ceph-mon[98288]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 79 KiB/s wr, 44 op/s 2023-07-21T22:16:43.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:43 smithi134 ceph-mon[97873]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 79 KiB/s wr, 44 op/s 2023-07-21T22:16:43.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:43 smithi093 ceph-mon[96579]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 79 KiB/s wr, 44 op/s 2023-07-21T22:16:45.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:45 smithi194 ceph-mon[98288]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:45.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:45 smithi134 ceph-mon[97873]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:45.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:45 smithi093 ceph-mon[96579]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:47.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:47 smithi194 ceph-mon[98288]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:47.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:47 smithi134 ceph-mon[97873]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:47.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:47 smithi093 ceph-mon[96579]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:16:48.425 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:16:48.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 -- 172.21.15.93:0/1367856082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80ff380 msgr2=0x7fb7c810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/1367856082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80ff380 0x7fb7c810b810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b0a0 tx=0x7fb7c000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 -- 172.21.15.93:0/1367856082 shutdown_connections 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/1367856082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80ff380 0x7fb7c810b810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/1367856082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c81049e0 0x7fb7c8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/1367856082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c80fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 -- 172.21.15.93:0/1367856082 >> 172.21.15.93:0/1367856082 conn(0x7fb7c80fa4b0 msgr2=0x7fb7c80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 -- 172.21.15.93:0/1367856082 shutdown_connections 2023-07-21T22:16:48.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.510+0000 7fb7cfc53700 1 -- 172.21.15.93:0/1367856082 wait complete. 2023-07-21T22:16:48.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 Processor -- start 2023-07-21T22:16:48.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 -- start start 2023-07-21T22:16:48.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 0x7fb7c819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c81049e0 0x7fb7c8194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81950a0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8195210 con 0x7fb7c81049e0 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cfc53700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81953d0 con 0x7fb7c80ff380 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44432/0 (socket says 172.21.15.93:44432) 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 -- 172.21.15.93:0/2195617045 learned_addr learned my addr 172.21.15.93:0/2195617045 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 0x7fb7c819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c81049e0 0x7fb7c8194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 -- 172.21.15.93:0/2195617045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 msgr2=0x7fb7c819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 0x7fb7c819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 -- 172.21.15.93:0/2195617045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c81049e0 msgr2=0x7fb7c8194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c81049e0 0x7fb7c8194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 -- 172.21.15.93:0/2195617045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0009d00 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 0x7fb7c819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:48.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.511+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400d980 tx=0x7fb7c400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.512+0000 7fb7beffd700 1 -- 172.21.15.93:0/2195617045 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400de60 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.512+0000 7fb7beffd700 1 -- 172.21.15.93:0/2195617045 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c40044e0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.512+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c81956b0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.512+0000 7fb7beffd700 1 -- 172.21.15.93:0/2195617045 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4003be0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.560 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.512+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c81a19b0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.515+0000 7fb7beffd700 1 -- 172.21.15.93:0/2195617045 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fb7c4009bb0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.515+0000 7fb7beffd700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 0x7fb7b40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.566 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.515+0000 7fb7beffd700 1 -- 172.21.15.93:0/2195617045 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb7c40ccba0 con 0x7fb7c80fe9d0 2023-07-21T22:16:48.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.515+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 0x7fb7b40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 Processor -- start 2023-07-21T22:16:48.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 0x7fb7b40a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005950 tx=0x7fb7b80058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 -- start start 2023-07-21T22:16:48.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d90350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10c95cb0 con 0x558e10d90930 2023-07-21T22:16:48.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10ce8a40 con 0x558e10d95ac0 2023-07-21T22:16:48.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10ce1ed0 con 0x558e10d951b0 2023-07-21T22:16:48.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7ce1f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7ce1f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44444/0 (socket says 172.21.15.93:44444) 2023-07-21T22:16:48.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/2310970487 learned_addr learned my addr 172.21.15.93:0/2310970487 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d90350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.516+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/2310970487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 msgr2=0x558e10d95580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d95580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/2310970487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 msgr2=0x558e10d90350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d90350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/2310970487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e10b0fde0 con 0x558e10d90930 2023-07-21T22:16:48.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b0a0 tx=0x7fb7c0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b886c2434c5f7c7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7b3fff700 1 -- 172.21.15.93:0/2310970487 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000e050 con 0x558e10d90930 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7b3fff700 1 -- 172.21.15.93:0/2310970487 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c00074e0 con 0x558e10d90930 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.517+0000 7fb7b3fff700 1 -- 172.21.15.93:0/2310970487 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00054a0 con 0x558e10d90930 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/2310970487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 msgr2=0x558e10d90da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b0a0 tx=0x7fb7c0009c60 comp rx=0 tx=0).stop 2023-07-21T22:16:48.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/2310970487 shutdown_connections 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d90930 0x558e10d90da0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/2310970487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d90350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/2310970487 >> 172.21.15.93:0/2310970487 conn(0x558e10d85ef0 msgr2=0x558e10d8b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/2310970487 shutdown_connections 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/2310970487 wait complete. 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 -- start start 2023-07-21T22:16:48.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.518+0000 7fb7dbdd8b80 1 Processor -- start 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d99e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10c81ac0 con 0x558e10d95ac0 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10c817c0 con 0x558e10d90930 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10b65920 con 0x558e10d951b0 2023-07-21T22:16:48.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 10 client.0 _ll_register_callbacks cb 0x558e10c7f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44450/0 (socket says 172.21.15.93:44450) 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/313601076 learned_addr learned my addr 172.21.15.93:0/313601076 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d99e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 msgr2=0x558e10d99e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d99e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/313601076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 msgr2=0x558e10d9cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 -- 172.21.15.93:0/313601076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c810c110 con 0x558e10d95ac0 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00060e0 tx=0x7fb7c0004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b5d25ad936beb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c001b840 con 0x558e10d95ac0 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb7c001b9a0 con 0x558e10d95ac0 2023-07-21T22:16:48.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e10c3b4f0 con 0x558e10d95ac0 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.519+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c001bb60 con 0x558e10d95ac0 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 10 client.15558 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558e10af8c60 con 0x558e10d95ac0 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e10c3b4f0 con 0x558e10d95ac0 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7c000b0d0 con 0x558e10d95ac0 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:48.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 20 client.15558 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10dac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 4 client.15558 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e10cdaeb0 con 0x558e10da9f80 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.520+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10dac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7b17fa700 10 client.15558 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10dac390 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4091039b5ec4a4a1 server_cookie=323d639defb95c81 in_seq=0 out_seq=0 2023-07-21T22:16:48.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7b17fa700 1 -- 172.21.15.93:0/313601076 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e10cdaeb0 con 0x558e10da9f80 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7b17fa700 10 client.15558 handle_command_reply: tid=0 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7dbdd8b80 1 client.15558 shutdown 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 msgr2=0x558e10d9a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.521+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00060e0 tx=0x7fb7c0004b00 comp rx=0 tx=0).stop 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 shutdown_connections 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10dac390 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d95ac0 0x558e10d9a510 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7b17fa700 0 client.15558 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/313601076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d951b0 0x558e10d99e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 >> 172.21.15.93:0/313601076 conn(0x558e10d85ef0 msgr2=0x558e10d8b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 shutdown_connections 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/313601076 wait complete. 2023-07-21T22:16:48.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.522+0000 7fb7dbdd8b80 20 client.15558 trim_cache size 0 max 16384 2023-07-21T22:16:48.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 Processor -- start 2023-07-21T22:16:48.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 -- start start 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 0x558e10d996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10d88010 con 0x558e10d951b0 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10d94d30 con 0x558e10d90930 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10d8e340 con 0x558e10d95ac0 2023-07-21T22:16:48.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7cd1ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7cd1ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44456/0 (socket says 172.21.15.93:44456) 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/3875935554 learned_addr learned my addr 172.21.15.93:0/3875935554 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.540+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 0x558e10d996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/3875935554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 msgr2=0x558e10d996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 0x558e10d996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.586 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/3875935554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 msgr2=0x558e10d9a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/3875935554 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e10b36250 con 0x558e10d951b0 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 0x558e10d996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558e10d8ce70 tx=0x7fb7b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3aac7375351c207 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7bcdac700 1 -- 172.21.15.93:0/3875935554 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8017040 con 0x558e10d951b0 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7bcdac700 1 -- 172.21.15.93:0/3875935554 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b8003b40 con 0x558e10d951b0 2023-07-21T22:16:48.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.541+0000 7fb7bcdac700 1 -- 172.21.15.93:0/3875935554 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8021cb0 con 0x558e10d951b0 2023-07-21T22:16:48.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/3875935554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 msgr2=0x558e10d9c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x558e10d8ce70 tx=0x7fb7b8012000 comp rx=0 tx=0).stop 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/3875935554 shutdown_connections 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d95ac0 0x558e10d996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d90930 0x558e10d9a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/3875935554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9c740 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/3875935554 >> 172.21.15.93:0/3875935554 conn(0x558e10d85ef0 msgr2=0x558e10d8e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/3875935554 shutdown_connections 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/3875935554 wait complete. 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- start start 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 Processor -- start 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d90930 0x558e10da1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10da05c0 con 0x558e10d951b0 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10da0700 con 0x558e10d95ac0 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e10e02260 con 0x558e10d90930 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.542+0000 7fb7dbdd8b80 10 client.0 _ll_register_callbacks cb 0x7fb7c0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44462/0 (socket says 172.21.15.93:44462) 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/1665794621 learned_addr learned my addr 172.21.15.93:0/1665794621 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d90930 0x558e10da1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7ce1f0700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d90930 msgr2=0x558e10da1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d90930 0x558e10da1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/1665794621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 msgr2=0x558e10d9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 -- 172.21.15.93:0/1665794621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b800b950 con 0x558e10d951b0 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7cd1ee700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8010010 tx=0x7fb7b8021670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd04c25492fa2d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:48.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8003880 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb7b8021890 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c00074e0 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b80039e0 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7dbdd8b80 10 client.15564 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.543+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558e10b0e2b0 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c00074e0 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7b803fcf0 con 0x558e10d951b0 2023-07-21T22:16:48.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 20 client.15564 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10e05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 4 client.15564 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e10c9d050 con 0x558e10da9f80 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.544+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10e05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7b17fa700 10 client.15564 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7cd9ef700 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10e05910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4811a801e176a5 server_cookie=d19768141009b942 in_seq=0 out_seq=0 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7b17fa700 1 -- 172.21.15.93:0/1665794621 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e10c9d050 con 0x558e10da9f80 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7b17fa700 10 client.15564 handle_command_reply: tid=0 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 client.15564 shutdown 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 msgr2=0x558e10d9f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8010010 tx=0x7fb7b8021670 comp rx=0 tx=0).stop 2023-07-21T22:16:48.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 shutdown_connections 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e10da9f80 0x558e10e05910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e10d95ac0 0x558e10d9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e10d90930 0x558e10da1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7b17fa700 0 client.15564 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 --2- 172.21.15.93:0/1665794621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e10d951b0 0x558e10d9f8f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 >> 172.21.15.93:0/1665794621 conn(0x558e10d85ef0 msgr2=0x558e10d8f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 shutdown_connections 2023-07-21T22:16:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 1 -- 172.21.15.93:0/1665794621 wait complete. 2023-07-21T22:16:48.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.545+0000 7fb7dbdd8b80 20 client.15564 trim_cache size 0 max 16384 2023-07-21T22:16:48.599 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:48.599 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "no active scrubs running", 2023-07-21T22:16:48.599 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": {} 2023-07-21T22:16:48.599 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:48.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 msgr2=0x7fb7b40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 0x7fb7b40a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8005950 tx=0x7fb7b80058e0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 msgr2=0x7fb7c819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400d980 tx=0x7fb7c400dd40 comp rx=0 tx=0).stop 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 shutdown_connections 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7c81049e0 0x7fb7c8194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7b40a1350 0x7fb7b40a3800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7c80fe9d0 0x7fb7c819a9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 --2- 172.21.15.93:0/2195617045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c80ff380 0x7fb7c819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 >> 172.21.15.93:0/2195617045 conn(0x7fb7c80fa4b0 msgr2=0x7fb7c8105830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 shutdown_connections 2023-07-21T22:16:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.546+0000 7fb7cfc53700 1 -- 172.21.15.93:0/2195617045 wait complete. 2023-07-21T22:16:48.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:16:48.603 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:16:48.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 -- 172.21.15.93:0/3663373603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1564104e20 msgr2=0x7f156410c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 --2- 172.21.15.93:0/3663373603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1564104e20 0x7f156410c9f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f155c00b0d0 tx=0x7f155c00b3e0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 -- 172.21.15.93:0/3663373603 shutdown_connections 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 --2- 172.21.15.93:0/3663373603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1564104e20 0x7f156410c9f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 --2- 172.21.15.93:0/3663373603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15640ffde0 0x7f15641001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 --2- 172.21.15.93:0/3663373603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1564100780 0x7f1564104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 -- 172.21.15.93:0/3663373603 >> 172.21.15.93:0/3663373603 conn(0x7f15640fb690 msgr2=0x7f15640fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 -- 172.21.15.93:0/3663373603 shutdown_connections 2023-07-21T22:16:48.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.683+0000 7f156b965700 1 -- 172.21.15.93:0/3663373603 wait complete. 2023-07-21T22:16:48.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 Processor -- start 2023-07-21T22:16:48.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 -- start start 2023-07-21T22:16:48.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1564100780 0x7f1564072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1564104e20 0x7f15641a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564073580 con 0x7f15640ffde0 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641a43f0 con 0x7f1564100780 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f156b965700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641a4530 con 0x7f1564104e20 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44474/0 (socket says 172.21.15.93:44474) 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 -- 172.21.15.93:0/652698025 learned_addr learned my addr 172.21.15.93:0/652698025 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1568f00700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1564100780 0x7f1564072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 -- 172.21.15.93:0/652698025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1564104e20 msgr2=0x7f15641a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1564104e20 0x7f15641a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 -- 172.21.15.93:0/652698025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1564100780 msgr2=0x7f1564072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1564100780 0x7f1564072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.684+0000 7f1569701700 1 -- 172.21.15.93:0/652698025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009d00 con 0x7f15640ffde0 2023-07-21T22:16:48.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f1569701700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f156000d980 tx=0x7f156000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560004220 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1560007640 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15641a4790 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560005610 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.685+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15641a4c70 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.686+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1564067750 con 0x7f15640ffde0 2023-07-21T22:16:48.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.687+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f1560004750 con 0x7f15640ffde0 2023-07-21T22:16:48.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.687+0000 7f155a7fc700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 0x7f15500a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:48.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.688+0000 7f1568f00700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 0x7f15500a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:48.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.688+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f15600cd210 con 0x7f15640ffde0 2023-07-21T22:16:48.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.688+0000 7f1568f00700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 0x7f15500a3840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1564074000 tx=0x7f155400a300 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:48.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.691+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f15600964d0 con 0x7f15640ffde0 2023-07-21T22:16:48.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.935+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1564073d40 con 0x7f15640ffde0 2023-07-21T22:16:48.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.935+0000 7f155a7fc700 1 -- 172.21.15.93:0/652698025 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f1560016090 con 0x7f15640ffde0 2023-07-21T22:16:48.979 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:16:48.980 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:16:48.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 msgr2=0x7f15500a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 0x7f15500a3840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1564074000 tx=0x7f155400a300 comp rx=0 tx=0).stop 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 msgr2=0x7f1564072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f156000d980 tx=0x7f156000dc90 comp rx=0 tx=0).stop 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 shutdown_connections 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1564104e20 0x7f15641a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15500a1390 0x7f15500a3840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15640ffde0 0x7f1564072810 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 --2- 172.21.15.93:0/652698025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1564100780 0x7f1564072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 >> 172.21.15.93:0/652698025 conn(0x7f15640fb690 msgr2=0x7f15640fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 shutdown_connections 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:48.938+0000 7f156b965700 1 -- 172.21.15.93:0/652698025 wait complete. 2023-07-21T22:16:48.983 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:16:48.994 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 -- 172.21.15.93:0/1871644636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f027810a070 msgr2=0x7f027810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 --2- 172.21.15.93:0/1871644636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f027810a070 0x7f027810c4b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f027000b0a0 tx=0x7f027000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 -- 172.21.15.93:0/1871644636 shutdown_connections 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 --2- 172.21.15.93:0/1871644636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f027810a070 0x7f027810c4b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 --2- 172.21.15.93:0/1871644636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0278100ca0 0x7f0278101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 --2- 172.21.15.93:0/1871644636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02781015b0 0x7f0278109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 -- 172.21.15.93:0/1871644636 >> 172.21.15.93:0/1871644636 conn(0x7f02780fa4b0 msgr2=0x7f02780fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 -- 172.21.15.93:0/1871644636 shutdown_connections 2023-07-21T22:16:49.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 -- 172.21.15.93:0/1871644636 wait complete. 2023-07-21T22:16:49.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.078+0000 7f027c980700 1 Processor -- start 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 -- start start 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0278100ca0 0x7f0278195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f027810a070 0x7f027819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819ad20 con 0x7f02781015b0 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819ae90 con 0x7f027810a070 2023-07-21T22:16:49.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027c980700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027819b050 con 0x7f0278100ca0 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44490/0 (socket says 172.21.15.93:44490) 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 -- 172.21.15.93:0/4193111556 learned_addr learned my addr 172.21.15.93:0/4193111556 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f027659c700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0278100ca0 0x7f0278195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0276d9d700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f027810a070 0x7f027819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 -- 172.21.15.93:0/4193111556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0278100ca0 msgr2=0x7f0278195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0278100ca0 0x7f0278195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 -- 172.21.15.93:0/4193111556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f027810a070 msgr2=0x7f027819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f027810a070 0x7f027819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 -- 172.21.15.93:0/4193111556 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009d00 con 0x7f02781015b0 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.079+0000 7f0275d9b700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d950 tx=0x7f026c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.080+0000 7f02677fe700 1 -- 172.21.15.93:0/4193111556 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00f890 con 0x7f02781015b0 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.080+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027819b330 con 0x7f02781015b0 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.080+0000 7f02677fe700 1 -- 172.21.15.93:0/4193111556 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026c010460 con 0x7f02781015b0 2023-07-21T22:16:49.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.080+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781a2b70 con 0x7f02781015b0 2023-07-21T22:16:49.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.080+0000 7f02677fe700 1 -- 172.21.15.93:0/4193111556 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c01b650 con 0x7f02781015b0 2023-07-21T22:16:49.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.082+0000 7f02677fe700 1 -- 172.21.15.93:0/4193111556 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f026c0108d0 con 0x7f02781015b0 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.083+0000 7f02677fe700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.083+0000 7f027659c700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.083+0000 7f02677fe700 1 -- 172.21.15.93:0/4193111556 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f026c0ccf10 con 0x7f02781015b0 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.083+0000 7f0288b05b80 1 Processor -- start 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.083+0000 7f0288b05b80 1 -- start start 2023-07-21T22:16:49.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f027659c700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 0x7f02600a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0268005950 tx=0x7f026800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f8a7cb0 con 0x56246f9a2930 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f8faa40 con 0x56246f9a71b0 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0288b05b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f8f3ed0 con 0x56246f9a7ac0 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44500/0 (socket says 172.21.15.93:44500) 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 -- 172.21.15.93:0/3975555779 learned_addr learned my addr 172.21.15.93:0/3975555779 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0275d9b700 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f027659c700 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 -- 172.21.15.93:0/3975555779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 msgr2=0x56246f9a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 -- 172.21.15.93:0/3975555779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 msgr2=0x56246f9a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 -- 172.21.15.93:0/3975555779 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56246f721de0 con 0x56246f9a2930 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0276d9d700 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0270015000 tx=0x7f027000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23ea416f415a59d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0264ff9700 1 -- 172.21.15.93:0/3975555779 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000e040 con 0x56246f9a2930 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.084+0000 7f0264ff9700 1 -- 172.21.15.93:0/3975555779 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0270009350 con 0x56246f9a2930 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0264ff9700 1 -- 172.21.15.93:0/3975555779 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027001c6a0 con 0x56246f9a2930 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 -- 172.21.15.93:0/3975555779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 msgr2=0x56246f9a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0270015000 tx=0x7f027000cde0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 -- 172.21.15.93:0/3975555779 shutdown_connections 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9a2da0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3975555779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 -- 172.21.15.93:0/3975555779 >> 172.21.15.93:0/3975555779 conn(0x56246f997ef0 msgr2=0x56246f99d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.085+0000 7f0288b05b80 1 -- 172.21.15.93:0/3975555779 shutdown_connections 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 -- 172.21.15.93:0/3975555779 wait complete. 2023-07-21T22:16:49.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 -- start start 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 Processor -- start 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f893ac0 con 0x56246f9a2930 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f8937c0 con 0x56246f9a71b0 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f777920 con 0x56246f9a7ac0 2023-07-21T22:16:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f0288b05b80 10 client.0 _ll_register_callbacks cb 0x56246f891760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44502/0 (socket says 172.21.15.93:44502) 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.086+0000 7f027659c700 1 -- 172.21.15.93:0/13709661 learned_addr learned my addr 172.21.15.93:0/13709661 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0276d9d700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0275d9b700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 -- 172.21.15.93:0/13709661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 msgr2=0x56246f9ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 -- 172.21.15.93:0/13709661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 msgr2=0x56246f9abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 -- 172.21.15.93:0/13709661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027810d170 con 0x56246f9a2930 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0276d9d700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0275d9b700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f027659c700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02680116c0 tx=0x7f02680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61727bb7c06855be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268004a30 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0268017070 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56246f84d4f0 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0268010e80 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 10 client.15582 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56246f70ac60 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56246f84d4f0 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f026801f6d0 con 0x56246f9a2930 2023-07-21T22:16:49.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 10 client.15582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 20 client.15582 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246f9bc380 0x56246f9be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.087+0000 7f0288b05b80 4 client.15582 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.088+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56246f8eceb0 con 0x56246f9bc380 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.088+0000 7f0275d9b700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246f9bc380 0x56246f9be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.088+0000 7f025dffb700 10 client.15582 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:49.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.088+0000 7f0275d9b700 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246f9bc380 0x56246f9be790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d103a3b247d50ffc server_cookie=e26ec4f8a2d3d3bb in_seq=0 out_seq=0 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f025dffb700 1 -- 172.21.15.93:0/13709661 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56246f8eceb0 con 0x56246f9bc380 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f025dffb700 10 client.15582 handle_command_reply: tid=0 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 client.15582 shutdown 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 msgr2=0x56246f9aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f02680116c0 tx=0x7f02680047c0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 shutdown_connections 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246f9bc380 0x56246f9be790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f025dffb700 0 client.15582 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9aeb70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 --2- 172.21.15.93:0/13709661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 >> 172.21.15.93:0/13709661 conn(0x56246f997ef0 msgr2=0x56246f99d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 shutdown_connections 2023-07-21T22:16:49.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 1 -- 172.21.15.93:0/13709661 wait complete. 2023-07-21T22:16:49.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.089+0000 7f0288b05b80 20 client.15582 trim_cache size 0 max 16384 2023-07-21T22:16:49.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 Processor -- start 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 -- start start 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f9a36d0 con 0x56246f9a2930 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f99a010 con 0x56246f9a71b0 2023-07-21T22:16:49.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.102+0000 7f0288b05b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f9a34a0 con 0x56246f9a7ac0 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44510/0 (socket says 172.21.15.93:44510) 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 -- 172.21.15.93:0/2231370954 learned_addr learned my addr 172.21.15.93:0/2231370954 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f0275d9b700 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f0276d9d700 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 -- 172.21.15.93:0/2231370954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 msgr2=0x56246f9ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 -- 172.21.15.93:0/2231370954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 msgr2=0x56246f9b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 -- 172.21.15.93:0/2231370954 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56246f748250 con 0x56246f9a2930 2023-07-21T22:16:49.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f027659c700 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f026801fab0 tx=0x7f026800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27f50483ee512be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f02657fa700 1 -- 172.21.15.93:0/2231370954 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026801f840 con 0x56246f9a2930 2023-07-21T22:16:49.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f02657fa700 1 -- 172.21.15.93:0/2231370954 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0268015070 con 0x56246f9a2930 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.103+0000 7f02657fa700 1 -- 172.21.15.93:0/2231370954 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268039440 con 0x56246f9a2930 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- 172.21.15.93:0/2231370954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 msgr2=0x56246f9abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f026801fab0 tx=0x7f026800b480 comp rx=0 tx=0).stop 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- 172.21.15.93:0/2231370954 shutdown_connections 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a7ac0 0x56246f9ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9abd30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 --2- 172.21.15.93:0/2231370954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a71b0 0x56246f9b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- 172.21.15.93:0/2231370954 >> 172.21.15.93:0/2231370954 conn(0x56246f997ef0 msgr2=0x56246f9ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- 172.21.15.93:0/2231370954 shutdown_connections 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- 172.21.15.93:0/2231370954 wait complete. 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 -- start start 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.104+0000 7f0288b05b80 1 Processor -- start 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a71b0 0x56246f9b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a7ac0 0x56246f9b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f9b1d00 con 0x56246f9a2930 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f9b1e70 con 0x56246f9a7ac0 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246f9b2030 con 0x56246f9a71b0 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0288b05b80 10 client.0 _ll_register_callbacks cb 0x7f0268010ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44516/0 (socket says 172.21.15.93:44516) 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 -- 172.21.15.93:0/3734048984 learned_addr learned my addr 172.21.15.93:0/3734048984 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0275d9b700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a71b0 0x56246f9b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f0276d9d700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a7ac0 0x56246f9b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 -- 172.21.15.93:0/3734048984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a71b0 msgr2=0x56246f9b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a71b0 0x56246f9b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 -- 172.21.15.93:0/3734048984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a7ac0 msgr2=0x56246f9b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a7ac0 0x56246f9b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 -- 172.21.15.93:0/3734048984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56246f70ac60 con 0x56246f9a2930 2023-07-21T22:16:49.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f027659c700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0268010ac0 tx=0x7f026800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bfcc0ba4dd41f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.105+0000 7f025dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268030460 con 0x56246f9a2930 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0268017070 con 0x56246f9a2930 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0270009350 con 0x56246f9a2930 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02680305c0 con 0x56246f9a2930 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 10 client.15588 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56246f7202b0 con 0x56246f9a2930 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0270009350 con 0x56246f9a2930 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0268039440 con 0x56246f9a2930 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 10 client.15588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 20 client.15588 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246fa11150 0x56246fa13560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 4 client.15588 mds_command: new command op to 14589 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56246f8af050 con 0x56246fa11150 2023-07-21T22:16:49.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.106+0000 7f0275d9b700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246fa11150 0x56246fa13560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f025dffb700 10 client.15588 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0275d9b700 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246fa11150 0x56246fa13560 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f92c9594d7724dcc server_cookie=d8e9a329420ad57c in_seq=0 out_seq=0 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f025dffb700 1 -- 172.21.15.93:0/3734048984 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56246f8af050 con 0x56246fa11150 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f025dffb700 10 client.15588 handle_command_reply: tid=0 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 client.15588 shutdown 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 msgr2=0x56246f9b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0268010ac0 tx=0x7f026800ba40 comp rx=0 tx=0).stop 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 shutdown_connections 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246f9a7ac0 0x56246f9b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56246f9a2930 0x56246f9b3180 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x56246fa11150 0x56246fa13560 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 --2- 172.21.15.93:0/3734048984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56246f9a71b0 0x56246f9b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f025dffb700 0 client.15588 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:16:49.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 >> 172.21.15.93:0/3734048984 conn(0x56246f997ef0 msgr2=0x56246f9a5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 shutdown_connections 2023-07-21T22:16:49.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 1 -- 172.21.15.93:0/3734048984 wait complete. 2023-07-21T22:16:49.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.107+0000 7f0288b05b80 20 client.15588 trim_cache size 0 max 16384 2023-07-21T22:16:49.161 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:16:49.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 msgr2=0x7f02600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 0x7f02600a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0268005950 tx=0x7f026800b410 comp rx=0 tx=0).stop 2023-07-21T22:16:49.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 msgr2=0x7f0278196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f026c00d950 tx=0x7f026c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 shutdown_connections 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f027810a070 0x7f027819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0278100ca0 0x7f0278195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f02600a1350 0x7f02600a3800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 --2- 172.21.15.93:0/4193111556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02781015b0 0x7f0278196280 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 >> 172.21.15.93:0/4193111556 conn(0x7f02780fa4b0 msgr2=0x7f027810c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 shutdown_connections 2023-07-21T22:16:49.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.108+0000 7f027c980700 1 -- 172.21.15.93:0/4193111556 wait complete. 2023-07-21T22:16:49.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:16:49.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/2911486065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f540ff390 msgr2=0x7f0f5410b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/2911486065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f540ff390 0x7f0f5410b820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00b0a0 tx=0x7f0f4c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/2911486065 shutdown_connections 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/2911486065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f540ff390 0x7f0f5410b820 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/2911486065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f541049f0 0x7f0f54104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/2911486065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 0x7f0f540fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/2911486065 >> 172.21.15.93:0/2911486065 conn(0x7f0f540fa4e0 msgr2=0x7f0f540fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/2911486065 shutdown_connections 2023-07-21T22:16:49.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.250+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/2911486065 wait complete. 2023-07-21T22:16:49.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 Processor -- start 2023-07-21T22:16:49.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 -- start start 2023-07-21T22:16:49.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 0x7f0f54196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f540ff390 0x7f0f541970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541979b0 con 0x7f0f541049f0 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541a2120 con 0x7f0f540ff390 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5c0f0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541a2290 con 0x7f0f540fe9e0 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44532/0 (socket says 172.21.15.93:44532) 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 -- 172.21.15.93:0/4174404319 learned_addr learned my addr 172.21.15.93:0/4174404319 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5968b700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f540ff390 0x7f0f541970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 0x7f0f54196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 -- 172.21.15.93:0/4174404319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 msgr2=0x7f0f54196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 0x7f0f54196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 -- 172.21.15.93:0/4174404319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f540ff390 msgr2=0x7f0f541970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f540ff390 0x7f0f541970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.251+0000 7f0f5a68d700 1 -- 172.21.15.93:0/4174404319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c009d00 con 0x7f0f541049f0 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00b780 tx=0x7f0f4c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f4affd700 1 -- 172.21.15.93:0/4174404319 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00e040 con 0x7f0f541049f0 2023-07-21T22:16:49.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f541a2510 con 0x7f0f541049f0 2023-07-21T22:16:49.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f4affd700 1 -- 172.21.15.93:0/4174404319 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f4c004e60 con 0x7f0f541049f0 2023-07-21T22:16:49.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f4affd700 1 -- 172.21.15.93:0/4174404319 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c005280 con 0x7f0f541049f0 2023-07-21T22:16:49.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.252+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f541a2990 con 0x7f0f541049f0 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.254+0000 7f0f4affd700 1 -- 172.21.15.93:0/4174404319 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f0f4c01d470 con 0x7f0f541049f0 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.254+0000 7f0f4affd700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 0x7f0f400a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.255+0000 7f0f4affd700 1 -- 172.21.15.93:0/4174404319 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0f4c0ce520 con 0x7f0f541049f0 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.255+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 0x7f0f400a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.255+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 0x7f0f400a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f540ffb20 tx=0x7f0f50009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.255+0000 7f0f68275b80 1 Processor -- start 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 -- start start 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f00930 0x561ac8f00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8e05cb0 con 0x561ac8f051b0 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8e58a40 con 0x561ac8f05ac0 2023-07-21T22:16:49.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f68275b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8e51ed0 con 0x561ac8f00930 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44536/0 (socket says 172.21.15.93:44536) 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2389301054 learned_addr learned my addr 172.21.15.93:0/2389301054 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f5968b700 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f00930 0x561ac8f00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2389301054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f00930 msgr2=0x561ac8f00da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f00930 0x561ac8f00da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2389301054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 msgr2=0x561ac8f00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f00350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2389301054 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ac8c7fde0 con 0x561ac8f051b0 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.256+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000fbe0 tx=0x7f0f50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30edd217d7454f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f3ffff700 1 -- 172.21.15.93:0/2389301054 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50011ad0 con 0x561ac8f051b0 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f3ffff700 1 -- 172.21.15.93:0/2389301054 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f50011c30 con 0x561ac8f051b0 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f3ffff700 1 -- 172.21.15.93:0/2389301054 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5001d610 con 0x561ac8f051b0 2023-07-21T22:16:49.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 -- 172.21.15.93:0/2389301054 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 msgr2=0x561ac8f05580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000fbe0 tx=0x7f0f50012000 comp rx=0 tx=0).stop 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 -- 172.21.15.93:0/2389301054 shutdown_connections 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f00930 0x561ac8f00da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f05580 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2389301054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 -- 172.21.15.93:0/2389301054 >> 172.21.15.93:0/2389301054 conn(0x561ac8ef5ef0 msgr2=0x561ac8efb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.257+0000 7f0f68275b80 1 -- 172.21.15.93:0/2389301054 shutdown_connections 2023-07-21T22:16:49.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 -- 172.21.15.93:0/2389301054 wait complete. 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 -- start start 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 Processor -- start 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f00930 0x561ac8f0cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8f0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8df1ac0 con 0x561ac8f051b0 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8df17c0 con 0x561ac8f00930 2023-07-21T22:16:49.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8cd5920 con 0x561ac8f05ac0 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f68275b80 10 client.0 _ll_register_callbacks cb 0x561ac8def760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f5968b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.258+0000 7f0f5968b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44544/0 (socket says 172.21.15.93:44544) 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 -- 172.21.15.93:0/2518030378 learned_addr learned my addr 172.21.15.93:0/2518030378 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f00930 0x561ac8f0cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8f0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 -- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 msgr2=0x561ac8f0a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8f0a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 -- 172.21.15.93:0/2518030378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f00930 msgr2=0x561ac8f0cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f00930 0x561ac8f0cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 -- 172.21.15.93:0/2518030378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5000f890 con 0x561ac8f051b0 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f5968b700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400bf10 tx=0x7f0f44012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6963af8079cabc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004500 con 0x561ac8f051b0 2023-07-21T22:16:49.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0f44003680 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ac8dab4f0 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f44003840 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f68275b80 10 client.15600 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.259+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561ac8c68c60 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ac8dab4f0 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f44024b90 con 0x561ac8f051b0 2023-07-21T22:16:49.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 10 client.15600 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 20 client.15600 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f1a510 0x561ac8f1c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 4 client.15600 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ac8e4aeb0 con 0x561ac8f1a510 2023-07-21T22:16:49.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.260+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f1a510 0x561ac8f1c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f3d7fa700 10 client.15600 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f1a510 0x561ac8f1c920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc030f1d09a19723 server_cookie=3ce32c5f596073dc in_seq=0 out_seq=0 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/2518030378 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561ac8e4aeb0 con 0x561ac8f1a510 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f3d7fa700 10 client.15600 handle_command_reply: tid=0 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 client.15600 shutdown 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 msgr2=0x561ac8f09e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400bf10 tx=0x7f0f44012730 comp rx=0 tx=0).stop 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 shutdown_connections 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8f0a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f00930 0x561ac8f0cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f1a510 0x561ac8f1c920 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2518030378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f051b0 0x561ac8f09e80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f3d7fa700 0 client.15600 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:49.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 >> 172.21.15.93:0/2518030378 conn(0x561ac8ef5ef0 msgr2=0x561ac8efb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.261+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 shutdown_connections 2023-07-21T22:16:49.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.262+0000 7f0f68275b80 1 -- 172.21.15.93:0/2518030378 wait complete. 2023-07-21T22:16:49.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.262+0000 7f0f68275b80 20 client.15600 trim_cache size 0 max 16384 2023-07-21T22:16:49.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 Processor -- start 2023-07-21T22:16:49.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 -- start start 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f051b0 0x561ac8f0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8efea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f01700 con 0x561ac8f00930 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f07ea0 con 0x561ac8f051b0 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f68275b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f04cf0 con 0x561ac8f05ac0 2023-07-21T22:16:49.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.275+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44550/0 (socket says 172.21.15.93:44550) 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2260562162 learned_addr learned my addr 172.21.15.93:0/2260562162 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f5968b700 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f051b0 0x561ac8f0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8efea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2260562162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 msgr2=0x561ac8efea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8efea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2260562162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f051b0 msgr2=0x561ac8f0cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f051b0 0x561ac8f0cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 -- 172.21.15.93:0/2260562162 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ac8ca6250 con 0x561ac8f00930 2023-07-21T22:16:49.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0f50011720 tx=0x7f0f5000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe5eb2f97fc789 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f48ff9700 1 -- 172.21.15.93:0/2260562162 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5002d040 con 0x561ac8f00930 2023-07-21T22:16:49.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f48ff9700 1 -- 172.21.15.93:0/2260562162 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f5002e040 con 0x561ac8f00930 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.276+0000 7f0f48ff9700 1 -- 172.21.15.93:0/2260562162 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5001e450 con 0x561ac8f00930 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- 172.21.15.93:0/2260562162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 msgr2=0x561ac8f0c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0f50011720 tx=0x7f0f5000ffd0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- 172.21.15.93:0/2260562162 shutdown_connections 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f05ac0 0x561ac8efea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f0c7a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 --2- 172.21.15.93:0/2260562162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f051b0 0x561ac8f0cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- 172.21.15.93:0/2260562162 >> 172.21.15.93:0/2260562162 conn(0x561ac8ef5ef0 msgr2=0x561ac8f21280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- 172.21.15.93:0/2260562162 shutdown_connections 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- 172.21.15.93:0/2260562162 wait complete. 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 -- start start 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.277+0000 7f0f68275b80 1 Processor -- start 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f051b0 0x561ac8f11560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f0f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f0fc50 con 0x561ac8f00930 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f0fdc0 con 0x561ac8f05ac0 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac8f0ff80 con 0x561ac8f051b0 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f68275b80 10 client.0 _ll_register_callbacks cb 0x7f0f4400b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44556/0 (socket says 172.21.15.93:44556) 2023-07-21T22:16:49.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 -- 172.21.15.93:0/914465254 learned_addr learned my addr 172.21.15.93:0/914465254 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f5968b700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f051b0 0x561ac8f11560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f0f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 -- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f051b0 msgr2=0x561ac8f11560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f051b0 0x561ac8f11560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 -- 172.21.15.93:0/914465254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 msgr2=0x561ac8f0f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f0f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 -- 172.21.15.93:0/914465254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ac8c7fde0 con 0x561ac8f00930 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f5a68d700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f0f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.278+0000 7f0f59e8c700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004770 tx=0x7f0f50009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ff95342f3cc0fc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50034bd0 con 0x561ac8f00930 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0f50034d30 con 0x561ac8f00930 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f50011c30 con 0x561ac8f00930 2023-07-21T22:16:49.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f5001e450 con 0x561ac8f00930 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 10 client.15606 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561ac8c7e2b0 con 0x561ac8f00930 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f50011c30 con 0x561ac8f00930 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f50027710 con 0x561ac8f00930 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 10 client.15606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 20 client.15606 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f6f0f0 0x561ac8f714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 4 client.15606 mds_command: new command op to 24448 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.279+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ac8e0d050 con 0x561ac8f6f0f0 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f5968b700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f6f0f0 0x561ac8f714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f3d7fa700 10 client.15606 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f5968b700 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f6f0f0 0x561ac8f714d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c48322466875144b server_cookie=d55eb410a387e99f in_seq=0 out_seq=0 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f3d7fa700 1 -- 172.21.15.93:0/914465254 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ac8e0d050 con 0x561ac8f6f0f0 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f3d7fa700 10 client.15606 handle_command_reply: tid=0 2023-07-21T22:16:49.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f68275b80 1 client.15606 shutdown 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 msgr2=0x561ac8f11020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f50004770 tx=0x7f0f50009890 comp rx=0 tx=0).stop 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 shutdown_connections 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.280+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561ac8f05ac0 0x561ac8f0f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x561ac8f6f0f0 0x561ac8f714d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ac8f00930 0x561ac8f11020 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f3d7fa700 0 client.15606 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 --2- 172.21.15.93:0/914465254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561ac8f051b0 0x561ac8f11560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 >> 172.21.15.93:0/914465254 conn(0x561ac8ef5ef0 msgr2=0x561ac8efea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 shutdown_connections 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 1 -- 172.21.15.93:0/914465254 wait complete. 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f68275b80 20 client.15606 trim_cache size 0 max 16384 2023-07-21T22:16:49.334 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:16:49.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 msgr2=0x7f0f400a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 0x7f0f400a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f540ffb20 tx=0x7f0f50009260 comp rx=0 tx=0).stop 2023-07-21T22:16:49.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 msgr2=0x7f0f541a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00b780 tx=0x7f0f4c00be10 comp rx=0 tx=0).stop 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 shutdown_connections 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f400a1350 0x7f0f400a3800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f541049f0 0x7f0f541a1be0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f540fe9e0 0x7f0f54196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 --2- 172.21.15.93:0/4174404319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f540ff390 0x7f0f541970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 >> 172.21.15.93:0/4174404319 conn(0x7f0f540fa4e0 msgr2=0x7f0f540fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 shutdown_connections 2023-07-21T22:16:49.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.281+0000 7f0f5c0f0700 1 -- 172.21.15.93:0/4174404319 wait complete. 2023-07-21T22:16:49.337 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:16:49.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/633280166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 msgr2=0x7fbcbc0fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/633280166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc0fe840 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0099c0 tx=0x7fbcac009cd0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/633280166 shutdown_connections 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/633280166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 0x7fbcbc10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/633280166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc0fe840 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/633280166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/633280166 >> 172.21.15.93:0/633280166 conn(0x7fbcbc076200 msgr2=0x7fbcbc076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/633280166 shutdown_connections 2023-07-21T22:16:49.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.423+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/633280166 wait complete. 2023-07-21T22:16:49.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.424+0000 7fbcc1d9b700 1 Processor -- start 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.424+0000 7fbcc1d9b700 1 -- start start 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.424+0000 7fbcc1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 0x7fbcbc193830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc193ea0 con 0x7fbcbc0fe470 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc194010 con 0x7fbcbc0fee10 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc1941d0 con 0x7fbcbc103430 2023-07-21T22:16:49.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44570/0 (socket says 172.21.15.93:44570) 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3298694432 learned_addr learned my addr 172.21.15.93:0/3298694432 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 0x7fbcbc193830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3298694432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 msgr2=0x7fbcbc193830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 0x7fbcbc193830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3298694432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 msgr2=0x7fbcbc199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3298694432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0095f0 con 0x7fbcbc0fe470 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b580 tx=0x7fbcac00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcb8ff9700 1 -- 172.21.15.93:0/3298694432 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac01d070 con 0x7fbcbc0fe470 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcb8ff9700 1 -- 172.21.15.93:0/3298694432 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac00fbf0 con 0x7fbcbc0fe470 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc194450 con 0x7fbcbc0fe470 2023-07-21T22:16:49.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcb8ff9700 1 -- 172.21.15.93:0/3298694432 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0173f0 con 0x7fbcbc0fe470 2023-07-21T22:16:49.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.425+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc1a07b0 con 0x7fbcbc0fe470 2023-07-21T22:16:49.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.427+0000 7fbcb8ff9700 1 -- 172.21.15.93:0/3298694432 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fbcac022470 con 0x7fbcbc0fe470 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.428+0000 7fbcb8ff9700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 0x7fbca40a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.428+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 0x7fbca40a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.428+0000 7fbcb8ff9700 1 -- 172.21.15.93:0/3298694432 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbcac0ce010 con 0x7fbcbc0fe470 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 0x7fbca40a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc104980 tx=0x7fbcb000a370 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 Processor -- start 2023-07-21T22:16:49.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 -- start start 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e4ac0 0x5633926df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633925e4cb0 con 0x5633926e41b0 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563392637a40 con 0x5633926e4ac0 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563392630ed0 con 0x5633926df930 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44578/0 (socket says 172.21.15.93:44578) 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.429+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/838195665 learned_addr learned my addr 172.21.15.93:0/838195665 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e4ac0 0x5633926df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/838195665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 msgr2=0x5633926dfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926dfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/838195665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e4ac0 msgr2=0x5633926df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e4ac0 0x5633926df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/838195665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339245ede0 con 0x5633926e41b0 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00ffd0 tx=0x7fbcac097470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d65169779c8d091 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbca1ffb700 1 -- 172.21.15.93:0/838195665 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac09f050 con 0x5633926e41b0 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbca1ffb700 1 -- 172.21.15.93:0/838195665 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac0a33e0 con 0x5633926e41b0 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.430+0000 7fbca1ffb700 1 -- 172.21.15.93:0/838195665 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac09b4e0 con 0x5633926e41b0 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- 172.21.15.93:0/838195665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 msgr2=0x5633926e4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00ffd0 tx=0x7fbcac097470 comp rx=0 tx=0).stop 2023-07-21T22:16:49.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- 172.21.15.93:0/838195665 shutdown_connections 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926dfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e41b0 0x5633926e4580 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/838195665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e4ac0 0x5633926df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- 172.21.15.93:0/838195665 >> 172.21.15.93:0/838195665 conn(0x5633926d4ef0 msgr2=0x5633926da830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- 172.21.15.93:0/838195665 shutdown_connections 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- 172.21.15.93:0/838195665 wait complete. 2023-07-21T22:16:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.431+0000 7fbccdf20b80 1 -- start start 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 Processor -- start 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633925d0ac0 con 0x5633926e4ac0 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633925d07c0 con 0x5633926e41b0 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633924b4920 con 0x5633926df930 2023-07-21T22:16:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbccdf20b80 10 client.0 _ll_register_callbacks cb 0x5633925ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:59896/0 (socket says 172.21.15.93:59896) 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/4254537066 learned_addr learned my addr 172.21.15.93:0/4254537066 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 -- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 msgr2=0x5633926ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 -- 172.21.15.93:0/4254537066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 msgr2=0x5633926e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbbfff700 1 -- 172.21.15.93:0/4254537066 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc1a07b0 con 0x5633926e4ac0 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.432+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e9280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4007a80 tx=0x7fbcb4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663615eb390dccd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400e070 con 0x5633926e4ac0 2023-07-21T22:16:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbcb4004db0 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339258a4f0 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb40128c0 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 10 client.15618 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563392447c60 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56339258a4f0 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbcb4014040 con 0x5633926e4ac0 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 10 client.15618 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 20 client.15618 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5633926f90e0 0x5633926fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 4 client.15618 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.433+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563392629eb0 con 0x5633926f90e0 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.434+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5633926f90e0 0x5633926fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.434+0000 7fbc9affd700 10 client.15618 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:49.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.434+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5633926f90e0 0x5633926fb4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f3beabe10737363 server_cookie=4148298fcc21bb8a in_seq=0 out_seq=0 2023-07-21T22:16:49.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbc9affd700 1 -- 172.21.15.93:0/4254537066 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563392629eb0 con 0x5633926f90e0 2023-07-21T22:16:49.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbc9affd700 10 client.15618 handle_command_reply: tid=0 2023-07-21T22:16:49.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 client.15618 shutdown 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 msgr2=0x5633926e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e9280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4007a80 tx=0x7fbcb4007d90 comp rx=0 tx=0).stop 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 shutdown_connections 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e9280 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5633926f90e0 0x5633926fb4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/4254537066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbc9affd700 0 client.15618 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:49.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 >> 172.21.15.93:0/4254537066 conn(0x5633926d4ef0 msgr2=0x5633926da3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 shutdown_connections 2023-07-21T22:16:49.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 1 -- 172.21.15.93:0/4254537066 wait complete. 2023-07-21T22:16:49.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.435+0000 7fbccdf20b80 20 client.15618 trim_cache size 0 max 16384 2023-07-21T22:16:49.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.448+0000 7fbccdf20b80 1 Processor -- start 2023-07-21T22:16:49.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.448+0000 7fbccdf20b80 1 -- start start 2023-07-21T22:16:49.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633926d9680 con 0x5633926e4ac0 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633926f6840 con 0x5633926e41b0 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633926d7010 con 0x5633926df930 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44596/0 (socket says 172.21.15.93:44596) 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 -- 172.21.15.93:0/1027307818 learned_addr learned my addr 172.21.15.93:0/1027307818 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 -- 172.21.15.93:0/1027307818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 msgr2=0x5633926f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 -- 172.21.15.93:0/1027307818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 msgr2=0x5633926e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 -- 172.21.15.93:0/1027307818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563392700120 con 0x5633926e4ac0 2023-07-21T22:16:49.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4016cf0 tx=0x7fbcb40125c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34582dc840eb8e73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbca27fc700 1 -- 172.21.15.93:0/1027307818 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400e050 con 0x5633926e4ac0 2023-07-21T22:16:49.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbca27fc700 1 -- 172.21.15.93:0/1027307818 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb400f040 con 0x5633926e4ac0 2023-07-21T22:16:49.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.449+0000 7fbca27fc700 1 -- 172.21.15.93:0/1027307818 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4015560 con 0x5633926e4ac0 2023-07-21T22:16:49.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 -- 172.21.15.93:0/1027307818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 msgr2=0x5633926e6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4016cf0 tx=0x7fbcb40125c0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 -- 172.21.15.93:0/1027307818 shutdown_connections 2023-07-21T22:16:49.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926e4ac0 0x5633926e6540 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926df930 0x5633926f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/1027307818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 -- 172.21.15.93:0/1027307818 >> 172.21.15.93:0/1027307818 conn(0x5633926d4ef0 msgr2=0x5633926dd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 -- 172.21.15.93:0/1027307818 shutdown_connections 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.450+0000 7fbccdf20b80 1 -- 172.21.15.93:0/1027307818 wait complete. 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 -- start start 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 Processor -- start 2023-07-21T22:16:49.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926e4ac0 0x563392744f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563392745490 con 0x5633926df930 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563392745600 con 0x5633926e41b0 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633927457c0 con 0x5633926e4ac0 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbccdf20b80 10 client.0 _ll_register_callbacks cb 0x7fbcb4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbcbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44598/0 (socket says 172.21.15.93:44598) 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3321192112 learned_addr learned my addr 172.21.15.93:0/3321192112 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.451+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbbfff700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926e4ac0 0x563392744f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926e4ac0 msgr2=0x563392744f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926e4ac0 0x563392744f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3321192112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 msgr2=0x5633926f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 -- 172.21.15.93:0/3321192112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563392447c60 con 0x5633926df930 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbcbb7fe700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000c00 tx=0x7fbcac09ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c0d97c6f0c5a16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbc9affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0cb600 con 0x5633926df930 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fbcac09a510 con 0x5633926df930 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcac0a33e0 con 0x5633926df930 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcac09a980 con 0x5633926df930 2023-07-21T22:16:49.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbccdf20b80 10 client.15624 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563392485250 con 0x5633926df930 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.452+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcac0a33e0 con 0x5633926df930 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbcac0a0050 con 0x5633926df930 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 10 client.15624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 20 client.15624 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x56339274fbd0 0x563392751fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 4 client.15624 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633925ec050 con 0x56339274fbd0 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x56339274fbd0 0x563392751fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbc9affd700 10 client.15624 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.453+0000 7fbcbaffd700 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x56339274fbd0 0x563392751fe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=802c94565ccd31e0 server_cookie=ca6f4acf77772e9e in_seq=0 out_seq=0 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbc9affd700 1 -- 172.21.15.93:0/3321192112 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633925ec050 con 0x56339274fbd0 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbc9affd700 10 client.15624 handle_command_reply: tid=0 2023-07-21T22:16:49.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 client.15624 shutdown 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 msgr2=0x5633927487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbcac000c00 tx=0x7fbcac09ac10 comp rx=0 tx=0).stop 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 shutdown_connections 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x56339274fbd0 0x563392751fe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5633926e4ac0 0x563392744f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5633926df930 0x5633927487d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 --2- 172.21.15.93:0/3321192112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633926e41b0 0x5633926f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 >> 172.21.15.93:0/3321192112 conn(0x5633926d4ef0 msgr2=0x5633926e74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbc9affd700 0 client.15624 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 shutdown_connections 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 1 -- 172.21.15.93:0/3321192112 wait complete. 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbccdf20b80 20 client.15624 trim_cache size 0 max 16384 2023-07-21T22:16:49.507 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:16:49.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 msgr2=0x7fbca40a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 0x7fbca40a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc104980 tx=0x7fbcb000a370 comp rx=0 tx=0).stop 2023-07-21T22:16:49.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 msgr2=0x7fbcbc1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b580 tx=0x7fbcac00bfd0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 shutdown_connections 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.454+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbcbc103430 0x7fbcbc193830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbca40a13d0 0x7fbca40a3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcbc0fe470 0x7fbcbc1997c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 --2- 172.21.15.93:0/3298694432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcbc0fee10 0x7fbcbc199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 >> 172.21.15.93:0/3298694432 conn(0x7fbcbc076200 msgr2=0x7fbcbc0fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 shutdown_connections 2023-07-21T22:16:49.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.455+0000 7fbcc1d9b700 1 -- 172.21.15.93:0/3298694432 wait complete. 2023-07-21T22:16:49.511 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:16:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:49 smithi194 ceph-mon[98288]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2023-07-21T22:16:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:49 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/652698025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:49 smithi134 ceph-mon[97873]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2023-07-21T22:16:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:49 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/652698025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:49.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 -- 172.21.15.93:0/2268330721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa78010a040 msgr2=0x7fa78010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 --2- 172.21.15.93:0/2268330721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa78010a040 0x7fa78010c4b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa77800b0a0 tx=0x7fa77800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 -- 172.21.15.93:0/2268330721 shutdown_connections 2023-07-21T22:16:49.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 --2- 172.21.15.93:0/2268330721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa78010a040 0x7fa78010c4b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 --2- 172.21.15.93:0/2268330721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa780100d00 0x7fa7801010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 --2- 172.21.15.93:0/2268330721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa780109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 -- 172.21.15.93:0/2268330721 >> 172.21.15.93:0/2268330721 conn(0x7fa7800fa4b0 msgr2=0x7fa7800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 -- 172.21.15.93:0/2268330721 shutdown_connections 2023-07-21T22:16:49.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.593+0000 7fa7878fe700 1 -- 172.21.15.93:0/2268330721 wait complete. 2023-07-21T22:16:49.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 Processor -- start 2023-07-21T22:16:49.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 -- start start 2023-07-21T22:16:49.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa7801962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa78010a040 0x7fa78019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019ad40 con 0x7fa780100d00 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.594+0000 7fa7878fe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019aeb0 con 0x7fa780101610 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7878fe700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019b070 con 0x7fa78010a040 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44612/0 (socket says 172.21.15.93:44612) 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 -- 172.21.15.93:0/656233775 learned_addr learned my addr 172.21.15.93:0/656233775 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa785e9b700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa78010a040 0x7fa78019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa784e99700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa7801962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 -- 172.21.15.93:0/656233775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa78010a040 msgr2=0x7fa78019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa78010a040 0x7fa78019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 -- 172.21.15.93:0/656233775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 msgr2=0x7fa7801962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa7801962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 -- 172.21.15.93:0/656233775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009d00 con 0x7fa780100d00 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa784e99700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa7801962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa78569a700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9a0 tx=0x7fa77c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7767fc700 1 -- 172.21.15.93:0/656233775 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c009790 con 0x7fa780100d00 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78019b350 con 0x7fa780100d00 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7767fc700 1 -- 172.21.15.93:0/656233775 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c00cbd0 con 0x7fa780100d00 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7767fc700 1 -- 172.21.15.93:0/656233775 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c018680 con 0x7fa780100d00 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.595+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801a2b90 con 0x7fa780100d00 2023-07-21T22:16:49.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.598+0000 7fa7767fc700 1 -- 172.21.15.93:0/656233775 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fa77c00cd40 con 0x7fa780100d00 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 Processor -- start 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 -- start start 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa7767fc700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 0x7fa76c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd589350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa785e9b700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 0x7fa76c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd48ecb0 con 0x5614cd58e1b0 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd4e1a40 con 0x5614cd58eac0 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa793a83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd4daed0 con 0x5614cd589930 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa7767fc700 1 -- 172.21.15.93:0/656233775 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa77c014070 con 0x7fa780100d00 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa78569a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa78569a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44620/0 (socket says 172.21.15.93:44620) 2023-07-21T22:16:49.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa78569a700 1 -- 172.21.15.93:0/3381777272 learned_addr learned my addr 172.21.15.93:0/3381777272 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.599+0000 7fa785e9b700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa784e99700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd589350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 -- 172.21.15.93:0/3381777272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 msgr2=0x5614cd589da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 -- 172.21.15.93:0/3381777272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 msgr2=0x5614cd589350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd589350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 -- 172.21.15.93:0/3381777272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614cd308de0 con 0x5614cd58e1b0 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa785e9b700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa784e99700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd589350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa785e9b700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 0x7fa76c0a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa778009fd0 tx=0x7fa77800bca0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa78569a700 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa77c099e40 tx=0x7fa77c096590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dcdac3ffcad7f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa76b7fe700 1 -- 172.21.15.93:0/3381777272 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c09e170 con 0x5614cd58e1b0 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa76b7fe700 1 -- 172.21.15.93:0/3381777272 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c0a2590 con 0x5614cd58e1b0 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.600+0000 7fa76b7fe700 1 -- 172.21.15.93:0/3381777272 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c099630 con 0x5614cd58e1b0 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- 172.21.15.93:0/3381777272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 msgr2=0x5614cd58e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa77c099e40 tx=0x7fa77c096590 comp rx=0 tx=0).stop 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- 172.21.15.93:0/3381777272 shutdown_connections 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd58e580 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3381777272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd589350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- 172.21.15.93:0/3381777272 >> 172.21.15.93:0/3381777272 conn(0x5614cd57eef0 msgr2=0x5614cd584830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- 172.21.15.93:0/3381777272 shutdown_connections 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- 172.21.15.93:0/3381777272 wait complete. 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 -- start start 2023-07-21T22:16:49.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.601+0000 7fa793a83b80 1 Processor -- start 2023-07-21T22:16:49.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd589930 0x5614cd595be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd58eac0 0x5614cd5935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd47aac0 con 0x5614cd58e1b0 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd47a7c0 con 0x5614cd589930 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd35e920 con 0x5614cd58eac0 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa793a83b80 10 client.0 _ll_register_callbacks cb 0x5614cd478760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44628/0 (socket says 172.21.15.93:44628) 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 -- 172.21.15.93:0/37218695 learned_addr learned my addr 172.21.15.93:0/37218695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa78569a700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd589930 0x5614cd595be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa785e9b700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd58eac0 0x5614cd5935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 -- 172.21.15.93:0/37218695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd58eac0 msgr2=0x5614cd5935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd58eac0 0x5614cd5935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 -- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd589930 msgr2=0x5614cd595be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd589930 0x5614cd595be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 -- 172.21.15.93:0/37218695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801a2b90 con 0x5614cd58e1b0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa784e99700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa77000ddb0 tx=0x7fa77000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d8890376b5c010 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.602+0000 7fa768ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770004830 con 0x5614cd58e1b0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa770015070 con 0x5614cd58e1b0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614cd4344f0 con 0x5614cd58e1b0 2023-07-21T22:16:49.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa770004990 con 0x5614cd58e1b0 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 10 client.15636 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5614cd2f1c60 con 0x5614cd58e1b0 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614cd4344f0 con 0x5614cd58e1b0 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa770019430 con 0x5614cd58e1b0 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 10 client.15636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 20 client.15636 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5a34d0 0x5614cd5a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 4 client.15636 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614cd4d3eb0 con 0x5614cd5a34d0 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.603+0000 7fa78569a700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5a34d0 0x5614cd5a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.604+0000 7fa768ff9700 10 client.15636 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:49.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.604+0000 7fa78569a700 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5a34d0 0x5614cd5a58e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3351b3e5a4ad28c7 server_cookie=2a6b5e1a34313053 in_seq=0 out_seq=0 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.604+0000 7fa768ff9700 1 -- 172.21.15.93:0/37218695 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5614cd4d3eb0 con 0x5614cd5a34d0 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.604+0000 7fa768ff9700 10 client.15636 handle_command_reply: tid=0 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 client.15636 shutdown 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 msgr2=0x5614cd592e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa77000ddb0 tx=0x7fa77000f640 comp rx=0 tx=0).stop 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 shutdown_connections 2023-07-21T22:16:49.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5a34d0 0x5614cd5a58e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd58eac0 0x5614cd5935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd589930 0x5614cd595be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa768ff9700 0 client.15636 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 --2- 172.21.15.93:0/37218695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd592e90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 >> 172.21.15.93:0/37218695 conn(0x5614cd57eef0 msgr2=0x5614cd5843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 shutdown_connections 2023-07-21T22:16:49.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 1 -- 172.21.15.93:0/37218695 wait complete. 2023-07-21T22:16:49.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.605+0000 7fa793a83b80 20 client.15636 trim_cache size 0 max 16384 2023-07-21T22:16:49.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 Processor -- start 2023-07-21T22:16:49.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 -- start start 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd595be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd581010 con 0x5614cd58e1b0 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd58de00 con 0x5614cd58eac0 2023-07-21T22:16:49.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa793a83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd57c020 con 0x5614cd589930 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44636/0 (socket says 172.21.15.93:44636) 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa78569a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 -- 172.21.15.93:0/1107979228 learned_addr learned my addr 172.21.15.93:0/1107979228 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa785e9b700 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd595be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 -- 172.21.15.93:0/1107979228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 msgr2=0x5614cd589d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 -- 172.21.15.93:0/1107979228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 msgr2=0x5614cd595be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd595be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 -- 172.21.15.93:0/1107979228 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614cd32f250 con 0x5614cd58e1b0 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.618+0000 7fa784e99700 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa770019760 tx=0x7fa77000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cb7c37af63fc20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa76bfff700 1 -- 172.21.15.93:0/1107979228 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000fd30 con 0x5614cd58e1b0 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa76bfff700 1 -- 172.21.15.93:0/1107979228 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77000fe90 con 0x5614cd58e1b0 2023-07-21T22:16:49.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa76bfff700 1 -- 172.21.15.93:0/1107979228 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7700052a0 con 0x5614cd58e1b0 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 -- 172.21.15.93:0/1107979228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 msgr2=0x5614cd5956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa770019760 tx=0x7fa77000df00 comp rx=0 tx=0).stop 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 -- 172.21.15.93:0/1107979228 shutdown_connections 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58eac0 0x5614cd595be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd589d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 --2- 172.21.15.93:0/1107979228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58e1b0 0x5614cd5956a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 -- 172.21.15.93:0/1107979228 >> 172.21.15.93:0/1107979228 conn(0x5614cd57eef0 msgr2=0x5614cd5aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 -- 172.21.15.93:0/1107979228 shutdown_connections 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.619+0000 7fa793a83b80 1 -- 172.21.15.93:0/1107979228 wait complete. 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 -- start start 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 Processor -- start 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd598750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 0x5614cd598c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd5f68f0 con 0x5614cd58eac0 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd5f6a30 con 0x5614cd58e1b0 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614cd5faf80 con 0x5614cd589930 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa793a83b80 10 client.0 _ll_register_callbacks cb 0x7fa77000d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa785e9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa785e9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44644/0 (socket says 172.21.15.93:44644) 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.620+0000 7fa785e9b700 1 -- 172.21.15.93:0/3726904180 learned_addr learned my addr 172.21.15.93:0/3726904180 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa784e99700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 0x5614cd598c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa78569a700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd598750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 -- 172.21.15.93:0/3726904180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 msgr2=0x5614cd598750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd598750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 -- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 msgr2=0x5614cd598c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 0x5614cd598c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 -- 172.21.15.93:0/3726904180 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614cd2f1c60 con 0x5614cd58eac0 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa784e99700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 0x5614cd598c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa785e9b700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7780078b0 tx=0x7fa77800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2105045b1ce7e9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77800e040 con 0x5614cd58eac0 2023-07-21T22:16:49.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa778005040 con 0x5614cd58eac0 2023-07-21T22:16:49.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa77c0a2590 con 0x5614cd58eac0 2023-07-21T22:16:49.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa778007990 con 0x5614cd58eac0 2023-07-21T22:16:49.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa793a83b80 10 client.15642 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5614cd3072b0 con 0x5614cd58eac0 2023-07-21T22:16:49.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa77c0a2590 con 0x5614cd58eac0 2023-07-21T22:16:49.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.621+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa77801e7e0 con 0x5614cd58eac0 2023-07-21T22:16:49.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 20 client.15642 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5fc700 0x5614cd5feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 4 client.15642 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614cd496050 con 0x5614cd5fc700 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa78569a700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5fc700 0x5614cd5feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa768ff9700 10 client.15642 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa78569a700 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5fc700 0x5614cd5feb10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96ad23ee2774e86 server_cookie=9d974e3b7ac77ce4 in_seq=0 out_seq=0 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa768ff9700 1 -- 172.21.15.93:0/3726904180 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614cd496050 con 0x5614cd5fc700 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa768ff9700 10 client.15642 handle_command_reply: tid=0 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 1 client.15642 shutdown 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.622+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 msgr2=0x5614cd5f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7780078b0 tx=0x7fa77800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 shutdown_connections 2023-07-21T22:16:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x5614cd5fc700 0x5614cd5feb10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614cd58eac0 0x5614cd5f63b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa768ff9700 0 client.15642 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5614cd589930 0x5614cd598750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 --2- 172.21.15.93:0/3726904180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614cd58e1b0 0x5614cd598c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 >> 172.21.15.93:0/3726904180 conn(0x5614cd57eef0 msgr2=0x5614cd587a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 shutdown_connections 2023-07-21T22:16:49.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 1 -- 172.21.15.93:0/3726904180 wait complete. 2023-07-21T22:16:49.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa793a83b80 20 client.15642 trim_cache size 0 max 16384 2023-07-21T22:16:49.683 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:16:49.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 msgr2=0x7fa76c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 0x7fa76c0a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa778009fd0 tx=0x7fa77800bca0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 msgr2=0x7fa780195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00e9a0 tx=0x7fa77c00ed60 comp rx=0 tx=0).stop 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 shutdown_connections 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa78010a040 0x7fa78019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa76c0a1350 0x7fa76c0a3800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa780100d00 0x7fa780195d60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 --2- 172.21.15.93:0/656233775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa780101610 0x7fa7801962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 >> 172.21.15.93:0/656233775 conn(0x7fa7800fa4b0 msgr2=0x7fa78010c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 shutdown_connections 2023-07-21T22:16:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.623+0000 7fa7878fe700 1 -- 172.21.15.93:0/656233775 wait complete. 2023-07-21T22:16:49.686 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:49 smithi093 ceph-mon[96579]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2023-07-21T22:16:49.686 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:49 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/652698025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:16:49.687 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:16:49.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 -- 172.21.15.93:0/3021206387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 msgr2=0x7f8f6c109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 --2- 172.21.15.93:0/3021206387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c109ae0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009990 tx=0x7f8f60009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 -- 172.21.15.93:0/3021206387 shutdown_connections 2023-07-21T22:16:49.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 --2- 172.21.15.93:0/3021206387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c10a020 0x7f8f6c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 --2- 172.21.15.93:0/3021206387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c100ce0 0x7f8f6c1010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 --2- 172.21.15.93:0/3021206387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c109ae0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 -- 172.21.15.93:0/3021206387 >> 172.21.15.93:0/3021206387 conn(0x7f8f6c0fa4b0 msgr2=0x7f8f6c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 -- 172.21.15.93:0/3021206387 shutdown_connections 2023-07-21T22:16:49.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.775+0000 7f8f72867700 1 -- 172.21.15.93:0/3021206387 wait complete. 2023-07-21T22:16:49.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.776+0000 7f8f72867700 1 Processor -- start 2023-07-21T22:16:49.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.776+0000 7f8f72867700 1 -- start start 2023-07-21T22:16:49.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c100ce0 0x7f8f6c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c10a020 0x7f8f6c1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c196050 con 0x7f8f6c1015f0 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1961c0 con 0x7f8f6c100ce0 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c196380 con 0x7f8f6c10a020 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44654/0 (socket says 172.21.15.93:44654) 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/285072908 learned_addr learned my addr 172.21.15.93:0/285072908 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c100ce0 0x7f8f6c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f70e04700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c10a020 0x7f8f6c1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/285072908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c10a020 msgr2=0x7f8f6c1959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c10a020 0x7f8f6c1959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/285072908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c100ce0 msgr2=0x7f8f6c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c100ce0 0x7f8f6c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/285072908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600095f0 con 0x7f8f6c1015f0 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000c00 tx=0x7f8f600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f697fa700 1 -- 172.21.15.93:0/285072908 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6001d070 con 0x7f8f6c1015f0 2023-07-21T22:16:49.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c196600 con 0x7f8f6c1015f0 2023-07-21T22:16:49.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f697fa700 1 -- 172.21.15.93:0/285072908 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6000fb80 con 0x7f8f6c1015f0 2023-07-21T22:16:49.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.777+0000 7f8f697fa700 1 -- 172.21.15.93:0/285072908 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600173f0 con 0x7f8f6c1015f0 2023-07-21T22:16:49.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.778+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c1a2960 con 0x7f8f6c1015f0 2023-07-21T22:16:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.780+0000 7f8f697fa700 1 -- 172.21.15.93:0/285072908 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f8f6000fcf0 con 0x7f8f6c1015f0 2023-07-21T22:16:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.780+0000 7f8f697fa700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 0x7f8f540a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.780+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 0x7f8f540a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.780+0000 7f8f697fa700 1 -- 172.21.15.93:0/285072908 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f600ce1f0 con 0x7f8f6c1015f0 2023-07-21T22:16:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 0x7f8f540a3800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009c60 tx=0x7f8f5c009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 Processor -- start 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 -- start start 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8d1b0 0x55933eb8d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ea8dcb0 con 0x55933eb88930 2023-07-21T22:16:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eae0a40 con 0x55933eb8dac0 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.781+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ead9ed0 con 0x55933eb8d1b0 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44664/0 (socket says 172.21.15.93:44664) 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 -- 172.21.15.93:0/1120444823 learned_addr learned my addr 172.21.15.93:0/1120444823 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8d1b0 0x55933eb8d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 -- 172.21.15.93:0/1120444823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8d1b0 msgr2=0x55933eb8d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8d1b0 0x55933eb8d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 -- 172.21.15.93:0/1120444823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 msgr2=0x55933eb88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 -- 172.21.15.93:0/1120444823 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933e907de0 con 0x55933eb88930 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f70e04700 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009cf0 tx=0x7f8f64007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c98817e4e5b963d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f527fc700 1 -- 172.21.15.93:0/1120444823 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64018070 con 0x55933eb88930 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f527fc700 1 -- 172.21.15.93:0/1120444823 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6400f050 con 0x55933eb88930 2023-07-21T22:16:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.782+0000 7f8f527fc700 1 -- 172.21.15.93:0/1120444823 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64014410 con 0x55933eb88930 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1120444823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 msgr2=0x55933eb88da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009cf0 tx=0x7f8f64007500 comp rx=0 tx=0).stop 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1120444823 shutdown_connections 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb88930 0x55933eb88da0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8d1b0 0x55933eb8d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1120444823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1120444823 >> 172.21.15.93:0/1120444823 conn(0x55933eb7def0 msgr2=0x55933eb83830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1120444823 shutdown_connections 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1120444823 wait complete. 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.783+0000 7f8f7e9ecb80 1 -- start start 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 Processor -- start 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb88930 0x55933eb94bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8dac0 0x55933eb92540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ea79ac0 con 0x55933eb8d1b0 2023-07-21T22:16:49.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ea797c0 con 0x55933eb88930 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933e95d920 con 0x55933eb8dac0 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f7e9ecb80 10 client.0 _ll_register_callbacks cb 0x55933ea77760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44668/0 (socket says 172.21.15.93:44668) 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/1421210669 learned_addr learned my addr 172.21.15.93:0/1421210669 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb88930 0x55933eb94bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f70e04700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8dac0 0x55933eb92540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8dac0 msgr2=0x55933eb92540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8dac0 0x55933eb92540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/1421210669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb88930 msgr2=0x55933eb94bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb88930 0x55933eb94bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/1421210669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f64009920 con 0x55933eb8d1b0 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cbf60 tx=0x7f8f6009ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baab3e079640d2fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.784+0000 7f8f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600a10d0 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f600a35c0 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55933ea334f0 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f6009f020 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 10 client.15654 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55933e8f0c60 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55933ea334f0 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f60096ff0 con 0x55933eb8d1b0 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 20 client.15654 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933eba2070 0x55933eba4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 4 client.15654 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.785+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55933ead2eb0 con 0x55933eba2070 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.786+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933eba2070 0x55933eba4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.786+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933eba2070 0x55933eba4450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75378bfa4eb16664 server_cookie=4b8558c010f73b90 in_seq=0 out_seq=0 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.786+0000 7f8f4b7fe700 10 client.15654 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:49.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/1421210669 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55933ead2eb0 con 0x55933eba2070 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f4b7fe700 10 client.15654 handle_command_reply: tid=0 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 client.15654 shutdown 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 msgr2=0x55933eb91e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cbf60 tx=0x7f8f6009ac90 comp rx=0 tx=0).stop 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 shutdown_connections 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933eba2070 0x55933eba4450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb8dac0 0x55933eb92540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb88930 0x55933eb94bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/1421210669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb91e00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f4b7fe700 0 client.15654 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 >> 172.21.15.93:0/1421210669 conn(0x55933eb7def0 msgr2=0x55933eb833c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 shutdown_connections 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/1421210669 wait complete. 2023-07-21T22:16:49.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.787+0000 7f8f7e9ecb80 20 client.15654 trim_cache size 0 max 16384 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 Processor -- start 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 -- start start 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8d1b0 0x55933eb94720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eb80040 con 0x55933eb8dac0 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eb8ce00 con 0x55933eb8d1b0 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eb84e30 con 0x55933eb88930 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44678/0 (socket says 172.21.15.93:44678) 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 -- 172.21.15.93:0/3666242986 learned_addr learned my addr 172.21.15.93:0/3666242986 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8d1b0 0x55933eb94720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 -- 172.21.15.93:0/3666242986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 msgr2=0x55933eb99580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb99580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 -- 172.21.15.93:0/3666242986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8d1b0 msgr2=0x55933eb94720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8d1b0 0x55933eb94720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.801+0000 7f8f70e04700 1 -- 172.21.15.93:0/3666242986 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55933e92e250 con 0x55933eb8dac0 2023-07-21T22:16:49.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f70e04700 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005710 tx=0x7f8f64006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c3a0ac43c82633 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f52ffd700 1 -- 172.21.15.93:0/3666242986 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64018040 con 0x55933eb8dac0 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f52ffd700 1 -- 172.21.15.93:0/3666242986 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f64019040 con 0x55933eb8dac0 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f52ffd700 1 -- 172.21.15.93:0/3666242986 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64004a90 con 0x55933eb8dac0 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/3666242986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 msgr2=0x55933eb86a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.802+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8f64005710 tx=0x7f8f64006000 comp rx=0 tx=0).stop 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/3666242986 shutdown_connections 2023-07-21T22:16:49.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8dac0 0x55933eb86a00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/3666242986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8d1b0 0x55933eb94720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/3666242986 >> 172.21.15.93:0/3666242986 conn(0x55933eb7def0 msgr2=0x55933eb88d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/3666242986 shutdown_connections 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/3666242986 wait complete. 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- start start 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 Processor -- start 2023-07-21T22:16:49.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eb98650 con 0x55933eb8d1b0 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933eb987c0 con 0x55933eb8dac0 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933ebf5f10 con 0x55933eb88930 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.803+0000 7f8f7e9ecb80 10 client.0 _ll_register_callbacks cb 0x7f8f6009ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:49.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44682/0 (socket says 172.21.15.93:44682) 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/746746342 learned_addr learned my addr 172.21.15.93:0/746746342 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f70e04700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 msgr2=0x55933eb995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/746746342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 msgr2=0x55933eb97f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:49.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb97f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 -- 172.21.15.93:0/746746342 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55933e907de0 con 0x55933eb8d1b0 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f70e04700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f6b7fe700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cbee0 tx=0x7f8f60097320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2278d59bcde45a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600a10d0 con 0x55933eb8d1b0 2023-07-21T22:16:49.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8f600a35c0 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f6400f050 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f600cb7e0 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f7e9ecb80 10 client.15660 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.804+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55933e9062b0 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6400f050 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f6009a160 con 0x55933eb8d1b0 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:49.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 20 client.15660 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933ebf7090 0x55933ebf9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 4 client.15660 mds_command: new command op to 24484 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55933ea95050 con 0x55933ebf7090 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933ebf7090 0x55933ebf9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f4b7fe700 10 client.15660 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:49.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.805+0000 7f8f6bfff700 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933ebf7090 0x55933ebf9470 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ecda96fe54cb9e server_cookie=c768f59dda09f50f in_seq=0 out_seq=0 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f4b7fe700 1 -- 172.21.15.93:0/746746342 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55933ea95050 con 0x55933ebf7090 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f4b7fe700 10 client.15660 handle_command_reply: tid=0 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 client.15660 shutdown 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 msgr2=0x55933eb979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8f600cbee0 tx=0x7f8f60097320 comp rx=0 tx=0).stop 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 shutdown_connections 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55933ebf7090 0x55933ebf9470 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55933eb8dac0 0x55933eb97f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f4b7fe700 0 client.15660 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55933eb88930 0x55933eb995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 --2- 172.21.15.93:0/746746342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55933eb8d1b0 0x55933eb979d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 >> 172.21.15.93:0/746746342 conn(0x55933eb7def0 msgr2=0x55933eb86a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 shutdown_connections 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 1 -- 172.21.15.93:0/746746342 wait complete. 2023-07-21T22:16:49.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.806+0000 7f8f7e9ecb80 20 client.15660 trim_cache size 0 max 16384 2023-07-21T22:16:49.860 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:16:49.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 msgr2=0x7f8f540a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 0x7f8f540a3800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009c60 tx=0x7f8f5c009260 comp rx=0 tx=0).stop 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 msgr2=0x7f8f6c19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000c00 tx=0x7f8f600047b0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 shutdown_connections 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f6c10a020 0x7f8f6c1959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f6c100ce0 0x7f8f6c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8f540a1350 0x7f8f540a3800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 --2- 172.21.15.93:0/285072908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f6c1015f0 0x7f8f6c19beb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 >> 172.21.15.93:0/285072908 conn(0x7f8f6c0fa4b0 msgr2=0x7f8f6c10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 shutdown_connections 2023-07-21T22:16:49.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:49.807+0000 7f8f72867700 1 -- 172.21.15.93:0/285072908 wait complete. 2023-07-21T22:16:50.863 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:16:50.864 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:16:50.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 -- 172.21.15.93:0/2141698188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c105ae0 msgr2=0x7fcb1c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:50.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/2141698188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c105eb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009990 tx=0x7fcb18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 -- 172.21.15.93:0/2141698188 shutdown_connections 2023-07-21T22:16:50.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/2141698188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 0x7fcb1c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/2141698188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c105eb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/2141698188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 -- 172.21.15.93:0/2141698188 >> 172.21.15.93:0/2141698188 conn(0x7fcb1c0fb5f0 msgr2=0x7fcb1c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 -- 172.21.15.93:0/2141698188 shutdown_connections 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.949+0000 7fcb23a0f700 1 -- 172.21.15.93:0/2141698188 wait complete. 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 Processor -- start 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 -- start start 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:50.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 0x7fcb1c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19cdb0 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19cf20 con 0x7fcb1c105ae0 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb23a0f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19d0e0 con 0x7fcb1c100480 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44694/0 (socket says 172.21.15.93:44694) 2023-07-21T22:16:50.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 -- 172.21.15.93:0/1835297324 learned_addr learned my addr 172.21.15.93:0/1835297324 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 0x7fcb1c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb21fac700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 -- 172.21.15.93:0/1835297324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 msgr2=0x7fcb1c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 0x7fcb1c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 -- 172.21.15.93:0/1835297324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c105ae0 msgr2=0x7fcb1c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:50.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 -- 172.21.15.93:0/1835297324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180095f0 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.950+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb180095c0 tx=0x7fcb1800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.951+0000 7fcb127fc700 1 -- 172.21.15.93:0/1835297324 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1801d070 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.951+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c19d360 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.951+0000 7fcb127fc700 1 -- 172.21.15.93:0/1835297324 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb180054f0 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.951+0000 7fcb127fc700 1 -- 172.21.15.93:0/1835297324 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18004a70 con 0x7fcb1c0ffad0 2023-07-21T22:16:50.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.951+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1a4c00 con 0x7fcb1c0ffad0 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.953+0000 7fcb127fc700 1 -- 172.21.15.93:0/1835297324 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fcb18004bd0 con 0x7fcb1c0ffad0 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.953+0000 7fcb127fc700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 0x7fcb080a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.953+0000 7fcb127fc700 1 -- 172.21.15.93:0/1835297324 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcb180cdbb0 con 0x7fcb1c0ffad0 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.953+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 0x7fcb080a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.954+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 0x7fcb080a3800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c1993e0 tx=0x7fcb0c00b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 Processor -- start 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 -- start start 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64b40 0x55e1aef5f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aee64d30 con 0x55e1aef5f9b0 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aeeb7ac0 con 0x55e1aef64230 2023-07-21T22:16:51.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aeeb0f50 con 0x55e1aef64b40 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb21fac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb21fac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44706/0 (socket says 172.21.15.93:44706) 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb21fac700 1 -- 172.21.15.93:0/1206396645 learned_addr learned my addr 172.21.15.93:0/1206396645 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.956+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64b40 0x55e1aef5f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 -- 172.21.15.93:0/1206396645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64b40 msgr2=0x55e1aef5f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64b40 0x55e1aef5f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 -- 172.21.15.93:0/1206396645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 msgr2=0x55e1aef64600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef64600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 -- 172.21.15.93:0/1206396645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1aecdee60 con 0x55e1aef5f9b0 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef64600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb21fac700 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400a730 tx=0x7fcb140078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e70bd8c5d980dc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb077fe700 1 -- 172.21.15.93:0/1206396645 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018070 con 0x55e1aef5f9b0 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb077fe700 1 -- 172.21.15.93:0/1206396645 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb14010040 con 0x55e1aef5f9b0 2023-07-21T22:16:51.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.957+0000 7fcb077fe700 1 -- 172.21.15.93:0/1206396645 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400f050 con 0x55e1aef5f9b0 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1206396645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 msgr2=0x55e1aef5fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400a730 tx=0x7fcb140078d0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1206396645 shutdown_connections 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5fe20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef64600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1206396645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64b40 0x55e1aef5f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1206396645 >> 172.21.15.93:0/1206396645 conn(0x55e1aef54f70 msgr2=0x55e1aef5a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1206396645 shutdown_connections 2023-07-21T22:16:51.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1206396645 wait complete. 2023-07-21T22:16:51.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 -- start start 2023-07-21T22:16:51.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.958+0000 7fcb2fb94b80 1 Processor -- start 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef68e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aee50b40 con 0x55e1aef64b40 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aee50840 con 0x55e1aef5f9b0 2023-07-21T22:16:51.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aed349a0 con 0x55e1aef64230 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 10 client.0 _ll_register_callbacks cb 0x55e1aee4e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb20faa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef68e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44712/0 (socket says 172.21.15.93:44712) 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb20faa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef68e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:60022/0 (socket says 172.21.15.93:60022) 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 -- 172.21.15.93:0/2853090044 learned_addr learned my addr 172.21.15.93:0/2853090044 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 -- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 msgr2=0x55e1aef68e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef68e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 -- 172.21.15.93:0/2853090044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 msgr2=0x55e1aef6bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 -- 172.21.15.93:0/2853090044 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c1a4c00 con 0x55e1aef64b40 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb21fac700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400a360 tx=0x7fcb1400a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1102b63d4e88a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:51.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018030 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcb14003e10 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.959+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1aee0a570 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb14003fd0 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 10 client.15672 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fcb1400a410 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1aee0a570 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb14014400 con 0x55e1aef64b40 2023-07-21T22:16:51.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 20 client.15672 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef79380 0x55e1aef7b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 4 client.15672 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1aeea9f30 con 0x55e1aef79380 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef79380 0x55e1aef7b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 10 client.15672 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef79380 0x55e1aef7b790 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594e1762468b8c2d server_cookie=6ad7c24ccacd0573 in_seq=0 out_seq=0 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 1 -- 172.21.15.93:0/2853090044 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e1aeea9f30 con 0x55e1aef79380 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb04ff9700 10 client.15672 handle_command_reply: tid=0 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 client.15672 shutdown 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 msgr2=0x55e1aef695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.960+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400a360 tx=0x7fcb1400a390 comp rx=0 tx=0).stop 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 shutdown_connections 2023-07-21T22:16:51.015 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef79380 0x55e1aef7b790 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef64b40 0x55e1aef695c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2853090044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef68e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 >> 172.21.15.93:0/2853090044 conn(0x55e1aef54f70 msgr2=0x55e1aef5a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb04ff9700 0 client.15672 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 shutdown_connections 2023-07-21T22:16:51.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2853090044 wait complete. 2023-07-21T22:16:51.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.961+0000 7fcb2fb94b80 20 client.15672 trim_cache size 0 max 16384 2023-07-21T22:16:51.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.977+0000 7fcb2fb94b80 1 Processor -- start 2023-07-21T22:16:51.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.977+0000 7fcb2fb94b80 1 -- start start 2023-07-21T22:16:51.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef6b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aef791a0 con 0x55e1aef54f70 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aef54a30 con 0x55e1aef5f9b0 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aef57090 con 0x55e1aef64230 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44716/0 (socket says 172.21.15.93:44716) 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 -- 172.21.15.93:0/2941165735 learned_addr learned my addr 172.21.15.93:0/2941165735 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb21fac700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef6b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb20faa700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 -- 172.21.15.93:0/2941165735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 msgr2=0x55e1aef6b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef6b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 -- 172.21.15.93:0/2941165735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 msgr2=0x55e1aef5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 -- 172.21.15.93:0/2941165735 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1aef592b0 con 0x55e1aef54f70 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb20faa700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb21fac700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef6b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.978+0000 7fcb217ab700 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb180a0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d6b8eef2b42779 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb07fff700 1 -- 172.21.15.93:0/2941165735 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1809ebf0 con 0x55e1aef54f70 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb07fff700 1 -- 172.21.15.93:0/2941165735 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb180a2f80 con 0x55e1aef54f70 2023-07-21T22:16:51.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb07fff700 1 -- 172.21.15.93:0/2941165735 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1809ae20 con 0x55e1aef54f70 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2941165735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 msgr2=0x55e1aef67f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb180a0040 comp rx=0 tx=0).stop 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2941165735 shutdown_connections 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef64230 0x55e1aef6b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef54f70 0x55e1aef67f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/2941165735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef5f9b0 0x55e1aef5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2941165735 >> 172.21.15.93:0/2941165735 conn(0x55e1aef75b70 msgr2=0x55e1aef5e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2941165735 shutdown_connections 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.979+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/2941165735 wait complete. 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 -- start start 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 Processor -- start 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef54f70 0x55e1aef6edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aefcd380 con 0x55e1aef5f9b0 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aefcd4f0 con 0x55e1aef64230 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1aefcd6b0 con 0x55e1aef54f70 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb2fb94b80 10 client.0 _ll_register_callbacks cb 0x7fcb180cb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb20faa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb20faa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44726/0 (socket says 172.21.15.93:44726) 2023-07-21T22:16:51.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb20faa700 1 -- 172.21.15.93:0/1051379830 learned_addr learned my addr 172.21.15.93:0/1051379830 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef54f70 0x55e1aef6edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.980+0000 7fcb21fac700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 -- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef54f70 msgr2=0x55e1aef6edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef54f70 0x55e1aef6edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 -- 172.21.15.93:0/1051379830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 msgr2=0x55e1aef6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef6f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 -- 172.21.15.93:0/1051379830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb180ba040 con 0x55e1aef5f9b0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb20faa700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005d90 tx=0x7fcb0c013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13d04f4b56bb36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c024040 con 0x55e1aef5f9b0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcb0c017070 con 0x55e1aef5f9b0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb14010040 con 0x55e1aef5f9b0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb0c029480 con 0x55e1aef5f9b0 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb2fb94b80 10 client.15678 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:51.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e1aecc7ce0 con 0x55e1aef5f9b0 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.981+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c036020 con 0x55e1aef5f9b0 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb0c01f6c0 con 0x55e1aef5f9b0 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 20 client.15678 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef64b40 0x55e1aefd0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 4 client.15678 mds_command: new command op to 24484 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e1aee6c0d0 con 0x55e1aef64b40 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef64b40 0x55e1aefd0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb04ff9700 10 client.15678 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.982+0000 7fcb217ab700 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef64b40 0x55e1aefd0f70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207a442c743add54 server_cookie=9ea4eb6cc02742c8 in_seq=0 out_seq=0 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.983+0000 7fcb04ff9700 1 -- 172.21.15.93:0/1051379830 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e1aee6c0d0 con 0x55e1aef64b40 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.983+0000 7fcb04ff9700 10 client.15678 handle_command_reply: tid=0 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.983+0000 7fcb2fb94b80 1 client.15678 shutdown 2023-07-21T22:16:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.983+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 msgr2=0x55e1aef6f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.983+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005d90 tx=0x7fcb0c013070 comp rx=0 tx=0).stop 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 shutdown_connections 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e1aef64b40 0x55e1aefd0f70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1aef64230 0x55e1aef6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e1aef54f70 0x55e1aef6edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb04ff9700 0 client.15678 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 --2- 172.21.15.93:0/1051379830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1aef5f9b0 0x55e1aef6f330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 >> 172.21.15.93:0/1051379830 conn(0x55e1aef75b70 msgr2=0x55e1aef62e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 shutdown_connections 2023-07-21T22:16:51.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 1 -- 172.21.15.93:0/1051379830 wait complete. 2023-07-21T22:16:51.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb2fb94b80 20 client.15678 trim_cache size 0 max 16384 2023-07-21T22:16:51.038 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:51.038 INFO:teuthology.orchestra.run.smithi093.stdout: "return_code": 0, 2023-07-21T22:16:51.039 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_tag": "a09c4c4a-082b-4b9a-9901-14633694a71a", 2023-07-21T22:16:51.039 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": "asynchronous" 2023-07-21T22:16:51.039 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 msgr2=0x7fcb080a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 0x7fcb080a3800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c1993e0 tx=0x7fcb0c00b410 comp rx=0 tx=0).stop 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 msgr2=0x7fcb1c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcb180095c0 tx=0x7fcb1800f710 comp rx=0 tx=0).stop 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 shutdown_connections 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1c105ae0 0x7fcb1c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcb080a1350 0x7fcb080a3800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb1c0ffad0 0x7fcb1c197dd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 --2- 172.21.15.93:0/1835297324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb1c100480 0x7fcb1c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.984+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 >> 172.21.15.93:0/1835297324 conn(0x7fcb1c0fb5f0 msgr2=0x7fcb1c10ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.985+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 shutdown_connections 2023-07-21T22:16:51.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:50.985+0000 7fcb23a0f700 1 -- 172.21.15.93:0/1835297324 wait complete. 2023-07-21T22:16:51.042 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:16:51.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 -- 172.21.15.93:0/4046380687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c10c430 msgr2=0x7fe95c10e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 --2- 172.21.15.93:0/4046380687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c10c430 0x7fe95c10e810 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b0a0 tx=0x7fe95400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 -- 172.21.15.93:0/4046380687 shutdown_connections 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 --2- 172.21.15.93:0/4046380687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c10c430 0x7fe95c10e810 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 --2- 172.21.15.93:0/4046380687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0fe880 0x7fe95c0fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 --2- 172.21.15.93:0/4046380687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c0ff190 0x7fe95c10bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 -- 172.21.15.93:0/4046380687 >> 172.21.15.93:0/4046380687 conn(0x7fe95c0fa4b0 msgr2=0x7fe95c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 -- 172.21.15.93:0/4046380687 shutdown_connections 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.125+0000 7fe9641fc700 1 -- 172.21.15.93:0/4046380687 wait complete. 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 Processor -- start 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 -- start start 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0ff190 0x7fe95c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 0x7fe95c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c19ce90 con 0x7fe95c0fe880 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c19d000 con 0x7fe95c10c430 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe9641fc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c19d1c0 con 0x7fe95c0ff190 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44736/0 (socket says 172.21.15.93:44736) 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 -- 172.21.15.93:0/2769062579 learned_addr learned my addr 172.21.15.93:0/2769062579 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961797700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0ff190 0x7fe95c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe962799700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 0x7fe95c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 -- 172.21.15.93:0/2769062579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0ff190 msgr2=0x7fe95c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0ff190 0x7fe95c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 -- 172.21.15.93:0/2769062579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 msgr2=0x7fe95c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 0x7fe95c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 -- 172.21.15.93:0/2769062579 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954009d00 con 0x7fe95c0fe880 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe962799700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 0x7fe95c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:51.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.126+0000 7fe961f98700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d950 tx=0x7fe95800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.127+0000 7fe952ffd700 1 -- 172.21.15.93:0/2769062579 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958009b40 con 0x7fe95c0fe880 2023-07-21T22:16:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.127+0000 7fe952ffd700 1 -- 172.21.15.93:0/2769062579 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe958009ca0 con 0x7fe95c0fe880 2023-07-21T22:16:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.127+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c19d4a0 con 0x7fe95c0fe880 2023-07-21T22:16:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.127+0000 7fe952ffd700 1 -- 172.21.15.93:0/2769062579 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95800f620 con 0x7fe95c0fe880 2023-07-21T22:16:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.127+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c1a4ce0 con 0x7fe95c0fe880 2023-07-21T22:16:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.129+0000 7fe952ffd700 1 -- 172.21.15.93:0/2769062579 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fe958010460 con 0x7fe95c0fe880 2023-07-21T22:16:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.129+0000 7fe952ffd700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 0x7fe9480a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.129+0000 7fe952ffd700 1 -- 172.21.15.93:0/2769062579 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe9580cd220 con 0x7fe95c0fe880 2023-07-21T22:16:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.129+0000 7fe961797700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 0x7fe9480a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.130+0000 7fe961797700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 0x7fe9480a3800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1994c0 tx=0x7fe94c009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.130+0000 7fe970381b80 1 Processor -- start 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.130+0000 7fe970381b80 1 -- start start 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b385580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b380da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b285cb0 con 0x55c96b385ac0 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b2d8a40 con 0x55c96b3851b0 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe970381b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b2d1ed0 con 0x55c96b380930 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44746/0 (socket says 172.21.15.93:44746) 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 -- 172.21.15.93:0/3583530611 learned_addr learned my addr 172.21.15.93:0/3583530611 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961f98700 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b385580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe962799700 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b380da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 -- 172.21.15.93:0/3583530611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 msgr2=0x55c96b380da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b380da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 -- 172.21.15.93:0/3583530611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 msgr2=0x55c96b385580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b385580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 -- 172.21.15.93:0/3583530611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96b0ffde0 con 0x55c96b385ac0 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe961797700 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010e00 tx=0x7fe94c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7973e6ad6ac086 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.131+0000 7fe947fff700 1 -- 172.21.15.93:0/3583530611 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c009960 con 0x55c96b385ac0 2023-07-21T22:16:51.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe947fff700 1 -- 172.21.15.93:0/3583530611 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c015070 con 0x55c96b385ac0 2023-07-21T22:16:51.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe947fff700 1 -- 172.21.15.93:0/3583530611 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c01c8d0 con 0x55c96b385ac0 2023-07-21T22:16:51.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 -- 172.21.15.93:0/3583530611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 msgr2=0x55c96b380350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010e00 tx=0x7fe94c013000 comp rx=0 tx=0).stop 2023-07-21T22:16:51.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 -- 172.21.15.93:0/3583530611 shutdown_connections 2023-07-21T22:16:51.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b380da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b385580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 --2- 172.21.15.93:0/3583530611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b380350 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.188 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 -- 172.21.15.93:0/3583530611 >> 172.21.15.93:0/3583530611 conn(0x55c96b375ef0 msgr2=0x55c96b37b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 -- 172.21.15.93:0/3583530611 shutdown_connections 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.132+0000 7fe970381b80 1 -- 172.21.15.93:0/3583530611 wait complete. 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 -- start start 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 Processor -- start 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b38cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.189 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b385ac0 0x55c96b38a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b271ac0 con 0x55c96b3851b0 2023-07-21T22:16:51.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b2717c0 con 0x55c96b385ac0 2023-07-21T22:16:51.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.133+0000 7fe970381b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b155920 con 0x55c96b380930 2023-07-21T22:16:51.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe970381b80 10 client.0 _ll_register_callbacks cb 0x55c96b26f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:51.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44752/0 (socket says 172.21.15.93:44752) 2023-07-21T22:16:51.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 -- 172.21.15.93:0/2978103164 learned_addr learned my addr 172.21.15.93:0/2978103164 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961f98700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b38cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe962799700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b385ac0 0x55c96b38a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 -- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 msgr2=0x55c96b38cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b38cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 -- 172.21.15.93:0/2978103164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b385ac0 msgr2=0x55c96b38a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b385ac0 0x55c96b38a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 -- 172.21.15.93:0/2978103164 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c010ae0 con 0x55c96b3851b0 2023-07-21T22:16:51.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961f98700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b38cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:16:51.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe961797700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010760 tx=0x7fe94c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71dce2aaf21690c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe9457fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:51.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c02d070 con 0x55c96b3851b0 2023-07-21T22:16:51.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe94c017070 con 0x55c96b3851b0 2023-07-21T22:16:51.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c96b22b4f0 con 0x55c96b3851b0 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe94c005ce0 con 0x55c96b3851b0 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe970381b80 10 client.15690 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.134+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c96b0e8c60 con 0x55c96b3851b0 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c96b22b4f0 con 0x55c96b3851b0 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe94c004b90 con 0x55c96b3851b0 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:51.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:51.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 20 client.15690 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:51.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b39a380 0x55c96b39c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 4 client.15690 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:51.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c96b2caeb0 con 0x55c96b39a380 2023-07-21T22:16:51.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.135+0000 7fe961f98700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b39a380 0x55c96b39c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe9457fa700 10 client.15690 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe961f98700 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b39a380 0x55c96b39c790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5226c44a15a7f641 server_cookie=bd45feef87b30f11 in_seq=0 out_seq=0 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe9457fa700 1 -- 172.21.15.93:0/2978103164 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c96b2caeb0 con 0x55c96b39a380 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe9457fa700 10 client.15690 handle_command_reply: tid=0 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe970381b80 1 client.15690 shutdown 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 msgr2=0x55c96b389e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010760 tx=0x7fe94c005590 comp rx=0 tx=0).stop 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 shutdown_connections 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.136+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b39a380 0x55c96b39c790 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b385ac0 0x55c96b38a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe9457fa700 0 client.15690 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b38cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 --2- 172.21.15.93:0/2978103164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b389e20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 >> 172.21.15.93:0/2978103164 conn(0x55c96b375ef0 msgr2=0x55c96b37b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 shutdown_connections 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 1 -- 172.21.15.93:0/2978103164 wait complete. 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.137+0000 7fe970381b80 20 client.15690 trim_cache size 0 max 16384 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 Processor -- start 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 -- start start 2023-07-21T22:16:51.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b380930 0x55c96b391580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b385ac0 0x55c96b37ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b38d350 con 0x55c96b3851b0 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b3883a0 con 0x55c96b380930 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe970381b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b391ac0 con 0x55c96b385ac0 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe961f98700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b380930 0x55c96b391580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe961797700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44760/0 (socket says 172.21.15.93:44760) 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe961797700 1 -- 172.21.15.93:0/1889863042 learned_addr learned my addr 172.21.15.93:0/1889863042 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.155+0000 7fe962799700 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b385ac0 0x55c96b37ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 -- 172.21.15.93:0/1889863042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b385ac0 msgr2=0x55c96b37ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b385ac0 0x55c96b37ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 -- 172.21.15.93:0/1889863042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b380930 msgr2=0x55c96b391580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b380930 0x55c96b391580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 -- 172.21.15.93:0/1889863042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96b126250 con 0x55c96b3851b0 2023-07-21T22:16:51.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe961797700 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe94c020020 tx=0x7fe94c004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1d23bf55ba7204 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe950ff9700 1 -- 172.21.15.93:0/1889863042 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c02d070 con 0x55c96b3851b0 2023-07-21T22:16:51.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe950ff9700 1 -- 172.21.15.93:0/1889863042 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c0056b0 con 0x55c96b3851b0 2023-07-21T22:16:51.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.156+0000 7fe950ff9700 1 -- 172.21.15.93:0/1889863042 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c004a00 con 0x55c96b3851b0 2023-07-21T22:16:51.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- 172.21.15.93:0/1889863042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 msgr2=0x55c96b38c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe94c020020 tx=0x7fe94c004fd0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- 172.21.15.93:0/1889863042 shutdown_connections 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b385ac0 0x55c96b37ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b380930 0x55c96b391580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 --2- 172.21.15.93:0/1889863042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b3851b0 0x55c96b38c740 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- 172.21.15.93:0/1889863042 >> 172.21.15.93:0/1889863042 conn(0x55c96b375ef0 msgr2=0x55c96b395d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- 172.21.15.93:0/1889863042 shutdown_connections 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- 172.21.15.93:0/1889863042 wait complete. 2023-07-21T22:16:51.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 -- start start 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.157+0000 7fe970381b80 1 Processor -- start 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b391540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b38f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b38fe00 con 0x55c96b385ac0 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b38ff70 con 0x55c96b3851b0 2023-07-21T22:16:51.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96b390130 con 0x55c96b380930 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe970381b80 10 client.0 _ll_register_callbacks cb 0x7fe94c010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe961f98700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b38f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe961f98700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:60076/0 (socket says 172.21.15.93:60076) 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe961f98700 1 -- 172.21.15.93:0/1048143071 learned_addr learned my addr 172.21.15.93:0/1048143071 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe961797700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b391540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 -- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 msgr2=0x55c96b391000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 -- 172.21.15.93:0/1048143071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 msgr2=0x55c96b391540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b391540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe962799700 1 -- 172.21.15.93:0/1048143071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c96b0e8c60 con 0x55c96b385ac0 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.158+0000 7fe961f98700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe962799700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b38f760 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe954009fd0 tx=0x7fe954004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c465cbacdd46b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:51.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400e040 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe954023070 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94c015070 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9540074e0 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 10 client.15696 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c96b0fe2b0 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c015070 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe95400b0d0 con 0x55c96b385ac0 2023-07-21T22:16:51.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 20 client.15696 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b3ef540 0x55c96b3f1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 4 client.15696 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.159+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c96b28d050 con 0x55c96b3ef540 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe961f98700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b3ef540 0x55c96b3f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe9457fa700 10 client.15696 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe961f98700 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b3ef540 0x55c96b3f1950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59b04d8124d5a08 server_cookie=463dbbbce4eb08a2 in_seq=0 out_seq=0 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe9457fa700 1 -- 172.21.15.93:0/1048143071 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c96b28d050 con 0x55c96b3ef540 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe9457fa700 10 client.15696 handle_command_reply: tid=0 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 client.15696 shutdown 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 msgr2=0x55c96b38f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b38f760 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe954009fd0 tx=0x7fe954004a50 comp rx=0 tx=0).stop 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 shutdown_connections 2023-07-21T22:16:51.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c96b3ef540 0x55c96b3f1950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c96b385ac0 0x55c96b38f760 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe9457fa700 0 client.15696 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c96b380930 0x55c96b391000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 --2- 172.21.15.93:0/1048143071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c96b3851b0 0x55c96b391540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 >> 172.21.15.93:0/1048143071 conn(0x55c96b375ef0 msgr2=0x55c96b37ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 shutdown_connections 2023-07-21T22:16:51.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.160+0000 7fe970381b80 1 -- 172.21.15.93:0/1048143071 wait complete. 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe970381b80 20 client.15696 trim_cache size 0 max 16384 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": { 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stdout: "a09c4c4a-082b-4b9a-9901-14633694a71a": { 2023-07-21T22:16:51.218 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/", 2023-07-21T22:16:51.219 INFO:teuthology.orchestra.run.smithi093.stdout: "tag": "a09c4c4a-082b-4b9a-9901-14633694a71a", 2023-07-21T22:16:51.219 INFO:teuthology.orchestra.run.smithi093.stdout: "options": "recursive,force" 2023-07-21T22:16:51.219 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:16:51.219 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:16:51.219 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:16:51.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 msgr2=0x7fe9480a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 0x7fe9480a3800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1994c0 tx=0x7fe94c009260 comp rx=0 tx=0).stop 2023-07-21T22:16:51.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 msgr2=0x7fe95c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:51.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe95800d950 tx=0x7fe95800dd10 comp rx=0 tx=0).stop 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 shutdown_connections 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe95c10c430 0x7fe95c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9480a1350 0x7fe9480a3800 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe95c0fe880 0x7fe95c197eb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 --2- 172.21.15.93:0/2769062579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe95c0ff190 0x7fe95c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 >> 172.21.15.93:0/2769062579 conn(0x7fe95c0fa4b0 msgr2=0x7fe95c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:51.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 shutdown_connections 2023-07-21T22:16:51.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:16:51.161+0000 7fe9641fc700 1 -- 172.21.15.93:0/2769062579 wait complete. 2023-07-21T22:16:51.222 INFO:tasks.cephfs.filesystem:scrub status for tag:a09c4c4a-082b-4b9a-9901-14633694a71a - {'path': '/', 'tag': 'a09c4c4a-082b-4b9a-9901-14633694a71a', 'options': 'recursive,force'} 2023-07-21T22:16:51.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:51 smithi194 ceph-mon[98288]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-07-21T22:16:51.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:51 smithi134 ceph-mon[97873]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-07-21T22:16:51.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:51 smithi093 ceph-mon[96579]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-07-21T22:16:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:52 smithi194 ceph-mon[98288]: scrub queued for path: / 2023-07-21T22:16:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:52 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:52 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:16:52.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:52 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:52 smithi134 ceph-mon[97873]: scrub queued for path: / 2023-07-21T22:16:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:52 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:52 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:16:52.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:52 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:52 smithi093 ceph-mon[96579]: scrub queued for path: / 2023-07-21T22:16:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:52 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:52 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:16:52.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:52 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:16:53.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:53 smithi194 ceph-mon[98288]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 17 op/s 2023-07-21T22:16:53.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:53 smithi134 ceph-mon[97873]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 17 op/s 2023-07-21T22:16:53.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:53 smithi093 ceph-mon[96579]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.4 KiB/s wr, 17 op/s 2023-07-21T22:16:54.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:54 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:16:54.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:54 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:16:54.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:54 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:16:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.685+0000 7f38e31af700 1 -- 172.21.15.93:0/1819135282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc0fea30 msgr2=0x7f38dc0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.685+0000 7f38e31af700 1 --2- 172.21.15.93:0/1819135282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc0fea30 0x7f38dc0feea0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009990 tx=0x7f38c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.685+0000 7f38e31af700 1 -- 172.21.15.93:0/1819135282 shutdown_connections 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.685+0000 7f38e31af700 1 --2- 172.21.15.93:0/1819135282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0ff3e0 0x7f38dc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.685+0000 7f38e31af700 1 --2- 172.21.15.93:0/1819135282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc104a30 0x7f38dc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 --2- 172.21.15.93:0/1819135282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc0fea30 0x7f38dc0feea0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- 172.21.15.93:0/1819135282 >> 172.21.15.93:0/1819135282 conn(0x7f38dc074760 msgr2=0x7f38dc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- 172.21.15.93:0/1819135282 shutdown_connections 2023-07-21T22:16:54.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- 172.21.15.93:0/1819135282 wait complete. 2023-07-21T22:16:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 Processor -- start 2023-07-21T22:16:54.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- start start 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0fea30 0x7f38dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc0ff3e0 0x7f38dc19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc194f30 con 0x7f38dc104a30 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc1950a0 con 0x7f38dc0fea30 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.686+0000 7f38e31af700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc195260 con 0x7f38dc0ff3e0 2023-07-21T22:16:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44780/0 (socket says 172.21.15.93:44780) 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 -- 172.21.15.93:0/1431147999 learned_addr learned my addr 172.21.15.93:0/1431147999 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38dbfff700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc0ff3e0 0x7f38dc19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0fea30 0x7f38dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 -- 172.21.15.93:0/1431147999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc0ff3e0 msgr2=0x7f38dc19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc0ff3e0 0x7f38dc19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 -- 172.21.15.93:0/1431147999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0fea30 msgr2=0x7f38dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0fea30 0x7f38dc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 -- 172.21.15.93:0/1431147999 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80095f0 con 0x7f38dc104a30 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e174c700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007f10 tx=0x7f38d400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38d9ffb700 1 -- 172.21.15.93:0/1431147999 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400fd30 con 0x7f38dc104a30 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc195540 con 0x7f38dc104a30 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38d9ffb700 1 -- 172.21.15.93:0/1431147999 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d400fe90 con 0x7f38dc104a30 2023-07-21T22:16:54.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.687+0000 7f38d9ffb700 1 -- 172.21.15.93:0/1431147999 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4005020 con 0x7f38dc104a30 2023-07-21T22:16:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.688+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc067ae0 con 0x7f38dc104a30 2023-07-21T22:16:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 Processor -- start 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 -- start start 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07ace9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07ace3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abe9c50 con 0x55b07acea660 2023-07-21T22:16:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac3c9e0 con 0x55b07ab406b0 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38ef334b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ac35e70 con 0x55b07acea210 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.691+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44786/0 (socket says 172.21.15.93:44786) 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 -- 172.21.15.93:0/1925029265 learned_addr learned my addr 172.21.15.93:0/1925029265 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07ace9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07ace3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 -- 172.21.15.93:0/1925029265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 msgr2=0x55b07ace3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07ace3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 -- 172.21.15.93:0/1925029265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 msgr2=0x55b07ace9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07ace9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 -- 172.21.15.93:0/1925029265 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b07aa63d80 con 0x55b07acea660 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38dbfff700 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38c8003730 tx=0x7f38c8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b697367c28bacf6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38c6ffd700 1 -- 172.21.15.93:0/1925029265 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003890 con 0x55b07acea660 2023-07-21T22:16:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38c6ffd700 1 -- 172.21.15.93:0/1925029265 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c801e070 con 0x55b07acea660 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.692+0000 7f38c6ffd700 1 -- 172.21.15.93:0/1925029265 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800f460 con 0x55b07acea660 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38d9ffb700 1 -- 172.21.15.93:0/1431147999 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f38d4012460 con 0x7f38dc104a30 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- 172.21.15.93:0/1925029265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 msgr2=0x55b07ace44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38c8003730 tx=0x7f38c8003680 comp rx=0 tx=0).stop 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- 172.21.15.93:0/1925029265 shutdown_connections 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07ace44a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07ace9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 --2- 172.21.15.93:0/1925029265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07ace3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- 172.21.15.93:0/1925029265 >> 172.21.15.93:0/1925029265 conn(0x55b07acd9ae0 msgr2=0x55b07acdf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- 172.21.15.93:0/1925029265 shutdown_connections 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- 172.21.15.93:0/1925029265 wait complete. 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38d9ffb700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 0x7f38cc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 0x7f38cc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38ef334b80 1 -- start start 2023-07-21T22:16:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.693+0000 7f38d9ffb700 1 -- 172.21.15.93:0/1431147999 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f38d40ccee0 con 0x7f38dc104a30 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 Processor -- start 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 0x7f38cc0a3710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0ffba0 tx=0x7f38d000b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07acf05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07aced7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abd5a60 con 0x55b07acea660 2023-07-21T22:16:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07abd5760 con 0x55b07ab406b0 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07aab98c0 con 0x55b07acea210 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38ef334b80 10 client.0 _ll_register_callbacks cb 0x55b07abd3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44794/0 (socket says 172.21.15.93:44794) 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 -- 172.21.15.93:0/3085329475 learned_addr learned my addr 172.21.15.93:0/3085329475 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07acf05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38dbfff700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07aced7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 -- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 msgr2=0x55b07aced7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07aced7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.694+0000 7f38e174c700 1 -- 172.21.15.93:0/3085329475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 msgr2=0x55b07acf05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38e174c700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07acf05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38e174c700 1 -- 172.21.15.93:0/3085329475 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc10c160 con 0x55b07acea660 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38e174c700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38d400f910 tx=0x7f38d409dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d2b35abeb89d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40caa40 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f38d409a0c0 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b07ab8f490 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38d409a280 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 10 client.15708 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b07aa4cc00 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d40b6080 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38d40adb60 con 0x55b07acea660 2023-07-21T22:16:54.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 20 client.15708 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07ad01e40 0x55b07ad04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 4 client.15708 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.695+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b07ac2ee50 con 0x55b07ad01e40 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.696+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07ad01e40 0x55b07ad04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.696+0000 7f38bbfff700 10 client.15708 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.696+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07ad01e40 0x55b07ad04220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e482247aa66281f server_cookie=fb7118488e21f282 in_seq=0 out_seq=0 2023-07-21T22:16:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38bbfff700 1 -- 172.21.15.93:0/3085329475 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b07ac2ee50 con 0x55b07ad01e40 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38bbfff700 10 client.15708 handle_command_reply: tid=0 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 client.15708 shutdown 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 msgr2=0x55b07acedf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38d400f910 tx=0x7f38d409dfc0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 shutdown_connections 2023-07-21T22:16:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07ad01e40 0x55b07ad04220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea660 0x55b07acedf10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07acf05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 --2- 172.21.15.93:0/3085329475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acea210 0x55b07aced7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 >> 172.21.15.93:0/3085329475 conn(0x55b07acd9ae0 msgr2=0x55b07acde3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38bbfff700 0 client.15708 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 shutdown_connections 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 1 -- 172.21.15.93:0/3085329475 wait complete. 2023-07-21T22:16:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.697+0000 7f38ef334b80 20 client.15708 trim_cache size 0 max 16384 2023-07-21T22:16:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.712+0000 7f38ef334b80 1 Processor -- start 2023-07-21T22:16:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 -- start start 2023-07-21T22:16:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 0x55b07acf8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07aceb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07ace48d0 con 0x55b07acea210 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07acdb4f0 con 0x55b07ab406b0 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38ef334b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07acd6ab0 con 0x55b07acd9ae0 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44802/0 (socket says 172.21.15.93:44802) 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 -- 172.21.15.93:0/980827088 learned_addr learned my addr 172.21.15.93:0/980827088 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 0x55b07acf8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38dbfff700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07aceb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 -- 172.21.15.93:0/980827088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 msgr2=0x55b07acf8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 0x55b07acf8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 -- 172.21.15.93:0/980827088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 msgr2=0x55b07aceb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07aceb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 -- 172.21.15.93:0/980827088 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b07aa8a1f0 con 0x55b07acea210 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38dbfff700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07aceb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 0x55b07acf8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.713+0000 7f38e174c700 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38d4099b60 tx=0x7f38d4099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f062b3d1fea6059 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38c77fe700 1 -- 172.21.15.93:0/980827088 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40ca4d0 con 0x55b07acea210 2023-07-21T22:16:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38c77fe700 1 -- 172.21.15.93:0/980827088 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b07ab8f490 con 0x55b07acea210 2023-07-21T22:16:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38c77fe700 1 -- 172.21.15.93:0/980827088 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40caaa0 con 0x55b07acea210 2023-07-21T22:16:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 -- 172.21.15.93:0/980827088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 msgr2=0x55b07acf00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38d4099b60 tx=0x7f38d4099890 comp rx=0 tx=0).stop 2023-07-21T22:16:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 -- 172.21.15.93:0/980827088 shutdown_connections 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acf00f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07acd9ae0 0x55b07acf8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 --2- 172.21.15.93:0/980827088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07ab406b0 0x55b07aceb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 -- 172.21.15.93:0/980827088 >> 172.21.15.93:0/980827088 conn(0x55b07acee080 msgr2=0x55b07aceaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.714+0000 7f38ef334b80 1 -- 172.21.15.93:0/980827088 shutdown_connections 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 -- 172.21.15.93:0/980827088 wait complete. 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 -- start start 2023-07-21T22:16:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 Processor -- start 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07ab406b0 0x55b07aced080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 0x55b07aced5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07acec270 con 0x55b07acea210 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07acec3e0 con 0x55b07acd9ae0 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b07acec5a0 con 0x55b07ab406b0 2023-07-21T22:16:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.715+0000 7f38ef334b80 10 client.0 _ll_register_callbacks cb 0x7f38d40997e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44808/0 (socket says 172.21.15.93:44808) 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 -- 172.21.15.93:0/2866129533 learned_addr learned my addr 172.21.15.93:0/2866129533 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07ab406b0 0x55b07aced080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38dbfff700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 0x55b07aced5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 -- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07ab406b0 msgr2=0x55b07aced080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07ab406b0 0x55b07aced080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 -- 172.21.15.93:0/2866129533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 msgr2=0x55b07aced5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 0x55b07aced5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 -- 172.21.15.93:0/2866129533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d0003770 con 0x55b07acea210 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38dbfff700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 0x55b07aced5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38e174c700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cac50 tx=0x7f38d40094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f14cc6c60a0ead server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4099550 con 0x55b07acea210 2023-07-21T22:16:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f38d40a1e20 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38c801e070 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38d40ca4d0 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38ef334b80 10 client.15714 fetch_fsmap learned FSMap version 26 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b07aa4cc00 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.716+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c801e070 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f38d4098dc0 con 0x55b07acea210 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:16:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 20 client.15714 populate_metadata read hostname 'smithi093' 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07acea660 0x55b07ad58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 4 client.15714 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b07abf0ff0 con 0x55b07acea660 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07acea660 0x55b07ad58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38bbfff700 10 client.15714 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.717+0000 7f38e0f4b700 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07acea660 0x55b07ad58670 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9376262495ae16da server_cookie=fcf1a38d448e84ff in_seq=0 out_seq=0 2023-07-21T22:16:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38bbfff700 1 -- 172.21.15.93:0/2866129533 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b07abf0ff0 con 0x55b07acea660 2023-07-21T22:16:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38bbfff700 10 client.15714 handle_command_reply: tid=0 2023-07-21T22:16:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 client.15714 shutdown 2023-07-21T22:16:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 msgr2=0x55b07acebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cac50 tx=0x7f38d40094d0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 shutdown_connections 2023-07-21T22:16:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b07acea660 0x55b07ad58670 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b07acea210 0x55b07acebd30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b07ab406b0 0x55b07aced080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 --2- 172.21.15.93:0/2866129533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07acd9ae0 0x55b07aced5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 >> 172.21.15.93:0/2866129533 conn(0x55b07acee080 msgr2=0x55b07aceb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38bbfff700 0 client.15714 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:16:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 shutdown_connections 2023-07-21T22:16:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 1 -- 172.21.15.93:0/2866129533 wait complete. 2023-07-21T22:16:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.722+0000 7f38ef334b80 20 client.15714 trim_cache size 0 max 16384 2023-07-21T22:16:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:16:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:16:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:16:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 msgr2=0x7f38cc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 0x7f38cc0a3710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0ffba0 tx=0x7f38d000b410 comp rx=0 tx=0).stop 2023-07-21T22:16:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 msgr2=0x7f38dc1948c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007f10 tx=0x7f38d400f5c0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 shutdown_connections 2023-07-21T22:16:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f38cc0a1260 0x7f38cc0a3710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc104a30 0x7f38dc1948c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38dc0fea30 0x7f38dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 --2- 172.21.15.93:0/1431147999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f38dc0ff3e0 0x7f38dc19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 >> 172.21.15.93:0/1431147999 conn(0x7f38dc074760 msgr2=0x7f38dc0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 shutdown_connections 2023-07-21T22:16:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:16:54.723+0000 7f38e31af700 1 -- 172.21.15.93:0/1431147999 wait complete. 2023-07-21T22:16:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:55 smithi194 ceph-mon[98288]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:55 smithi134 ceph-mon[97873]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:55 smithi093 ceph-mon[96579]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:57 smithi134 ceph-mon[97873]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:57.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:57 smithi093 ceph-mon[96579]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:57.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:57 smithi194 ceph-mon[98288]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2023-07-21T22:16:59.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:16:59 smithi134 ceph-mon[97873]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-07-21T22:16:59.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:16:59 smithi093 ceph-mon[96579]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-07-21T22:16:59.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:16:59 smithi194 ceph-mon[98288]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 14 op/s 2023-07-21T22:17:00.941 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:17:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:17:00.510+0000 7fb65101e700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:17:00.942 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:17:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:17:00.510+0000 7fb65101e700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:17:00.942 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:17:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:17:00.510+0000 7fb65101e700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:17:00.942 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:17:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:17:00.510+0000 7fb65101e700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:17:00.942 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:17:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:17:00.510+0000 7fb65101e700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:17:01.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:01 smithi134 ceph-mon[97873]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:17:01.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:01 smithi093 ceph-mon[96579]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:17:01.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:01 smithi194 ceph-mon[98288]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:17:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:03 smithi134 ceph-mon[97873]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-07-21T22:17:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:03 smithi134 ceph-mon[97873]: mgrmap e23: x(active, since 37s), standbys: z, y 2023-07-21T22:17:03.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:03 smithi093 ceph-mon[96579]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-07-21T22:17:03.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:03 smithi093 ceph-mon[96579]: mgrmap e23: x(active, since 37s), standbys: z, y 2023-07-21T22:17:03.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:03 smithi194 ceph-mon[98288]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2023-07-21T22:17:03.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:03 smithi194 ceph-mon[98288]: mgrmap e23: x(active, since 37s), standbys: z, y 2023-07-21T22:17:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:05 smithi134 ceph-mon[97873]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:05.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:05 smithi093 ceph-mon[96579]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:05.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:05 smithi194 ceph-mon[98288]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:07.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:07 smithi134 ceph-mon[97873]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:07.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:07 smithi093 ceph-mon[96579]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:07.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:07 smithi194 ceph-mon[98288]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.893+0000 7fd63ca58700 1 -- 172.21.15.93:0/3838279526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd638104ad0 msgr2=0x7fd638104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.893+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3838279526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd638104ad0 0x7fd638104ea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd628009990 tx=0x7fd628009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 -- 172.21.15.93:0/3838279526 shutdown_connections 2023-07-21T22:17:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3838279526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff470 0x7fd63810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3838279526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd638104ad0 0x7fd638104ea0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3838279526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380feac0 0x7fd6380fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 -- 172.21.15.93:0/3838279526 >> 172.21.15.93:0/3838279526 conn(0x7fd638075bd0 msgr2=0x7fd638075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 -- 172.21.15.93:0/3838279526 shutdown_connections 2023-07-21T22:17:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.894+0000 7fd63ca58700 1 -- 172.21.15.93:0/3838279526 wait complete. 2023-07-21T22:17:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.895+0000 7fd63ca58700 1 Processor -- start 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.895+0000 7fd63ca58700 1 -- start start 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 0x7fd63819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638104ad0 0x7fd6381959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638196030 con 0x7fd6380feac0 2023-07-21T22:17:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381961a0 con 0x7fd6380ff470 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63ca58700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638196360 con 0x7fd638104ad0 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44836/0 (socket says 172.21.15.93:44836) 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 -- 172.21.15.93:0/3974059259 learned_addr learned my addr 172.21.15.93:0/3974059259 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd636d9d700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638104ad0 0x7fd6381959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 0x7fd63819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 -- 172.21.15.93:0/3974059259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638104ad0 msgr2=0x7fd6381959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638104ad0 0x7fd6381959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 -- 172.21.15.93:0/3974059259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 msgr2=0x7fd63819c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 0x7fd63819c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 -- 172.21.15.93:0/3974059259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280095f0 con 0x7fd6380feac0 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 0x7fd63819c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.896+0000 7fd63659c700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd628003730 tx=0x7fd62800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.897+0000 7fd6277fe700 1 -- 172.21.15.93:0/3974059259 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62801d070 con 0x7fd6380feac0 2023-07-21T22:17:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.897+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6381965e0 con 0x7fd6380feac0 2023-07-21T22:17:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.897+0000 7fd6277fe700 1 -- 172.21.15.93:0/3974059259 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62800fb80 con 0x7fd6380feac0 2023-07-21T22:17:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.897+0000 7fd6277fe700 1 -- 172.21.15.93:0/3974059259 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280173f0 con 0x7fd6380feac0 2023-07-21T22:17:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.897+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63810c5d0 con 0x7fd6380feac0 2023-07-21T22:17:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.900+0000 7fd6277fe700 1 -- 172.21.15.93:0/3974059259 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd628022470 con 0x7fd6380feac0 2023-07-21T22:17:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.901+0000 7fd6277fe700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 0x7fd6200a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.901+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 0x7fd6200a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.901+0000 7fd6277fe700 1 -- 172.21.15.93:0/3974059259 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd6280ce380 con 0x7fd6380feac0 2023-07-21T22:17:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.901+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 0x7fd6200a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c0058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.901+0000 7fd648bddb80 1 Processor -- start 2023-07-21T22:17:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 -- start start 2023-07-21T22:17:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38dabcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38da64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38cabc50 con 0x561d38dac210 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38cfe9e0 con 0x561d38dac660 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd648bddb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38cf7e70 con 0x561d38c026b0 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd635d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd63659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38dabcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd635d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44846/0 (socket says 172.21.15.93:44846) 2023-07-21T22:17:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd635d9b700 1 -- 172.21.15.93:0/3775511045 learned_addr learned my addr 172.21.15.93:0/3775511045 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.902+0000 7fd636d9d700 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38da64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 -- 172.21.15.93:0/3775511045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 msgr2=0x561d38dabcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38dabcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 -- 172.21.15.93:0/3775511045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 msgr2=0x561d38da64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38da64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 -- 172.21.15.93:0/3775511045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d38b25d80 con 0x561d38dac210 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd63659c700 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38dabcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd635d9b700 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00fed0 tx=0x7fd62c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc346d29bb3e802 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd624ff9700 1 -- 172.21.15.93:0/3775511045 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c017040 con 0x561d38dac210 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd624ff9700 1 -- 172.21.15.93:0/3775511045 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c003d40 con 0x561d38dac210 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.903+0000 7fd624ff9700 1 -- 172.21.15.93:0/3775511045 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01fcd0 con 0x561d38dac210 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 -- 172.21.15.93:0/3775511045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 msgr2=0x561d38da5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00fed0 tx=0x7fd62c012000 comp rx=0 tx=0).stop 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 -- 172.21.15.93:0/3775511045 shutdown_connections 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38da64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38dabcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 --2- 172.21.15.93:0/3775511045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38da5f40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 -- 172.21.15.93:0/3775511045 >> 172.21.15.93:0/3775511045 conn(0x561d38d9bae0 msgr2=0x561d38da10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 -- 172.21.15.93:0/3775511045 shutdown_connections 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.904+0000 7fd648bddb80 1 -- 172.21.15.93:0/3775511045 wait complete. 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 -- start start 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 Processor -- start 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38dafe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38c97a60 con 0x561d38dac210 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38c97760 con 0x561d38dac660 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38b7b8c0 con 0x561d38c026b0 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd648bddb80 10 client.0 _ll_register_callbacks cb 0x561d38c95700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd635d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd635d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44852/0 (socket says 172.21.15.93:44852) 2023-07-21T22:17:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.905+0000 7fd635d9b700 1 -- 172.21.15.93:0/1161504609 learned_addr learned my addr 172.21.15.93:0/1161504609 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd63659c700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd636d9d700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38dafe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 -- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 msgr2=0x561d38db2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 -- 172.21.15.93:0/1161504609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 msgr2=0x561d38dafe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38dafe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 -- 172.21.15.93:0/1161504609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63810c5d0 con 0x561d38dac210 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85d7ab980d52003 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd61dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01fc80 con 0x561d38dac210 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd62c01f460 con 0x561d38dac210 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d38c51490 con 0x561d38dac210 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd62c01fde0 con 0x561d38dac210 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd648bddb80 10 client.15732 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.906+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fd62c00fbb0 con 0x561d38dac210 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d38c51490 con 0x561d38dac210 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd62c01d040 con 0x561d38dac210 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 20 client.15732 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38dbfa50 0x561d38dc1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 4 client.15732 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d38cf0e50 con 0x561d38dbfa50 2023-07-21T22:17:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.907+0000 7fd63659c700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38dbfa50 0x561d38dc1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.908+0000 7fd61dffb700 10 client.15732 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.908+0000 7fd63659c700 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38dbfa50 0x561d38dc1e60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8d6ec3c090c002 server_cookie=6ba144c6a2d12b86 in_seq=0 out_seq=0 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.908+0000 7fd61dffb700 1 -- 172.21.15.93:0/1161504609 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561d38cf0e50 con 0x561d38dbfa50 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.908+0000 7fd61dffb700 10 client.15732 handle_command_reply: tid=0 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.908+0000 7fd648bddb80 1 client.15732 shutdown 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 msgr2=0x561d38daf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c0039e0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 shutdown_connections 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38dbfa50 0x561d38dc1e60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac660 0x561d38dafe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd61dffb700 0 client.15732 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1161504609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac210 0x561d38daf7c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 >> 172.21.15.93:0/1161504609 conn(0x561d38d9bae0 msgr2=0x561d38da03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 shutdown_connections 2023-07-21T22:17:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 1 -- 172.21.15.93:0/1161504609 wait complete. 2023-07-21T22:17:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.909+0000 7fd648bddb80 20 client.15732 trim_cache size 0 max 16384 2023-07-21T22:17:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 Processor -- start 2023-07-21T22:17:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 -- start start 2023-07-21T22:17:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac210 0x561d38dad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da6d30 con 0x561d38dac660 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da7160 con 0x561d38dac210 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd648bddb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da1d50 con 0x561d38c026b0 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd636d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd636d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44860/0 (socket says 172.21.15.93:44860) 2023-07-21T22:17:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd636d9d700 1 -- 172.21.15.93:0/1628668506 learned_addr learned my addr 172.21.15.93:0/1628668506 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac210 0x561d38dad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.937+0000 7fd63659c700 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 -- 172.21.15.93:0/1628668506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 msgr2=0x561d38db6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 -- 172.21.15.93:0/1628668506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac210 msgr2=0x561d38dad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac210 0x561d38dad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 -- 172.21.15.93:0/1628668506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d38b0ec00 con 0x561d38dac660 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd636d9d700 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd630009f40 tx=0x7fd630009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9075e6c1972c1828 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd6257fa700 1 -- 172.21.15.93:0/1628668506 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630009a90 con 0x561d38dac660 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd6257fa700 1 -- 172.21.15.93:0/1628668506 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63000f040 con 0x561d38dac660 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.938+0000 7fd6257fa700 1 -- 172.21.15.93:0/1628668506 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630016800 con 0x561d38dac660 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 -- 172.21.15.93:0/1628668506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 msgr2=0x561d38da0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd630009f40 tx=0x7fd630009f70 comp rx=0 tx=0).stop 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 -- 172.21.15.93:0/1628668506 shutdown_connections 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da0b60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38c026b0 0x561d38db6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1628668506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38dac210 0x561d38dad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 -- 172.21.15.93:0/1628668506 >> 172.21.15.93:0/1628668506 conn(0x561d38d9bae0 msgr2=0x561d38dacaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 -- 172.21.15.93:0/1628668506 shutdown_connections 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.939+0000 7fd648bddb80 1 -- 172.21.15.93:0/1628668506 wait complete. 2023-07-21T22:17:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 -- start start 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 Processor -- start 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38dac210 0x561d38da7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da9210 con 0x561d38dac660 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da9380 con 0x561d38c026b0 2023-07-21T22:17:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38da81b0 con 0x561d38dac210 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd648bddb80 10 client.0 _ll_register_callbacks cb 0x7fd630003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd63659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd63659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56026/0 (socket says 172.21.15.93:56026) 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd63659c700 1 -- 172.21.15.93:0/1592112894 learned_addr learned my addr 172.21.15.93:0/1592112894 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd636d9d700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.940+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38dac210 0x561d38da7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 -- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38dac210 msgr2=0x561d38da7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38dac210 0x561d38da7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 -- 172.21.15.93:0/1592112894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 msgr2=0x561d38da7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 -- 172.21.15.93:0/1592112894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd630009bf0 con 0x561d38c026b0 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd63659c700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cba80 tx=0x7fd628009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6037727c2f9e36c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd61dffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280a3670 con 0x561d38c026b0 2023-07-21T22:17:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd62c003d40 con 0x561d38c026b0 2023-07-21T22:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd628022720 con 0x561d38c026b0 2023-07-21T22:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6280a10b0 con 0x561d38c026b0 2023-07-21T22:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd648bddb80 10 client.25346 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561d38b4c1f0 con 0x561d38c026b0 2023-07-21T22:17:07.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.941+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c003d40 con 0x561d38c026b0 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd62809a7a0 con 0x561d38c026b0 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 10 client.25346 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 20 client.25346 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38e14680 0x561d38e16a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 4 client.25346 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:17:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d38cb2ff0 con 0x561d38e14680 2023-07-21T22:17:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38e14680 0x561d38e16a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd61dffb700 10 client.25346 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.942+0000 7fd635d9b700 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38e14680 0x561d38e16a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce0045dbafe1eea server_cookie=77e117a3d62b849e in_seq=0 out_seq=0 2023-07-21T22:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd61dffb700 1 -- 172.21.15.93:0/1592112894 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d38cb2ff0 con 0x561d38e14680 2023-07-21T22:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd61dffb700 10 client.25346 handle_command_reply: tid=0 2023-07-21T22:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 client.25346 shutdown 2023-07-21T22:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 msgr2=0x561d38da8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cba80 tx=0x7fd628009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 shutdown_connections 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561d38e14680 0x561d38e16a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561d38dac660 0x561d38da7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d38c026b0 0x561d38da8bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 --2- 172.21.15.93:0/1592112894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d38dac210 0x561d38da7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 >> 172.21.15.93:0/1592112894 conn(0x561d38d9bae0 msgr2=0x561d38dad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd61dffb700 0 client.25346 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 shutdown_connections 2023-07-21T22:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 1 -- 172.21.15.93:0/1592112894 wait complete. 2023-07-21T22:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.951+0000 7fd648bddb80 20 client.25346 trim_cache size 0 max 16384 2023-07-21T22:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:17:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:17:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:17:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:17:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 msgr2=0x7fd6200a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 0x7fd6200a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 msgr2=0x7fd63819bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd628003730 tx=0x7fd62800f690 comp rx=0 tx=0).stop 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 shutdown_connections 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638104ad0 0x7fd6381959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd6200a13e0 0x7fd6200a3890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6380feac0 0x7fd63819bb60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 --2- 172.21.15.93:0/3974059259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6380ff470 0x7fd63819c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 >> 172.21.15.93:0/3974059259 conn(0x7fd638075bd0 msgr2=0x7fd6381026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 shutdown_connections 2023-07-21T22:17:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:07.952+0000 7fd63ca58700 1 -- 172.21.15.93:0/3974059259 wait complete. 2023-07-21T22:17:09.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:09 smithi134 ceph-mon[97873]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2023-07-21T22:17:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:09 smithi093 ceph-mon[96579]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2023-07-21T22:17:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:09 smithi194 ceph-mon[98288]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2023-07-21T22:17:11.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:11 smithi134 ceph-mon[97873]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:11.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:11 smithi093 ceph-mon[96579]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:11.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:11 smithi194 ceph-mon[98288]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-21T22:17:13.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:13 smithi134 ceph-mon[97873]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-07-21T22:17:13.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:13 smithi093 ceph-mon[96579]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-07-21T22:17:13.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:13 smithi194 ceph-mon[98288]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-07-21T22:17:15.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:15 smithi134 ceph-mon[97873]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:15.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:15 smithi093 ceph-mon[96579]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:15.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:15 smithi194 ceph-mon[98288]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:17.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:17 smithi134 ceph-mon[97873]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:17 smithi093 ceph-mon[96579]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:17.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:17 smithi194 ceph-mon[98288]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-21T22:17:19.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:19 smithi093 ceph-mon[96579]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2023-07-21T22:17:19.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:19 smithi194 ceph-mon[98288]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2023-07-21T22:17:19.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:19 smithi134 ceph-mon[97873]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2023-07-21T22:17:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 -- 172.21.15.93:0/1845084426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9700682b0 msgr2=0x7fe97010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 --2- 172.21.15.93:0/1845084426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9700682b0 0x7fe97010c000 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe964009990 tx=0x7fe964009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 -- 172.21.15.93:0/1845084426 shutdown_connections 2023-07-21T22:17:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 --2- 172.21.15.93:0/1845084426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe97010c540 0x7fe97010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 --2- 172.21.15.93:0/1845084426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 0x7fe970067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 --2- 172.21.15.93:0/1845084426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9700682b0 0x7fe97010c000 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 -- 172.21.15.93:0/1845084426 >> 172.21.15.93:0/1845084426 conn(0x7fe970074760 msgr2=0x7fe970074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 -- 172.21.15.93:0/1845084426 shutdown_connections 2023-07-21T22:17:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 -- 172.21.15.93:0/1845084426 wait complete. 2023-07-21T22:17:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.071+0000 7fe974d71700 1 Processor -- start 2023-07-21T22:17:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 -- start start 2023-07-21T22:17:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 0x7fe970197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9700682b0 0x7fe970198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019cf40 con 0x7fe97010c540 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019d0b0 con 0x7fe9700682b0 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe974d71700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019d270 con 0x7fe9700679a0 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44882/0 (socket says 172.21.15.93:44882) 2023-07-21T22:17:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 0x7fe970197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 -- 172.21.15.93:0/1350154590 learned_addr learned my addr 172.21.15.93:0/1350154590 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9700682b0 0x7fe970198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 -- 172.21.15.93:0/1350154590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 msgr2=0x7fe970197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 0x7fe970197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 -- 172.21.15.93:0/1350154590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9700682b0 msgr2=0x7fe970198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9700682b0 0x7fe970198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 -- 172.21.15.93:0/1350154590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe97010c540 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.072+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe9680094f0 tx=0x7fe96800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.073+0000 7fe95f7fe700 1 -- 172.21.15.93:0/1350154590 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968004db0 con 0x7fe97010c540 2023-07-21T22:17:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.073+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97019d550 con 0x7fe97010c540 2023-07-21T22:17:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.073+0000 7fe95f7fe700 1 -- 172.21.15.93:0/1350154590 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96800f040 con 0x7fe97010c540 2023-07-21T22:17:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.073+0000 7fe95f7fe700 1 -- 172.21.15.93:0/1350154590 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968005980 con 0x7fe97010c540 2023-07-21T22:17:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.073+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9701a4d90 con 0x7fe97010c540 2023-07-21T22:17:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.075+0000 7fe95f7fe700 1 -- 172.21.15.93:0/1350154590 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fe968005020 con 0x7fe97010c540 2023-07-21T22:17:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.076+0000 7fe95f7fe700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 0x7fe9580a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.076+0000 7fe96e59c700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 0x7fe9580a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.076+0000 7fe95f7fe700 1 -- 172.21.15.93:0/1350154590 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe9680cc8c0 con 0x7fe97010c540 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.076+0000 7fe96e59c700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 0x7fe9580a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ac50 tx=0x7fe96000a380 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.077+0000 7fe980ef6b80 1 Processor -- start 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 -- start start 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c210 0x56141e965f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e86bc50 con 0x56141e7c26b0 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e8be9e0 con 0x56141e96c660 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e8b7e70 con 0x56141e96c210 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44886/0 (socket says 172.21.15.93:44886) 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 -- 172.21.15.93:0/1433331215 learned_addr learned my addr 172.21.15.93:0/1433331215 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c210 0x56141e965f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 -- 172.21.15.93:0/1433331215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c210 msgr2=0x56141e965f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c210 0x56141e965f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 -- 172.21.15.93:0/1433331215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 msgr2=0x56141e9664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.078+0000 7fe96e59c700 1 -- 172.21.15.93:0/1433331215 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e6e5d80 con 0x56141e7c26b0 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe96e59c700 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ff90 tx=0x7fe960012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1b2e85d2ad7806 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe95cff9700 1 -- 172.21.15.93:0/1433331215 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960003a90 con 0x56141e7c26b0 2023-07-21T22:17:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe95cff9700 1 -- 172.21.15.93:0/1433331215 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe960003bf0 con 0x56141e7c26b0 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe95cff9700 1 -- 172.21.15.93:0/1433331215 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96001d660 con 0x56141e7c26b0 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe980ef6b80 1 -- 172.21.15.93:0/1433331215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 msgr2=0x56141e96bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.079+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ff90 tx=0x7fe960012000 comp rx=0 tx=0).stop 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- 172.21.15.93:0/1433331215 shutdown_connections 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e96bcd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/1433331215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c210 0x56141e965f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- 172.21.15.93:0/1433331215 >> 172.21.15.93:0/1433331215 conn(0x56141e95bae0 msgr2=0x56141e9610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- 172.21.15.93:0/1433331215 shutdown_connections 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- 172.21.15.93:0/1433331215 wait complete. 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- start start 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 Processor -- start 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 0x56141e975bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e96f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e857a60 con 0x56141e7c26b0 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e857760 con 0x56141e96c210 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e73b8c0 con 0x56141e96c660 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe980ef6b80 10 client.0 _ll_register_callbacks cb 0x56141e855700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe96e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44892/0 (socket says 172.21.15.93:44892) 2023-07-21T22:17:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.080+0000 7fe96e59c700 1 -- 172.21.15.93:0/2599775705 learned_addr learned my addr 172.21.15.93:0/2599775705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 0x56141e975bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 -- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 msgr2=0x56141e96f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e96f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e96f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 -- 172.21.15.93:0/2599775705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 msgr2=0x56141e975bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 0x56141e975bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 -- 172.21.15.93:0/2599775705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96000fc40 con 0x56141e7c26b0 2023-07-21T22:17:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 0x56141e975bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe96e59c700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe96000fa30 tx=0x7fe960003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8a83f7b5926eab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe955ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96002b070 con 0x56141e7c26b0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe960010430 con 0x56141e7c26b0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56141e811490 con 0x56141e7c26b0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9600105f0 con 0x56141e7c26b0 2023-07-21T22:17:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe980ef6b80 10 client.15750 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56141e6cec00 con 0x56141e7c26b0 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.081+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96003c020 con 0x56141e7c26b0 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe96001dca0 con 0x56141e7c26b0 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 20 client.15750 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e97f9f0 0x56141e981dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 4 client.15750 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56141e8b0e50 con 0x56141e97f9f0 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e97f9f0 0x56141e981dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe955ffb700 10 client.15750 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.082+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e97f9f0 0x56141e981dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f9f2434f38e514 server_cookie=4c4b418f104e7531 in_seq=0 out_seq=0 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe955ffb700 1 -- 172.21.15.93:0/2599775705 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56141e8b0e50 con 0x56141e97f9f0 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe955ffb700 10 client.15750 handle_command_reply: tid=0 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 client.15750 shutdown 2023-07-21T22:17:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 msgr2=0x56141e9756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe96000fa30 tx=0x7fe960003f90 comp rx=0 tx=0).stop 2023-07-21T22:17:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 shutdown_connections 2023-07-21T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e97f9f0 0x56141e981dd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e96f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e7c26b0 0x56141e9756b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2599775705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c210 0x56141e975bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe955ffb700 0 client.15750 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 >> 172.21.15.93:0/2599775705 conn(0x56141e95bae0 msgr2=0x56141e9603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 shutdown_connections 2023-07-21T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2599775705 wait complete. 2023-07-21T22:17:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.083+0000 7fe980ef6b80 20 client.15750 trim_cache size 0 max 16384 2023-07-21T22:17:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 Processor -- start 2023-07-21T22:17:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 -- start start 2023-07-21T22:17:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e7c26b0 0x56141e96b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e9610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e96b920 con 0x56141e96c210 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e962760 con 0x56141e7c26b0 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e9615e0 con 0x56141e96c660 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.098+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44902/0 (socket says 172.21.15.93:44902) 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 -- 172.21.15.93:0/2505044143 learned_addr learned my addr 172.21.15.93:0/2505044143 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96e59c700 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e7c26b0 0x56141e96b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e9610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 -- 172.21.15.93:0/2505044143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 msgr2=0x56141e9610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e9610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 -- 172.21.15.93:0/2505044143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e7c26b0 msgr2=0x56141e96b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e7c26b0 0x56141e96b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 -- 172.21.15.93:0/2505044143 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141e70c1f0 con 0x56141e96c210 2023-07-21T22:17:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe96400b5c0 tx=0x7fe964005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce2c2d8a3421f7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe95d7fa700 1 -- 172.21.15.93:0/2505044143 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96401d040 con 0x56141e96c210 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe95d7fa700 1 -- 172.21.15.93:0/2505044143 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96401e040 con 0x56141e96c210 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.099+0000 7fe95d7fa700 1 -- 172.21.15.93:0/2505044143 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964003d50 con 0x56141e96c210 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2505044143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 msgr2=0x56141e960b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe96400b5c0 tx=0x7fe964005960 comp rx=0 tx=0).stop 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2505044143 shutdown_connections 2023-07-21T22:17:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e96c660 0x56141e9610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e7c26b0 0x56141e96b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/2505044143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e960b60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2505044143 >> 172.21.15.93:0/2505044143 conn(0x56141e95bae0 msgr2=0x56141e96ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2505044143 shutdown_connections 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.100+0000 7fe980ef6b80 1 -- 172.21.15.93:0/2505044143 wait complete. 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 -- start start 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 Processor -- start 2023-07-21T22:17:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e7c26b0 0x56141e9c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e9cc110 con 0x56141e96c210 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e9cc280 con 0x56141e96c660 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141e9cc440 con 0x56141e7c26b0 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe980ef6b80 10 client.0 _ll_register_callbacks cb 0x7fe9680953e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44908/0 (socket says 172.21.15.93:44908) 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe96dd9b700 1 -- 172.21.15.93:0/3146152804 learned_addr learned my addr 172.21.15.93:0/3146152804 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe96ed9d700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.101+0000 7fe96e59c700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e7c26b0 0x56141e9c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 -- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e7c26b0 msgr2=0x56141e9c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e7c26b0 0x56141e9c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 -- 172.21.15.93:0/3146152804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 msgr2=0x56141e9cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 -- 172.21.15.93:0/3146152804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97010f420 con 0x56141e96c210 2023-07-21T22:17:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe96dd9b700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe9640098e0 tx=0x7fe96400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542e4ba58618eb49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96401d030 con 0x56141e96c210 2023-07-21T22:17:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe964028220 con 0x56141e96c210 2023-07-21T22:17:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe960003bf0 con 0x56141e96c210 2023-07-21T22:17:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9640283e0 con 0x56141e96c210 2023-07-21T22:17:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe980ef6b80 10 client.15756 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56141e6e4250 con 0x56141e96c210 2023-07-21T22:17:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96403e020 con 0x56141e96c210 2023-07-21T22:17:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.102+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe964005500 con 0x56141e96c210 2023-07-21T22:17:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 20 client.15756 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e9d49f0 0x56141e9d6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 4 client.15756 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56141e872ff0 con 0x56141e9d49f0 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe96e59c700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e9d49f0 0x56141e9d6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe96e59c700 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e9d49f0 0x56141e9d6e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1ba6af104c69ee server_cookie=3424ebb2f7497387 in_seq=0 out_seq=0 2023-07-21T22:17:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.103+0000 7fe955ffb700 10 client.15756 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe955ffb700 1 -- 172.21.15.93:0/3146152804 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56141e872ff0 con 0x56141e9d49f0 2023-07-21T22:17:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe955ffb700 10 client.15756 handle_command_reply: tid=0 2023-07-21T22:17:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 client.15756 shutdown 2023-07-21T22:17:21.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 msgr2=0x56141e9c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe9640098e0 tx=0x7fe96400ba10 comp rx=0 tx=0).stop 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 shutdown_connections 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56141e9d49f0 0x56141e9d6e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56141e96c660 0x56141e9cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe955ffb700 0 client.15756 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56141e7c26b0 0x56141e9c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 --2- 172.21.15.93:0/3146152804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56141e96c210 0x56141e9c7f10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.107+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 >> 172.21.15.93:0/3146152804 conn(0x56141e95bae0 msgr2=0x56141e96d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 shutdown_connections 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe980ef6b80 1 -- 172.21.15.93:0/3146152804 wait complete. 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe980ef6b80 20 client.15756 trim_cache size 0 max 16384 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:17:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:17:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:17:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:17:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 msgr2=0x7fe9580a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 0x7fe9580a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ac50 tx=0x7fe96000a380 comp rx=0 tx=0).stop 2023-07-21T22:17:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 msgr2=0x7fe97019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe9680094f0 tx=0x7fe96800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 shutdown_connections 2023-07-21T22:17:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9580a13e0 0x7fe9580a3890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe97010c540 0x7fe97019c900 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9700679a0 0x7fe970197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 --2- 172.21.15.93:0/1350154590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9700682b0 0x7fe970198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 >> 172.21.15.93:0/1350154590 conn(0x7fe970074760 msgr2=0x7fe9700fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 shutdown_connections 2023-07-21T22:17:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:21.108+0000 7fe974d71700 1 -- 172.21.15.93:0/1350154590 wait complete. 2023-07-21T22:17:21.223 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:17:21.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/931583625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94100d00 msgr2=0x7f1e941010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/931583625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94100d00 0x7f1e941010d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009990 tx=0x7f1e90009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/931583625 shutdown_connections 2023-07-21T22:17:21.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/931583625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e9410a0d0 0x7f1e9410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/931583625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94100d00 0x7f1e941010d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/931583625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e94101610 0x7f1e94109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/931583625 >> 172.21.15.93:0/931583625 conn(0x7f1e940fa530 msgr2=0x7f1e940fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/931583625 shutdown_connections 2023-07-21T22:17:21.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/931583625 wait complete. 2023-07-21T22:17:21.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 Processor -- start 2023-07-21T22:17:21.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.324+0000 7f1e9c2fe700 1 -- start start 2023-07-21T22:17:21.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e94100d00 0x7f1e9419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 0x7f1e94195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941960c0 con 0x7f1e94101610 2023-07-21T22:17:21.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94196230 con 0x7f1e9410a0d0 2023-07-21T22:17:21.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941963f0 con 0x7f1e94100d00 2023-07-21T22:17:21.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44922/0 (socket says 172.21.15.93:44922) 2023-07-21T22:17:21.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 -- 172.21.15.93:0/3979128382 learned_addr learned my addr 172.21.15.93:0/3979128382 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 0x7f1e94195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e94100d00 0x7f1e9419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 -- 172.21.15.93:0/3979128382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e94100d00 msgr2=0x7f1e9419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e94100d00 0x7f1e9419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 -- 172.21.15.93:0/3979128382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 msgr2=0x7f1e94195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 0x7f1e94195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 -- 172.21.15.93:0/3979128382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900095f0 con 0x7f1e94101610 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 0x7f1e94195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e99899700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400d980 tx=0x7f1e8400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e8b7fe700 1 -- 172.21.15.93:0/3979128382 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400bb60 con 0x7f1e94101610 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e941966d0 con 0x7f1e94101610 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e8b7fe700 1 -- 172.21.15.93:0/3979128382 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e84007640 con 0x7f1e94101610 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e8b7fe700 1 -- 172.21.15.93:0/3979128382 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84004e90 con 0x7f1e94101610 2023-07-21T22:17:21.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.325+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941a29d0 con 0x7f1e94101610 2023-07-21T22:17:21.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 Processor -- start 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 -- start start 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d95580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d90da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1c95cb0 con 0x5626c1d951b0 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1ce8a40 con 0x5626c1d95ac0 2023-07-21T22:17:21.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1ce1ed0 con 0x5626c1d90930 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.329+0000 7f1e9a09a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56090/0 (socket says 172.21.15.93:56090) 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 -- 172.21.15.93:0/348642641 learned_addr learned my addr 172.21.15.93:0/348642641 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d95580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 -- 172.21.15.93:0/348642641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 msgr2=0x5626c1d90da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d90da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 -- 172.21.15.93:0/348642641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 msgr2=0x5626c1d95580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d95580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 -- 172.21.15.93:0/348642641 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c1b0fde0 con 0x5626c1d95ac0 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e90005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4389284b3545528d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e88dac700 1 -- 172.21.15.93:0/348642641 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000b7d0 con 0x5626c1d95ac0 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e88dac700 1 -- 172.21.15.93:0/348642641 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e9001e040 con 0x5626c1d95ac0 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.330+0000 7f1e88dac700 1 -- 172.21.15.93:0/348642641 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900054d0 con 0x5626c1d95ac0 2023-07-21T22:17:21.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1e8b7fe700 1 -- 172.21.15.93:0/3979128382 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f1e8400bcc0 con 0x7f1e94101610 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 -- 172.21.15.93:0/348642641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 msgr2=0x5626c1d90350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e90005e80 comp rx=0 tx=0).stop 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1e8b7fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 0x7f1e800a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 -- 172.21.15.93:0/348642641 shutdown_connections 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d90da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d95580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/348642641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d90350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 -- 172.21.15.93:0/348642641 >> 172.21.15.93:0/348642641 conn(0x5626c1d85ef0 msgr2=0x5626c1d8b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 0x7f1e800a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 -- 172.21.15.93:0/348642641 shutdown_connections 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1e8b7fe700 1 -- 172.21.15.93:0/3979128382 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e840cef80 con 0x7f1e94101610 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.331+0000 7f1ea8483b80 1 -- 172.21.15.93:0/348642641 wait complete. 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 0x7f1e800a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e9419c6e0 tx=0x7f1e8c009040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 -- start start 2023-07-21T22:17:21.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 Processor -- start 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d9cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1c81ac0 con 0x5626c1d951b0 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1c817c0 con 0x5626c1d95ac0 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1b65920 con 0x5626c1d90930 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1ea8483b80 10 client.0 _ll_register_callbacks cb 0x5626c1c7f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44936/0 (socket says 172.21.15.93:44936) 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e99899700 1 -- 172.21.15.93:0/1921455730 learned_addr learned my addr 172.21.15.93:0/1921455730 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.332+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d9cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 -- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 msgr2=0x5626c1d9cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d9cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 -- 172.21.15.93:0/1921455730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 msgr2=0x5626c1d9a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d9a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 -- 172.21.15.93:0/1921455730 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9410d1d0 con 0x5626c1d951b0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e99899700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e840cccf0 tx=0x7f1e8400f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354320b7bf26571f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8409be20 con 0x5626c1d951b0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e840cce40 con 0x5626c1d951b0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626c1c3b4f0 con 0x5626c1d951b0 2023-07-21T22:17:21.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e8409fe50 con 0x5626c1d951b0 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1ea8483b80 10 client.15768 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5626c1af8c60 con 0x5626c1d951b0 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e840ba030 con 0x5626c1d951b0 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e840cc850 con 0x5626c1d951b0 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1ea8483b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.333+0000 7f1ea8483b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:21.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1ea8483b80 20 client.15768 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1daa450 0x5626c1dac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1ea8483b80 4 client.15768 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626c1cdaeb0 con 0x5626c1daa450 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1daa450 0x5626c1dac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1e7dffb700 10 client.15768 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.334+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1daa450 0x5626c1dac860 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc10acd0654373de server_cookie=26bb3f8d03a630c9 in_seq=0 out_seq=0 2023-07-21T22:17:21.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1e7dffb700 1 -- 172.21.15.93:0/1921455730 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626c1cdaeb0 con 0x5626c1daa450 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1e7dffb700 10 client.15768 handle_command_reply: tid=0 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 client.15768 shutdown 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 msgr2=0x5626c1d99e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e840cccf0 tx=0x7f1e8400f440 comp rx=0 tx=0).stop 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 shutdown_connections 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1daa450 0x5626c1dac860 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d95ac0 0x5626c1d9a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d90930 0x5626c1d9cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1921455730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d99e20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 >> 172.21.15.93:0/1921455730 conn(0x5626c1d85ef0 msgr2=0x5626c1d8b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1e7dffb700 0 client.15768 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 shutdown_connections 2023-07-21T22:17:21.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1921455730 wait complete. 2023-07-21T22:17:21.394 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.335+0000 7f1ea8483b80 20 client.15768 trim_cache size 0 max 16384 2023-07-21T22:17:21.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.352+0000 7f1ea8483b80 1 Processor -- start 2023-07-21T22:17:21.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.352+0000 7f1ea8483b80 1 -- start start 2023-07-21T22:17:21.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 0x5626c1da5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1dac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1d916d0 con 0x5626c1d90930 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1d9adb0 con 0x5626c1d951b0 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1d9cee0 con 0x5626c1d95ac0 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44942/0 (socket says 172.21.15.93:44942) 2023-07-21T22:17:21.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 -- 172.21.15.93:0/1419283800 learned_addr learned my addr 172.21.15.93:0/1419283800 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1dac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e99899700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 0x5626c1da5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 -- 172.21.15.93:0/1419283800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 msgr2=0x5626c1dac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1dac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 -- 172.21.15.93:0/1419283800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 msgr2=0x5626c1da5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 0x5626c1da5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 -- 172.21.15.93:0/1419283800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c1b36250 con 0x5626c1d90930 2023-07-21T22:17:21.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e99899700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 0x5626c1da5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:21.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1e900098e0 tx=0x7f1e90019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545cb30a526b7dab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e895ad700 1 -- 172.21.15.93:0/1419283800 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000b540 con 0x5626c1d90930 2023-07-21T22:17:21.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e895ad700 1 -- 172.21.15.93:0/1419283800 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e90030160 con 0x5626c1d90930 2023-07-21T22:17:21.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.353+0000 7f1e895ad700 1 -- 172.21.15.93:0/1419283800 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900050b0 con 0x5626c1d90930 2023-07-21T22:17:21.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1419283800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 msgr2=0x5626c1da1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1e900098e0 tx=0x7f1e90019040 comp rx=0 tx=0).stop 2023-07-21T22:17:21.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1419283800 shutdown_connections 2023-07-21T22:17:21.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1dac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d90930 0x5626c1da1550 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/1419283800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d951b0 0x5626c1da5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1419283800 >> 172.21.15.93:0/1419283800 conn(0x5626c1d85ef0 msgr2=0x5626c1d9c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1419283800 shutdown_connections 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- 172.21.15.93:0/1419283800 wait complete. 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.354+0000 7f1ea8483b80 1 -- start start 2023-07-21T22:17:21.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 Processor -- start 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d90930 0x5626c1d9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1df72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1df7820 con 0x5626c1d951b0 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1df7990 con 0x5626c1d90930 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c1df7b50 con 0x5626c1d95ac0 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1ea8483b80 10 client.0 _ll_register_callbacks cb 0x7f1e8409b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44950/0 (socket says 172.21.15.93:44950) 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 -- 172.21.15.93:0/3593883799 learned_addr learned my addr 172.21.15.93:0/3593883799 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d90930 0x5626c1d9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e9a89b700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1df72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 -- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 msgr2=0x5626c1df72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1df72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 -- 172.21.15.93:0/3593883799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d90930 msgr2=0x5626c1d9f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d90930 0x5626c1d9f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 -- 172.21.15.93:0/3593883799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626c1b0fde0 con 0x5626c1d951b0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e99899700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ccc70 tx=0x7f1e8400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124ea3df0896cfe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.355+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84014040 con 0x5626c1d951b0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e840cce40 con 0x5626c1d951b0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e9001e040 con 0x5626c1d951b0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e840cc4b0 con 0x5626c1d951b0 2023-07-21T22:17:21.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 10 client.15774 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5626c1b0e2b0 con 0x5626c1d951b0 2023-07-21T22:17:21.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e9001e040 con 0x5626c1d951b0 2023-07-21T22:17:21.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1e8409ad80 con 0x5626c1d951b0 2023-07-21T22:17:21.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:21.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 20 client.15774 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1dfedd0 0x5626c1e011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 4 client.15774 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626c1c9d050 con 0x5626c1dfedd0 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.356+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1dfedd0 0x5626c1e011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1e7dffb700 10 client.15774 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1e9a09a700 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1dfedd0 0x5626c1e011b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba8527360dbbd91 server_cookie=ef324b0360f04101 in_seq=0 out_seq=0 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1e7dffb700 1 -- 172.21.15.93:0/3593883799 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5626c1c9d050 con 0x5626c1dfedd0 2023-07-21T22:17:21.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1e7dffb700 10 client.15774 handle_command_reply: tid=0 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 client.15774 shutdown 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 msgr2=0x5626c1d9fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1e840ccc70 tx=0x7f1e8400df10 comp rx=0 tx=0).stop 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 shutdown_connections 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626c1dfedd0 0x5626c1e011b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626c1d95ac0 0x5626c1df72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1e7dffb700 0 client.15774 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626c1d90930 0x5626c1d9f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 --2- 172.21.15.93:0/3593883799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626c1d951b0 0x5626c1d9fc60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 >> 172.21.15.93:0/3593883799 conn(0x5626c1d85ef0 msgr2=0x5626c1dac460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 shutdown_connections 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 1 -- 172.21.15.93:0/3593883799 wait complete. 2023-07-21T22:17:21.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.357+0000 7f1ea8483b80 20 client.15774 trim_cache size 0 max 16384 2023-07-21T22:17:21.413 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:21.413 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "no active scrubs running", 2023-07-21T22:17:21.413 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": {} 2023-07-21T22:17:21.413 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:21.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 msgr2=0x7f1e800a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 0x7f1e800a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e9419c6e0 tx=0x7f1e8c009040 comp rx=0 tx=0).stop 2023-07-21T22:17:21.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 msgr2=0x7f1e9419bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400d980 tx=0x7f1e8400dd40 comp rx=0 tx=0).stop 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 shutdown_connections 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9410a0d0 0x7f1e94195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e94100d00 0x7f1e9419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e800a13e0 0x7f1e800a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 --2- 172.21.15.93:0/3979128382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e94101610 0x7f1e9419bf20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 >> 172.21.15.93:0/3979128382 conn(0x7f1e940fa530 msgr2=0x7f1e9410c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 shutdown_connections 2023-07-21T22:17:21.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.358+0000 7f1e9c2fe700 1 -- 172.21.15.93:0/3979128382 wait complete. 2023-07-21T22:17:21.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:17:21.416 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:17:21.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 -- 172.21.15.93:0/4228897516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f252410a110 msgr2=0x7f252410c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 --2- 172.21.15.93:0/4228897516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f252410a110 0x7f252410c550 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b0a0 tx=0x7f251c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 -- 172.21.15.93:0/4228897516 shutdown_connections 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 --2- 172.21.15.93:0/4228897516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f252410a110 0x7f252410c550 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 --2- 172.21.15.93:0/4228897516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2524100d40 0x7f2524101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 --2- 172.21.15.93:0/4228897516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2524101650 0x7f2524109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 -- 172.21.15.93:0/4228897516 >> 172.21.15.93:0/4228897516 conn(0x7f25240fa530 msgr2=0x7f25240fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 -- 172.21.15.93:0/4228897516 shutdown_connections 2023-07-21T22:17:21.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.499+0000 7f252c696700 1 -- 172.21.15.93:0/4228897516 wait complete. 2023-07-21T22:17:21.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 Processor -- start 2023-07-21T22:17:21.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 -- start start 2023-07-21T22:17:21.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2524101650 0x7f25241961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252410a110 0x7f25241995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524199ba0 con 0x7f2524100d40 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524199d10 con 0x7f252410a110 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252c696700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524199ed0 con 0x7f2524101650 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252a432700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252a432700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44962/0 (socket says 172.21.15.93:44962) 2023-07-21T22:17:21.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.500+0000 7f252a432700 1 -- 172.21.15.93:0/3903520612 learned_addr learned my addr 172.21.15.93:0/3903520612 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252ac33700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252410a110 0x7f25241995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 -- 172.21.15.93:0/3903520612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2524101650 msgr2=0x7f25241961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2524101650 0x7f25241961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 -- 172.21.15.93:0/3903520612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252410a110 msgr2=0x7f25241995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252410a110 0x7f25241995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 -- 172.21.15.93:0/3903520612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c009d00 con 0x7f2524100d40 2023-07-21T22:17:21.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252a432700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f252000c920 tx=0x7f252000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520007b00 con 0x7f2524100d40 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252000ce70 con 0x7f2524100d40 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252419a1b0 con 0x7f2524100d40 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200054e0 con 0x7f2524100d40 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.501+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241a2b10 con 0x7f2524100d40 2023-07-21T22:17:21.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.502+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25241049a0 con 0x7f2524100d40 2023-07-21T22:17:21.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.504+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f2520004750 con 0x7f2524100d40 2023-07-21T22:17:21.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.505+0000 7f251b7fe700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 0x7f25100a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.505+0000 7f2529c31700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 0x7f25100a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.505+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f25200ccf20 con 0x7f2524100d40 2023-07-21T22:17:21.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.505+0000 7f2529c31700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 0x7f25100a38e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f2514005ee0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.507+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2520096100 con 0x7f2524100d40 2023-07-21T22:17:21.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:21 smithi093 ceph-mon[96579]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:17:21.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.749+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2524067750 con 0x7f2524100d40 2023-07-21T22:17:21.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.749+0000 7f251b7fe700 1 -- 172.21.15.93:0/3903520612 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10855 (secure 0 0 0) 0x7f2520099720 con 0x7f2524100d40 2023-07-21T22:17:21.793 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:17:21.794 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:14:58.012216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:17:21.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 msgr2=0x7f25100a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 0x7f25100a38e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2514005fd0 tx=0x7f2514005ee0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 msgr2=0x7f2524195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f252000c920 tx=0x7f252000cc30 comp rx=0 tx=0).stop 2023-07-21T22:17:21.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 shutdown_connections 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f252410a110 0x7f25241995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f25100a1430 0x7f25100a38e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2524100d40 0x7f2524195cb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 --2- 172.21.15.93:0/3903520612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2524101650 0x7f25241961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 >> 172.21.15.93:0/3903520612 conn(0x7f25240fa530 msgr2=0x7f2524101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 shutdown_connections 2023-07-21T22:17:21.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.752+0000 7f252c696700 1 -- 172.21.15.93:0/3903520612 wait complete. 2023-07-21T22:17:21.798 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 26 2023-07-21T22:17:21.808 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:17:21.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:21 smithi194 ceph-mon[98288]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:17:21.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:21 smithi134 ceph-mon[97873]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:17:21.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/3183973192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb810c460 msgr2=0x7f7eb810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/3183973192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb810c460 0x7f7eb810e840 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b0a0 tx=0x7f7eb000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/3183973192 shutdown_connections 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/3183973192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb810c460 0x7f7eb810e840 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/3183973192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb80fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/3183973192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 0x7f7eb810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/3183973192 >> 172.21.15.93:0/3183973192 conn(0x7f7eb80fa4e0 msgr2=0x7f7eb80fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/3183973192 shutdown_connections 2023-07-21T22:17:21.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.893+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/3183973192 wait complete. 2023-07-21T22:17:21.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 Processor -- start 2023-07-21T22:17:21.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 -- start start 2023-07-21T22:17:21.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 0x7f7eb8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb810c460 0x7f7eb819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819ceb0 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819d020 con 0x7f7eb810c460 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7ebd7c1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819d1e0 con 0x7f7eb80ff1c0 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7eb6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7eb6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44976/0 (socket says 172.21.15.93:44976) 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/320368285 learned_addr learned my addr 172.21.15.93:0/320368285 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.894+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 0x7f7eb8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb77fe700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb810c460 0x7f7eb819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/320368285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 msgr2=0x7f7eb8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 0x7f7eb8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/320368285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb810c460 msgr2=0x7f7eb819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb810c460 0x7f7eb819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/320368285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0009d00 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d980 tx=0x7f7ea800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7e9ffff700 1 -- 172.21.15.93:0/320368285 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8004220 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7e9ffff700 1 -- 172.21.15.93:0/320368285 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea8007640 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7e9ffff700 1 -- 172.21.15.93:0/320368285 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8005690 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb819d4c0 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.895+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb81a4d00 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.898+0000 7f7e9ffff700 1 -- 172.21.15.93:0/320368285 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f7ea8004750 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.898+0000 7f7e9ffff700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 0x7f7ea00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.898+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 0x7f7ea00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7e9ffff700 1 -- 172.21.15.93:0/320368285 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7ea80ccd80 con 0x7f7eb80fe8b0 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 0x7f7ea00a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb81994e0 tx=0x7f7eac0073d0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 Processor -- start 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 -- start start 2023-07-21T22:17:21.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 0x55d90fd1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd19930 0x55d90fd19da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fc1ecb0 con 0x55d90fd1eac0 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fc71a40 con 0x55d90fd1e1b0 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.899+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fc6aed0 con 0x55d90fd19930 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44986/0 (socket says 172.21.15.93:44986) 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 -- 172.21.15.93:0/1717099611 learned_addr learned my addr 172.21.15.93:0/1717099611 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 0x55d90fd1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 -- 172.21.15.93:0/1717099611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd19930 msgr2=0x55d90fd19da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd19930 0x55d90fd19da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 -- 172.21.15.93:0/1717099611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 msgr2=0x55d90fd1e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 0x55d90fd1e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 -- 172.21.15.93:0/1717099611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d90fa98de0 con 0x55d90fd1eac0 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 0x55d90fd1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010d60 tx=0x7f7eac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a12a78bd522d7ee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7e9d7fa700 1 -- 172.21.15.93:0/1717099611 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac01e420 con 0x55d90fd1eac0 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7e9d7fa700 1 -- 172.21.15.93:0/1717099611 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eac015070 con 0x55d90fd1eac0 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.900+0000 7f7e9d7fa700 1 -- 172.21.15.93:0/1717099611 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac01c850 con 0x55d90fd1eac0 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1717099611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 msgr2=0x55d90fd19350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010d60 tx=0x7f7eac013000 comp rx=0 tx=0).stop 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1717099611 shutdown_connections 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd19930 0x55d90fd19da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1e1b0 0x55d90fd1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1717099611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1eac0 0x55d90fd19350 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1717099611 >> 172.21.15.93:0/1717099611 conn(0x55d90fd0eef0 msgr2=0x55d90fd14830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1717099611 shutdown_connections 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1717099611 wait complete. 2023-07-21T22:17:21.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 -- start start 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.901+0000 7f7ec9946b80 1 Processor -- start 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd25c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fc0aac0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fc0a7c0 con 0x55d90fd19930 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90faee920 con 0x55d90fd1eac0 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7ec9946b80 10 client.0 _ll_register_callbacks cb 0x55d90fc08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44992/0 (socket says 172.21.15.93:44992) 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 -- 172.21.15.93:0/498531273 learned_addr learned my addr 172.21.15.93:0/498531273 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd25c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb77fe700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 -- 172.21.15.93:0/498531273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 msgr2=0x55d90fd235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 -- 172.21.15.93:0/498531273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 msgr2=0x55d90fd25c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd25c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 -- 172.21.15.93:0/498531273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac010a10 con 0x55d90fd1e1b0 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0109b0 tx=0x7f7eac005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f53de80f91ab613 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.902+0000 7f7e9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac02d070 con 0x55d90fd1e1b0 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7eac017070 con 0x55d90fd1e1b0 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d90fbc44f0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7eac0055f0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 10 client.15792 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d90fa81c60 con 0x55d90fd1e1b0 2023-07-21T22:17:21.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d90fbc44f0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7eac01e720 con 0x55d90fd1e1b0 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 10 client.15792 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 20 client.15792 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd33340 0x55d90fd35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 4 client.15792 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:21.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d90fc63eb0 con 0x55d90fd33340 2023-07-21T22:17:21.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.903+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd33340 0x55d90fd35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7e9aff5700 10 client.15792 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:21.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd33340 0x55d90fd35750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3efa8a21691bd615 server_cookie=f2323e2db85b4dde in_seq=0 out_seq=0 2023-07-21T22:17:21.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7e9aff5700 1 -- 172.21.15.93:0/498531273 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d90fc63eb0 con 0x55d90fd33340 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7e9aff5700 10 client.15792 handle_command_reply: tid=0 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 client.15792 shutdown 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 msgr2=0x55d90fd22e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0109b0 tx=0x7f7eac005270 comp rx=0 tx=0).stop 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 shutdown_connections 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd33340 0x55d90fd35750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7e9aff5700 0 client.15792 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd25c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/498531273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd22e60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 >> 172.21.15.93:0/498531273 conn(0x55d90fd0eef0 msgr2=0x55d90fd14420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 shutdown_connections 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.904+0000 7f7ec9946b80 1 -- 172.21.15.93:0/498531273 wait complete. 2023-07-21T22:17:21.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.905+0000 7f7ec9946b80 20 client.15792 trim_cache size 0 max 16384 2023-07-21T22:17:21.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.917+0000 7f7ec9946b80 1 Processor -- start 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.917+0000 7f7ec9946b80 1 -- start start 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd17fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd0e9b0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd11010 con 0x55d90fd19930 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd15b30 con 0x55d90fd1eac0 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45000/0 (socket says 172.21.15.93:45000) 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 -- 172.21.15.93:0/3443300732 learned_addr learned my addr 172.21.15.93:0/3443300732 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb77fe700 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd17fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 -- 172.21.15.93:0/3443300732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 msgr2=0x55d90fd17fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd17fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 -- 172.21.15.93:0/3443300732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 msgr2=0x55d90fd2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 -- 172.21.15.93:0/3443300732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d90fabf250 con 0x55d90fd1e1b0 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.918+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01ead0 tx=0x7f7eac0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5224bfacc30a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7e9dffb700 1 -- 172.21.15.93:0/3443300732 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0055f0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7e9dffb700 1 -- 172.21.15.93:0/3443300732 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eac02e020 con 0x55d90fd1e1b0 2023-07-21T22:17:21.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7e9dffb700 1 -- 172.21.15.93:0/3443300732 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac003e70 con 0x55d90fd1e1b0 2023-07-21T22:17:21.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 -- 172.21.15.93:0/3443300732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 msgr2=0x55d90fd17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01ead0 tx=0x7f7eac0052e0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 -- 172.21.15.93:0/3443300732 shutdown_connections 2023-07-21T22:17:21.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1eac0 0x55d90fd17fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd19930 0x55d90fd2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/3443300732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd1e1b0 0x55d90fd17a60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 -- 172.21.15.93:0/3443300732 >> 172.21.15.93:0/3443300732 conn(0x55d90fd0eef0 msgr2=0x55d90fd142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 -- 172.21.15.93:0/3443300732 shutdown_connections 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.919+0000 7f7ec9946b80 1 -- 172.21.15.93:0/3443300732 wait complete. 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 -- start start 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 Processor -- start 2023-07-21T22:17:21.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1e1b0 0x55d90fd28c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1eac0 0x55d90fd86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd86740 con 0x55d90fd19930 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd86880 con 0x55d90fd1eac0 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d90fd869c0 con 0x55d90fd1e1b0 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7ec9946b80 10 client.0 _ll_register_callbacks cb 0x7f7eac010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:21.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45004/0 (socket says 172.21.15.93:45004) 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/1517198168 learned_addr learned my addr 172.21.15.93:0/1517198168 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb77fe700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1eac0 0x55d90fd86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1e1b0 0x55d90fd28c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/1517198168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1e1b0 msgr2=0x55d90fd28c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1e1b0 0x55d90fd28c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/1517198168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1eac0 msgr2=0x55d90fd86200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:21.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1eac0 0x55d90fd86200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.920+0000 7f7eb6ffd700 1 -- 172.21.15.93:0/1517198168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d90fa81c60 con 0x55d90fd19930 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7eb6ffd700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8099b60 tx=0x7f7ea80997d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b864877bbe66682d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80b6070 con 0x55d90fd19930 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7ea809dc50 con 0x55d90fd19930 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7eac015070 con 0x55d90fd19930 2023-07-21T22:17:21.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ea80a2070 con 0x55d90fd19930 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 10 client.15798 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d90fa972b0 con 0x55d90fd19930 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea8063080 con 0x55d90fd19930 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ea8099230 con 0x55d90fd19930 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 10 client.15798 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 20 client.15798 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:21.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd88020 0x55d90fd8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 4 client.15798 mds_command: new command op to 14589 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d90fc26050 con 0x55d90fd88020 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.921+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd88020 0x55d90fd8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7e9aff5700 10 client.15798 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7eb67fc700 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd88020 0x55d90fd8a430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e40160a61cde9dff server_cookie=fedeaa211d7029c1 in_seq=0 out_seq=0 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7e9aff5700 1 -- 172.21.15.93:0/1517198168 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d90fc26050 con 0x55d90fd88020 2023-07-21T22:17:21.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7e9aff5700 10 client.15798 handle_command_reply: tid=0 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 client.15798 shutdown 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 msgr2=0x55d90fd28750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8099b60 tx=0x7f7ea80997d0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 shutdown_connections 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d90fd1eac0 0x55d90fd86200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d90fd19930 0x55d90fd28750 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55d90fd88020 0x55d90fd8a430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.976 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 --2- 172.21.15.93:0/1517198168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d90fd1e1b0 0x55d90fd28c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7e9aff5700 0 client.15798 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 >> 172.21.15.93:0/1517198168 conn(0x55d90fd0eef0 msgr2=0x55d90fd142b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 shutdown_connections 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 1 -- 172.21.15.93:0/1517198168 wait complete. 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.922+0000 7f7ec9946b80 20 client.15798 trim_cache size 0 max 16384 2023-07-21T22:17:21.977 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 msgr2=0x7f7ea00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 0x7f7ea00a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb81994e0 tx=0x7f7eac0073d0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 msgr2=0x7f7eb8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d980 tx=0x7f7ea800dc90 comp rx=0 tx=0).stop 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 shutdown_connections 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eb810c460 0x7f7eb819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ea00a13e0 0x7f7ea00a3890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7eb80fe8b0 0x7f7eb8197ed0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 --2- 172.21.15.93:0/320368285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7eb80ff1c0 0x7f7eb8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:21.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 >> 172.21.15.93:0/320368285 conn(0x7f7eb80fa4e0 msgr2=0x7f7eb810d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:21.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 shutdown_connections 2023-07-21T22:17:21.980 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:21.923+0000 7f7ebd7c1700 1 -- 172.21.15.93:0/320368285 wait complete. 2023-07-21T22:17:21.980 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:17:22.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 -- 172.21.15.93:0/1260639968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c0678e0 msgr2=0x7f5f4c067cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.112 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 --2- 172.21.15.93:0/1260639968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c067cb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5f480099c0 tx=0x7f5f48009cd0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 -- 172.21.15.93:0/1260639968 shutdown_connections 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 --2- 172.21.15.93:0/1260639968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c104e00 0x7f5f4c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 --2- 172.21.15.93:0/1260639968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c067cb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 --2- 172.21.15.93:0/1260639968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 -- 172.21.15.93:0/1260639968 >> 172.21.15.93:0/1260639968 conn(0x7f5f4c074b10 msgr2=0x7f5f4c074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 -- 172.21.15.93:0/1260639968 shutdown_connections 2023-07-21T22:17:22.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 -- 172.21.15.93:0/1260639968 wait complete. 2023-07-21T22:17:22.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.068+0000 7f5f53d24700 1 Processor -- start 2023-07-21T22:17:22.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 -- start start 2023-07-21T22:17:22.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c0ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c0ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c100320 con 0x7f5f4c104e00 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a43b0 con 0x7f5f4c068280 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a44f0 con 0x7f5f4c0678e0 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45020/0 (socket says 172.21.15.93:45020) 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 -- 172.21.15.93:0/4135539005 learned_addr learned my addr 172.21.15.93:0/4135539005 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c0ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f512bf700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c0ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 -- 172.21.15.93:0/4135539005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c0678e0 msgr2=0x7f5f4c0ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c0ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 -- 172.21.15.93:0/4135539005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 msgr2=0x7f5f4c0ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c0ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 -- 172.21.15.93:0/4135539005 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480095f0 con 0x7f5f4c104e00 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f512bf700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f522c1700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b970 tx=0x7f5f4400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f42ffd700 1 -- 172.21.15.93:0/4135539005 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400c9b0 con 0x7f5f4c104e00 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f42ffd700 1 -- 172.21.15.93:0/4135539005 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f44014440 con 0x7f5f4c104e00 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.069+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1a4750 con 0x7f5f4c104e00 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.070+0000 7f5f42ffd700 1 -- 172.21.15.93:0/4135539005 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440075d0 con 0x7f5f4c104e00 2023-07-21T22:17:22.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.070+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c1a4be0 con 0x7f5f4c104e00 2023-07-21T22:17:22.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.073+0000 7f5f42ffd700 1 -- 172.21.15.93:0/4135539005 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f5f440145b0 con 0x7f5f4c104e00 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.073+0000 7f5f42ffd700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 0x7f5f380a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.073+0000 7f5f42ffd700 1 -- 172.21.15.93:0/4135539005 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5f44006e40 con 0x7f5f4c104e00 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.073+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 0x7f5f380a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.073+0000 7f5f5fea9b80 1 Processor -- start 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 -- start start 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 0x7f5f380a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f48005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 0x562ce2fdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2ee2cb0 con 0x562ce2fdd930 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2f35a40 con 0x562ce2fe2ac0 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2f2eed0 con 0x562ce2fe21b0 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45028/0 (socket says 172.21.15.93:45028) 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 -- 172.21.15.93:0/193805399 learned_addr learned my addr 172.21.15.93:0/193805399 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f512bf700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 0x562ce2fdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 -- 172.21.15.93:0/193805399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 msgr2=0x562ce2fe2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 -- 172.21.15.93:0/193805399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 msgr2=0x562ce2fdd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 0x562ce2fdd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f522c1700 1 -- 172.21.15.93:0/193805399 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce2d5cde0 con 0x562ce2fdd930 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.074+0000 7f5f512bf700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 0x562ce2fdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f522c1700 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b940 tx=0x7f5f44098f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54e75afaeea2e1b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f37fff700 1 -- 172.21.15.93:0/193805399 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4409d640 con 0x562ce2fdd930 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f37fff700 1 -- 172.21.15.93:0/193805399 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f440a1a60 con 0x562ce2fdd930 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f37fff700 1 -- 172.21.15.93:0/193805399 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44098d60 con 0x562ce2fdd930 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/193805399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 msgr2=0x562ce2fddda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b940 tx=0x7f5f44098f70 comp rx=0 tx=0).stop 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/193805399 shutdown_connections 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fddda0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/193805399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe2ac0 0x562ce2fdd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.075+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/193805399 >> 172.21.15.93:0/193805399 conn(0x562ce2fd2ef0 msgr2=0x562ce2fd8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/193805399 shutdown_connections 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/193805399 wait complete. 2023-07-21T22:17:22.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- start start 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 Processor -- start 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fdd930 0x562ce2fe9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2eceac0 con 0x562ce2fe2ac0 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2ece7c0 con 0x562ce2fdd930 2023-07-21T22:17:22.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2db2920 con 0x562ce2fe21b0 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.076+0000 7f5f5fea9b80 10 client.0 _ll_register_callbacks cb 0x562ce2ecc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45034/0 (socket says 172.21.15.93:45034) 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 -- 172.21.15.93:0/2051402469 learned_addr learned my addr 172.21.15.93:0/2051402469 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f512bf700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fdd930 0x562ce2fe9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 -- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 msgr2=0x562ce2fe6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 -- 172.21.15.93:0/2051402469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fdd930 msgr2=0x562ce2fe9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fdd930 0x562ce2fe9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 -- 172.21.15.93:0/2051402469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c1a4be0 con 0x562ce2fe2ac0 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f522c1700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5f440a1530 tx=0x7f5f4409d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f664a81cbec974 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44098b00 con 0x562ce2fe2ac0 2023-07-21T22:17:22.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5f440c3e40 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ce2e884f0 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f44098c60 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f5fea9b80 10 client.15810 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.077+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562ce2d45c60 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ce2e884f0 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f4400cb10 con 0x562ce2fe2ac0 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 10 client.15810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 20 client.15810 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2ff7120 0x562ce2ff9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 4 client.15810 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ce2f27eb0 con 0x562ce2ff7120 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2ff7120 0x562ce2ff9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f357fa700 10 client.15810 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:22.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.078+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2ff7120 0x562ce2ff9500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a639ba1816eccaa2 server_cookie=6adccccde0a25368 in_seq=0 out_seq=0 2023-07-21T22:17:22.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f357fa700 1 -- 172.21.15.93:0/2051402469 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ce2f27eb0 con 0x562ce2ff7120 2023-07-21T22:17:22.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f357fa700 10 client.15810 handle_command_reply: tid=0 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 client.15810 shutdown 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 msgr2=0x562ce2fe7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5f440a1530 tx=0x7f5f4409d640 comp rx=0 tx=0).stop 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 shutdown_connections 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2ff7120 0x562ce2ff9500 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fe2ac0 0x562ce2fe7610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f357fa700 0 client.15810 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fdd930 0x562ce2fe9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2051402469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 >> 172.21.15.93:0/2051402469 conn(0x562ce2fd2ef0 msgr2=0x562ce2fd8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 shutdown_connections 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.079+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2051402469 wait complete. 2023-07-21T22:17:22.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.080+0000 7f5f5fea9b80 20 client.15810 trim_cache size 0 max 16384 2023-07-21T22:17:22.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.092+0000 7f5f5fea9b80 1 Processor -- start 2023-07-21T22:17:22.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 -- start start 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fd2ef0 0x562ce2fdd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2fd9320 con 0x562ce2fdd930 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2fe7950 con 0x562ce2fd2ef0 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce2ff9b30 con 0x562ce2fe21b0 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45040/0 (socket says 172.21.15.93:45040) 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 -- 172.21.15.93:0/2065787256 learned_addr learned my addr 172.21.15.93:0/2065787256 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fd2ef0 0x562ce2fdd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f522c1700 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 -- 172.21.15.93:0/2065787256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 msgr2=0x562ce2fe9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 -- 172.21.15.93:0/2065787256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fd2ef0 msgr2=0x562ce2fdd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fd2ef0 0x562ce2fdd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 -- 172.21.15.93:0/2065787256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce2fd7230 con 0x562ce2fdd930 2023-07-21T22:17:22.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f512bf700 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c073330 tx=0x7f5f3c00f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a696209dfa18ffbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.093+0000 7f5f40ff9700 1 -- 172.21.15.93:0/2065787256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c014070 con 0x562ce2fdd930 2023-07-21T22:17:22.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f40ff9700 1 -- 172.21.15.93:0/2065787256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c004cf0 con 0x562ce2fdd930 2023-07-21T22:17:22.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f40ff9700 1 -- 172.21.15.93:0/2065787256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0056e0 con 0x562ce2fdd930 2023-07-21T22:17:22.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2065787256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 msgr2=0x562ce2fe9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c073330 tx=0x7f5f3c00f420 comp rx=0 tx=0).stop 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2065787256 shutdown_connections 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fe21b0 0x562ce2fe9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fd2ef0 0x562ce2fdd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/2065787256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fdd930 0x562ce2fe9780 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2065787256 >> 172.21.15.93:0/2065787256 conn(0x562ce2ff3af0 msgr2=0x562ce2fdca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2065787256 shutdown_connections 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.094+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/2065787256 wait complete. 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 -- start start 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 Processor -- start 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 0x562ce2fed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe21b0 0x562ce2fed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce304af20 con 0x562ce2fd2ef0 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce304b060 con 0x562ce2fe21b0 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce304b1f0 con 0x562ce2fdd930 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f5fea9b80 10 client.0 _ll_register_callbacks cb 0x7f5f44099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.095+0000 7f5f51ac0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45044/0 (socket says 172.21.15.93:45044) 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 -- 172.21.15.93:0/812279207 learned_addr learned my addr 172.21.15.93:0/812279207 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f522c1700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe21b0 0x562ce2fed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f512bf700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 0x562ce2fed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 -- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 msgr2=0x562ce2fed160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 0x562ce2fed160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 -- 172.21.15.93:0/812279207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe21b0 msgr2=0x562ce2fed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe21b0 0x562ce2fed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 -- 172.21.15.93:0/812279207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c00c9e0 con 0x562ce2fd2ef0 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f512bf700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 0x562ce2fed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:22.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f51ac0700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005850 tx=0x7f5f48009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f46b6fe1a52bc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f357fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48034d20 con 0x562ce2fd2ef0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5f48034070 con 0x562ce2fd2ef0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f440a1a60 con 0x562ce2fd2ef0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f48034e80 con 0x562ce2fd2ef0 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f5fea9b80 10 client.15816 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562ce2d83250 con 0x562ce2fd2ef0 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.096+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f440a1a60 con 0x562ce2fd2ef0 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f48003950 con 0x562ce2fd2ef0 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 10 client.15816 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 20 client.15816 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2fe2ac0 0x562ce304e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 4 client.15816 mds_command: new command op to 24448 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ce2eea050 con 0x562ce2fe2ac0 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.097+0000 7f5f512bf700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2fe2ac0 0x562ce304e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f357fa700 10 client.15816 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f512bf700 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2fe2ac0 0x562ce304e8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64dc25604ec49d73 server_cookie=d890244d5c419588 in_seq=0 out_seq=0 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f357fa700 1 -- 172.21.15.93:0/812279207 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ce2eea050 con 0x562ce2fe2ac0 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f357fa700 10 client.15816 handle_command_reply: tid=0 2023-07-21T22:17:22.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 client.15816 shutdown 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 msgr2=0x562ce2fecc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5f48005850 tx=0x7f5f48009cd0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 shutdown_connections 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ce2fe21b0 0x562ce2fed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x562ce2fe2ac0 0x562ce304e8d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562ce2fd2ef0 0x562ce2fecc20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 --2- 172.21.15.93:0/812279207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562ce2fdd930 0x562ce2fed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f357fa700 0 client.15816 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:22.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 >> 172.21.15.93:0/812279207 conn(0x562ce2ff3af0 msgr2=0x562ce2fe0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 shutdown_connections 2023-07-21T22:17:22.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 1 -- 172.21.15.93:0/812279207 wait complete. 2023-07-21T22:17:22.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.098+0000 7f5f5fea9b80 20 client.15816 trim_cache size 0 max 16384 2023-07-21T22:17:22.152 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:22.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 msgr2=0x7f5f380a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 0x7f5f380a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f48005fb0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 msgr2=0x7f5f4c1a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b970 tx=0x7f5f4400bd30 comp rx=0 tx=0).stop 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 shutdown_connections 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f380a13e0 0x7f5f380a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f4c104e00 0x7f5f4c1a3e70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f4c0678e0 0x7f5f4c0ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 --2- 172.21.15.93:0/4135539005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c068280 0x7f5f4c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 >> 172.21.15.93:0/4135539005 conn(0x7f5f4c074b10 msgr2=0x7f5f4c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 shutdown_connections 2023-07-21T22:17:22.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.099+0000 7f5f53d24700 1 -- 172.21.15.93:0/4135539005 wait complete. 2023-07-21T22:17:22.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:17:22.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 -- 172.21.15.93:0/1228998089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f102810a1e0 msgr2=0x7f102810c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 --2- 172.21.15.93:0/1228998089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f102810a1e0 0x7f102810c600 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f102000b0a0 tx=0x7f102000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 -- 172.21.15.93:0/1228998089 shutdown_connections 2023-07-21T22:17:22.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 --2- 172.21.15.93:0/1228998089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f102810a1e0 0x7f102810c600 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 --2- 172.21.15.93:0/1228998089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1028100cc0 0x7f1028101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 --2- 172.21.15.93:0/1228998089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10281015d0 0x7f1028109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 -- 172.21.15.93:0/1228998089 >> 172.21.15.93:0/1228998089 conn(0x7f10280fa630 msgr2=0x7f10280fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 -- 172.21.15.93:0/1228998089 shutdown_connections 2023-07-21T22:17:22.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 -- 172.21.15.93:0/1228998089 wait complete. 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.241+0000 7f102d005700 1 Processor -- start 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 -- start start 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10281015d0 0x7f10280ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f102810a1e0 0x7f10281a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280ff860 con 0x7f1028100cc0 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10281a45d0 con 0x7f10281015d0 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102d005700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10281a4710 con 0x7f102810a1e0 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45058/0 (socket says 172.21.15.93:45058) 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 -- 172.21.15.93:0/939418043 learned_addr learned my addr 172.21.15.93:0/939418043 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102759e700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f102810a1e0 0x7f10281a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f102659c700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10281015d0 0x7f10280ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 -- 172.21.15.93:0/939418043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f102810a1e0 msgr2=0x7f10281a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f102810a1e0 0x7f10281a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 -- 172.21.15.93:0/939418043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10281015d0 msgr2=0x7f10280ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10281015d0 0x7f10280ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 -- 172.21.15.93:0/939418043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020009d00 con 0x7f1028100cc0 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.242+0000 7f1026d9d700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f101000b550 tx=0x7f101000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.243+0000 7f101ffff700 1 -- 172.21.15.93:0/939418043 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010011830 con 0x7f1028100cc0 2023-07-21T22:17:22.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.243+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10281a4970 con 0x7f1028100cc0 2023-07-21T22:17:22.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.243+0000 7f101ffff700 1 -- 172.21.15.93:0/939418043 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1010011e50 con 0x7f1028100cc0 2023-07-21T22:17:22.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.243+0000 7f101ffff700 1 -- 172.21.15.93:0/939418043 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101000f2e0 con 0x7f1028100cc0 2023-07-21T22:17:22.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.243+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281a4ea0 con 0x7f1028100cc0 2023-07-21T22:17:22.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.245+0000 7f101ffff700 1 -- 172.21.15.93:0/939418043 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f1010011990 con 0x7f1028100cc0 2023-07-21T22:17:22.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f101ffff700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 0x7f10140a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f102659c700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 0x7f10140a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f101ffff700 1 -- 172.21.15.93:0/939418043 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f10100cc930 con 0x7f1028100cc0 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f102659c700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 0x7f10140a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10281002e0 tx=0x7f1018009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f103918ab80 1 Processor -- start 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.246+0000 7f103918ab80 1 -- start start 2023-07-21T22:17:22.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04c8acb0 con 0x55ed04d8aac0 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04cdda40 con 0x55ed04d8a1b0 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f103918ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04cd6ed0 con 0x55ed04d85930 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45068/0 (socket says 172.21.15.93:45068) 2023-07-21T22:17:22.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 -- 172.21.15.93:0/1733333843 learned_addr learned my addr 172.21.15.93:0/1733333843 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f1026d9d700 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102759e700 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 -- 172.21.15.93:0/1733333843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 msgr2=0x55ed04d85da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d85da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 -- 172.21.15.93:0/1733333843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8a1b0 msgr2=0x55ed04d8a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.247+0000 7f102659c700 1 -- 172.21.15.93:0/1733333843 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed04b04de0 con 0x55ed04d8aac0 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f102659c700 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1018011e10 tx=0x7f1018014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8aa93288aac532 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f101d7fa700 1 -- 172.21.15.93:0/1733333843 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018012bc0 con 0x55ed04d8aac0 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f101d7fa700 1 -- 172.21.15.93:0/1733333843 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1018016070 con 0x55ed04d8aac0 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f101d7fa700 1 -- 172.21.15.93:0/1733333843 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101801d610 con 0x55ed04d8aac0 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f103918ab80 1 -- 172.21.15.93:0/1733333843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 msgr2=0x55ed04d85350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.248+0000 7f103918ab80 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1018011e10 tx=0x7f1018014000 comp rx=0 tx=0).stop 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- 172.21.15.93:0/1733333843 shutdown_connections 2023-07-21T22:17:22.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- 172.21.15.93:0/1733333843 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d85350 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- 172.21.15.93:0/1733333843 >> 172.21.15.93:0/1733333843 conn(0x55ed04d7aef0 msgr2=0x55ed04d80830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- 172.21.15.93:0/1733333843 shutdown_connections 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- 172.21.15.93:0/1733333843 wait complete. 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- start start 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 Processor -- start 2023-07-21T22:17:22.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8aac0 0x55ed04d8f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04c76ac0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04c767c0 con 0x55ed04d85930 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04b5a920 con 0x55ed04d8aac0 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f103918ab80 10 client.0 _ll_register_callbacks cb 0x55ed04c74760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.249+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45074/0 (socket says 172.21.15.93:45074) 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 -- 172.21.15.93:0/684527572 learned_addr learned my addr 172.21.15.93:0/684527572 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f1026d9d700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102759e700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8aac0 0x55ed04d8f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 -- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8aac0 msgr2=0x55ed04d8f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8aac0 0x55ed04d8f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 -- 172.21.15.93:0/684527572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 msgr2=0x55ed04d91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 -- 172.21.15.93:0/684527572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1018011af0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f1026d9d700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f102659c700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f10180097d0 tx=0x7f101801f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461ec1b30203e544 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f100e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101802c070 con 0x55ed04d8a1b0 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1018018070 con 0x55ed04d8a1b0 2023-07-21T22:17:22.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed04c304f0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1018005ce0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f103918ab80 10 client.15828 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.250+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ed04aedc60 con 0x55ed04d8a1b0 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101803e020 con 0x55ed04d8a1b0 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1018004af0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 20 client.15828 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04d9f330 0x55ed04da1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 4 client.15828 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed04ccfeb0 con 0x55ed04d9f330 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.251+0000 7f1026d9d700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04d9f330 0x55ed04da1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f100e7fc700 10 client.15828 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:22.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f1026d9d700 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04d9f330 0x55ed04da1740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb67ae621a5966a server_cookie=4b8827fb81d29546 in_seq=0 out_seq=0 2023-07-21T22:17:22.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f100e7fc700 1 -- 172.21.15.93:0/684527572 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed04ccfeb0 con 0x55ed04d9f330 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f100e7fc700 10 client.15828 handle_command_reply: tid=0 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 client.15828 shutdown 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 msgr2=0x55ed04d8ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f10180097d0 tx=0x7f101801f6a0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 shutdown_connections 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04d9f330 0x55ed04da1740 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8aac0 0x55ed04d8f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f100e7fc700 0 client.15828 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.252+0000 7f103918ab80 1 --2- 172.21.15.93:0/684527572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d8ee20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.253+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 >> 172.21.15.93:0/684527572 conn(0x55ed04d7aef0 msgr2=0x55ed04d80420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.253+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 shutdown_connections 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.253+0000 7f103918ab80 1 -- 172.21.15.93:0/684527572 wait complete. 2023-07-21T22:17:22.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.253+0000 7f103918ab80 20 client.15828 trim_cache size 0 max 16384 2023-07-21T22:17:22.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.265+0000 7f103918ab80 1 Processor -- start 2023-07-21T22:17:22.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 -- start start 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8aac0 0x55ed04d83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d9b430 con 0x55ed04d8a1b0 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d8d3a0 con 0x55ed04d8aac0 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f103918ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d96ac0 con 0x55ed04d85930 2023-07-21T22:17:22.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45082/0 (socket says 172.21.15.93:45082) 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 -- 172.21.15.93:0/3896431257 learned_addr learned my addr 172.21.15.93:0/3896431257 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f1026d9d700 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102759e700 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8aac0 0x55ed04d83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 -- 172.21.15.93:0/3896431257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 msgr2=0x55ed04d96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 -- 172.21.15.93:0/3896431257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8aac0 msgr2=0x55ed04d83a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8aac0 0x55ed04d83a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 -- 172.21.15.93:0/3896431257 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed04b2b250 con 0x55ed04d8a1b0 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.266+0000 7f102659c700 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1018022040 tx=0x7f1018005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e93f30d8c26da41 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.267+0000 7f101dffb700 1 -- 172.21.15.93:0/3896431257 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101802c040 con 0x55ed04d8a1b0 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.267+0000 7f101dffb700 1 -- 172.21.15.93:0/3896431257 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101803e030 con 0x55ed04d8a1b0 2023-07-21T22:17:22.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.267+0000 7f101dffb700 1 -- 172.21.15.93:0/3896431257 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018004950 con 0x55ed04d8a1b0 2023-07-21T22:17:22.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- 172.21.15.93:0/3896431257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 msgr2=0x55ed04d91740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1018022040 tx=0x7f1018005a20 comp rx=0 tx=0).stop 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- 172.21.15.93:0/3896431257 shutdown_connections 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d8aac0 0x55ed04d83a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d85930 0x55ed04d96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- 172.21.15.93:0/3896431257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8a1b0 0x55ed04d91740 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- 172.21.15.93:0/3896431257 >> 172.21.15.93:0/3896431257 conn(0x55ed04d7aef0 msgr2=0x55ed04d9ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- 172.21.15.93:0/3896431257 shutdown_connections 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- 172.21.15.93:0/3896431257 wait complete. 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- start start 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 Processor -- start 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8a1b0 0x55ed04d96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d94bb0 con 0x55ed04d8aac0 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d94d20 con 0x55ed04d85930 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed04d94ee0 con 0x55ed04d8a1b0 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.268+0000 7f103918ab80 10 client.0 _ll_register_callbacks cb 0x7f1010009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45090/0 (socket says 172.21.15.93:45090) 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 -- 172.21.15.93:0/4209098513 learned_addr learned my addr 172.21.15.93:0/4209098513 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f1026d9d700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102659c700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8a1b0 0x55ed04d96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 -- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8a1b0 msgr2=0x55ed04d96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8a1b0 0x55ed04d96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 -- 172.21.15.93:0/4209098513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 msgr2=0x55ed04d95d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d95d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 -- 172.21.15.93:0/4209098513 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed04aedc60 con 0x55ed04d8aac0 2023-07-21T22:17:22.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f102759e700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1028068520 tx=0x7f1020004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2905371b86edd6e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f100e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000e040 con 0x55ed04d8aac0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1020024070 con 0x55ed04d8aac0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1018016070 con 0x55ed04d8aac0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1020005040 con 0x55ed04d8aac0 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f103918ab80 10 client.15834 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.269+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ed04b032b0 con 0x55ed04d8aac0 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1018016070 con 0x55ed04d8aac0 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f102000b0d0 con 0x55ed04d8aac0 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 10 client.15834 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 20 client.15834 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04df42f0 0x55ed04df6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 4 client.15834 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:22.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed04c92050 con 0x55ed04df42f0 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f1026d9d700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04df42f0 0x55ed04df6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f100e7fc700 10 client.15834 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.270+0000 7f1026d9d700 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04df42f0 0x55ed04df6700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2df36cc95909412e server_cookie=2091388739e810ba in_seq=0 out_seq=0 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f100e7fc700 1 -- 172.21.15.93:0/4209098513 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed04c92050 con 0x55ed04df42f0 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f100e7fc700 10 client.15834 handle_command_reply: tid=0 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 client.15834 shutdown 2023-07-21T22:17:22.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 msgr2=0x55ed04d94560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1028068520 tx=0x7f1020004f50 comp rx=0 tx=0).stop 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 shutdown_connections 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed04d8aac0 0x55ed04d94560 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed04d85930 0x55ed04d95d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x55ed04df42f0 0x55ed04df6700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 --2- 172.21.15.93:0/4209098513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed04d8a1b0 0x55ed04d96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f100e7fc700 0 client.15834 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:22.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 >> 172.21.15.93:0/4209098513 conn(0x55ed04d7aef0 msgr2=0x55ed04d83a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 shutdown_connections 2023-07-21T22:17:22.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 1 -- 172.21.15.93:0/4209098513 wait complete. 2023-07-21T22:17:22.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f103918ab80 20 client.15834 trim_cache size 0 max 16384 2023-07-21T22:17:22.325 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:22.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 msgr2=0x7f10140a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 0x7f10140a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10281002e0 tx=0x7f1018009260 comp rx=0 tx=0).stop 2023-07-21T22:17:22.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 msgr2=0x7f10280feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.271+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f101000b550 tx=0x7f101000b860 comp rx=0 tx=0).stop 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 shutdown_connections 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f102810a1e0 0x7f10281a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f10140a13e0 0x7f10140a3890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1028100cc0 0x7f10280feaf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 --2- 172.21.15.93:0/939418043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10281015d0 0x7f10280ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 >> 172.21.15.93:0/939418043 conn(0x7f10280fa630 msgr2=0x7f102810c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 shutdown_connections 2023-07-21T22:17:22.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.272+0000 7f102d005700 1 -- 172.21.15.93:0/939418043 wait complete. 2023-07-21T22:17:22.329 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:17:22.345 INFO:tasks.workunit.client.0.smithi093.stderr:Note: switching to 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'. 2023-07-21T22:17:22.345 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.345 INFO:tasks.workunit.client.0.smithi093.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-21T22:17:22.345 INFO:tasks.workunit.client.0.smithi093.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr:state without impacting any branches by switching back to a branch. 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr: git switch -c 2023-07-21T22:17:22.346 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr:Or undo this operation with: 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr: git switch - 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-07-21T22:17:22.347 INFO:tasks.workunit.client.0.smithi093.stderr:HEAD is now at bffe049d41f temp: trigger builds on shaman 2023-07-21T22:17:22.351 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-21T22:17:22.412 INFO:tasks.workunit.client.0.smithi093.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-21T22:17:22.414 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:17:22.414 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-21T22:17:22.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:22 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3903520612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:22.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/4195083300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c80ff3e0 msgr2=0x7ff9c810b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/4195083300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c810b870 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b0a0 tx=0x7ff9c000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/4195083300 shutdown_connections 2023-07-21T22:17:22.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/4195083300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c810b870 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/4195083300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c8104a40 0x7ff9c8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/4195083300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c80feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/4195083300 >> 172.21.15.93:0/4195083300 conn(0x7ff9c80fa530 msgr2=0x7ff9c80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/4195083300 shutdown_connections 2023-07-21T22:17:22.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.414+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/4195083300 wait complete. 2023-07-21T22:17:22.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 Processor -- start 2023-07-21T22:17:22.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 -- start start 2023-07-21T22:17:22.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c8196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8197960 con 0x7ff9c8104a40 2023-07-21T22:17:22.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81a20a0 con 0x7ff9c80ff3e0 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9ce8ab700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81a2260 con 0x7ff9c80fea30 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9cce48700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9cce48700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45102/0 (socket says 172.21.15.93:45102) 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9cce48700 1 -- 172.21.15.93:0/3192579459 learned_addr learned my addr 172.21.15.93:0/3192579459 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.415+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c8196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 -- 172.21.15.93:0/3192579459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 msgr2=0x7ff9c8196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c8196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 -- 172.21.15.93:0/3192579459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c80ff3e0 msgr2=0x7ff9c8197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c8197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 -- 172.21.15.93:0/3192579459 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0009d00 con 0x7ff9c8104a40 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c8196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:22.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9cce48700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9c57fa700 1 -- 172.21.15.93:0/3192579459 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000e040 con 0x7ff9c8104a40 2023-07-21T22:17:22.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c81a24e0 con 0x7ff9c8104a40 2023-07-21T22:17:22.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9c57fa700 1 -- 172.21.15.93:0/3192579459 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9c000ceb0 con 0x7ff9c8104a40 2023-07-21T22:17:22.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9c57fa700 1 -- 172.21.15.93:0/3192579459 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0005750 con 0x7ff9c8104a40 2023-07-21T22:17:22.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.416+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c81a2960 con 0x7ff9c8104a40 2023-07-21T22:17:22.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.419+0000 7ff9c57fa700 1 -- 172.21.15.93:0/3192579459 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff9c00058b0 con 0x7ff9c8104a40 2023-07-21T22:17:22.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.419+0000 7ff9c57fa700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 0x7ff9b00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.419+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 0x7ff9b00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.419+0000 7ff9c57fa700 1 -- 172.21.15.93:0/3192579459 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff9c00ce890 con 0x7ff9c8104a40 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 0x7ff9b00a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ffb70 tx=0x7ff9bc00b5f0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 Processor -- start 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 -- start start 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a52ad0 0x55f654a3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654941cb0 con 0x55f654a521c0 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654994a40 con 0x55f654a3c930 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65498ded0 con 0x55f654a52ad0 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45106/0 (socket says 172.21.15.93:45106) 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 -- 172.21.15.93:0/2198239563 learned_addr learned my addr 172.21.15.93:0/2198239563 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9cce48700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a52ad0 0x55f654a3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 -- 172.21.15.93:0/2198239563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a52ad0 msgr2=0x55f654a3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a52ad0 0x55f654a3c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.420+0000 7ff9c7fff700 1 -- 172.21.15.93:0/2198239563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 msgr2=0x55f654a3cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a3cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9c7fff700 1 -- 172.21.15.93:0/2198239563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6547bbde0 con 0x55f654a521c0 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9cce48700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:22.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00fd10 tx=0x7ff9bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1a4f1680141191 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9ae7fc700 1 -- 172.21.15.93:0/2198239563 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc011b80 con 0x55f654a521c0 2023-07-21T22:17:22.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9ae7fc700 1 -- 172.21.15.93:0/2198239563 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc011ce0 con 0x55f654a521c0 2023-07-21T22:17:22.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9ae7fc700 1 -- 172.21.15.93:0/2198239563 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc03f820 con 0x55f654a521c0 2023-07-21T22:17:22.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2198239563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 msgr2=0x55f654a52590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.421+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00fd10 tx=0x7ff9bc012000 comp rx=0 tx=0).stop 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2198239563 shutdown_connections 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a521c0 0x55f654a52590 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2198239563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a52ad0 0x55f654a3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2198239563 >> 172.21.15.93:0/2198239563 conn(0x55f654a31ef0 msgr2=0x55f654a37830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2198239563 shutdown_connections 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2198239563 wait complete. 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 -- start start 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.422+0000 7ff9daa30b80 1 Processor -- start 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a521c0 0x55f654a56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65492dac0 con 0x55f654a52ad0 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65492d7c0 con 0x55f654a3c930 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654811920 con 0x55f654a521c0 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 10 client.0 _ll_register_callbacks cb 0x55f65492b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45116/0 (socket says 172.21.15.93:45116) 2023-07-21T22:17:22.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 -- 172.21.15.93:0/961430258 learned_addr learned my addr 172.21.15.93:0/961430258 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a521c0 0x55f654a56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 -- 172.21.15.93:0/961430258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a521c0 msgr2=0x55f654a56ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a521c0 0x55f654a56ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 -- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 msgr2=0x55f654a59c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a59c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 -- 172.21.15.93:0/961430258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc00f9c0 con 0x55f654a52ad0 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9cce48700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009850 tx=0x7ff9c0004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460a1441677719 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c009b7c0 con 0x55f654a52ad0 2023-07-21T22:17:22.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff9c009f7f0 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.423+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6548e74f0 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c00a3b80 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 10 client.15846 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f6547a4c60 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6548e74f0 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff9c009acb0 con 0x55f654a52ad0 2023-07-21T22:17:22.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 20 client.15846 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654a4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 4 client.15846 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f654986eb0 con 0x55f654a4a300 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.424+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654a4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9a77fe700 10 client.15846 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654a4c6e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da54f8f517f907e9 server_cookie=b1b1f432ed697334 in_seq=0 out_seq=0 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9a77fe700 1 -- 172.21.15.93:0/961430258 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f654986eb0 con 0x55f654a4a300 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9a77fe700 10 client.15846 handle_command_reply: tid=0 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 client.15846 shutdown 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 msgr2=0x55f654a57610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009850 tx=0x7ff9c0004f60 comp rx=0 tx=0).stop 2023-07-21T22:17:22.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 shutdown_connections 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654a4c6e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a52ad0 0x55f654a57610 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9a77fe700 0 client.15846 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a3c930 0x55f654a59c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.425+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/961430258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a521c0 0x55f654a56ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.426+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 >> 172.21.15.93:0/961430258 conn(0x55f654a31ef0 msgr2=0x55f654a37420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.426+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 shutdown_connections 2023-07-21T22:17:22.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.426+0000 7ff9daa30b80 1 -- 172.21.15.93:0/961430258 wait complete. 2023-07-21T22:17:22.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.426+0000 7ff9daa30b80 20 client.15846 trim_cache size 0 max 16384 2023-07-21T22:17:22.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.438+0000 7ff9daa30b80 1 Processor -- start 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.438+0000 7ff9daa30b80 1 -- start start 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a57780 0x55f654a4dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a3c930 0x55f654a3c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6548e74f0 con 0x55f654a31ef0 2023-07-21T22:17:22.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654a34010 con 0x55f654a57780 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654a4e6e0 con 0x55f654a3c930 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45122/0 (socket says 172.21.15.93:45122) 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 -- 172.21.15.93:0/661619745 learned_addr learned my addr 172.21.15.93:0/661619745 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9cce48700 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a3c930 0x55f654a3c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 -- 172.21.15.93:0/661619745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a3c930 msgr2=0x55f654a3c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a3c930 0x55f654a3c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 -- 172.21.15.93:0/661619745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a57780 msgr2=0x55f654a4dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a57780 0x55f654a4dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 -- 172.21.15.93:0/661619745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6547e2250 con 0x55f654a31ef0 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800aca0 tx=0x7ff9b8012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0f4a750588c599 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9aeffd700 1 -- 172.21.15.93:0/661619745 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8017070 con 0x55f654a31ef0 2023-07-21T22:17:22.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9aeffd700 1 -- 172.21.15.93:0/661619745 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b8004510 con 0x55f654a31ef0 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.439+0000 7ff9aeffd700 1 -- 172.21.15.93:0/661619745 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8003e70 con 0x55f654a31ef0 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- 172.21.15.93:0/661619745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 msgr2=0x55f654a5e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800aca0 tx=0x7ff9b8012450 comp rx=0 tx=0).stop 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- 172.21.15.93:0/661619745 shutdown_connections 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a3c930 0x55f654a3c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a57780 0x55f654a4dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/661619745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a31ef0 0x55f654a5e330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- 172.21.15.93:0/661619745 >> 172.21.15.93:0/661619745 conn(0x55f654a46c30 msgr2=0x55f654a3ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- 172.21.15.93:0/661619745 shutdown_connections 2023-07-21T22:17:22.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- 172.21.15.93:0/661619745 wait complete. 2023-07-21T22:17:22.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.440+0000 7ff9daa30b80 1 -- start start 2023-07-21T22:17:22.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 Processor -- start 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 0x55f654a5e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 0x55f654a5c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654a5cd60 con 0x55f654a3c930 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654a5cea0 con 0x55f654a31ef0 2023-07-21T22:17:22.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f654a5d060 con 0x55f654a57780 2023-07-21T22:17:22.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9daa30b80 10 client.0 _ll_register_callbacks cb 0x7ff9c0097b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45128/0 (socket says 172.21.15.93:45128) 2023-07-21T22:17:22.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 -- 172.21.15.93:0/2356128174 learned_addr learned my addr 172.21.15.93:0/2356128174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 0x55f654a5e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9cce48700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 0x55f654a5c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 -- 172.21.15.93:0/2356128174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 msgr2=0x55f654a5c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 0x55f654a5c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 -- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 msgr2=0x55f654a5e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 0x55f654a5e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 -- 172.21.15.93:0/2356128174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b800a980 con 0x55f654a3c930 2023-07-21T22:17:22.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 0x55f654a5e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:22.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9cce48700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 0x55f654a5c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:22.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9c77fe700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005920 tx=0x7ff9b8004330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb04b60a18ae9b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.441+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8017030 con 0x55f654a3c930 2023-07-21T22:17:22.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff9b8018070 con 0x55f654a3c930 2023-07-21T22:17:22.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9bc011ce0 con 0x55f654a3c930 2023-07-21T22:17:22.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b801c420 con 0x55f654a3c930 2023-07-21T22:17:22.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 10 client.15852 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55f6547ba2b0 con 0x55f654a3c930 2023-07-21T22:17:22.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b8032020 con 0x55f654a3c930 2023-07-21T22:17:22.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff9b800f640 con 0x55f654a3c930 2023-07-21T22:17:22.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 10 client.15852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:17:22.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 20 client.15852 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654aad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 4 client.15852 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:22.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f654949050 con 0x55f654a4a300 2023-07-21T22:17:22.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.442+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654aad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9a77fe700 10 client.15852 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:22.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9c7fff700 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654aad860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f959e15c92514c25 server_cookie=29826a27bc0593cb in_seq=0 out_seq=0 2023-07-21T22:17:22.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9a77fe700 1 -- 172.21.15.93:0/2356128174 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f654949050 con 0x55f654a4a300 2023-07-21T22:17:22.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9a77fe700 10 client.15852 handle_command_reply: tid=0 2023-07-21T22:17:22.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 client.15852 shutdown 2023-07-21T22:17:22.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 msgr2=0x55f654a5c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005920 tx=0x7ff9b8004330 comp rx=0 tx=0).stop 2023-07-21T22:17:22.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 shutdown_connections 2023-07-21T22:17:22.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55f654a4a300 0x55f654aad860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f654a57780 0x55f654a5c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f654a31ef0 0x55f654a5e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 --2- 172.21.15.93:0/2356128174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f654a3c930 0x55f654a5c290 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9a77fe700 0 client.15852 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:22.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 >> 172.21.15.93:0/2356128174 conn(0x55f654a46c30 msgr2=0x55f654a53fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 shutdown_connections 2023-07-21T22:17:22.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 1 -- 172.21.15.93:0/2356128174 wait complete. 2023-07-21T22:17:22.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.443+0000 7ff9daa30b80 20 client.15852 trim_cache size 0 max 16384 2023-07-21T22:17:22.508 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:22.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 msgr2=0x7ff9b00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 0x7ff9b00a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80ffb70 tx=0x7ff9bc00b5f0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 msgr2=0x7ff9c81a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c0004c80 comp rx=0 tx=0).stop 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 shutdown_connections 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9b00a13e0 0x7ff9b00a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9c8104a40 0x7ff9c81a1b60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c80fea30 0x7ff9c8196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 --2- 172.21.15.93:0/3192579459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9c80ff3e0 0x7ff9c8197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 >> 172.21.15.93:0/3192579459 conn(0x7ff9c80fa530 msgr2=0x7ff9c8105800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 shutdown_connections 2023-07-21T22:17:22.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.444+0000 7ff9ce8ab700 1 -- 172.21.15.93:0/3192579459 wait complete. 2023-07-21T22:17:22.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:17:22.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.629+0000 7ffb2ae25700 1 -- 172.21.15.93:0/307675499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb24104a30 msgr2=0x7ffb24104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.629+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/307675499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb24104a30 0x7ffb24104e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009990 tx=0x7ffb18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.629+0000 7ffb2ae25700 1 -- 172.21.15.93:0/307675499 shutdown_connections 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.629+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/307675499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240ff3d0 0x7ffb2410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/307675499 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb24104a30 0x7ffb24104e00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/307675499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb240fea20 0x7ffb240fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- 172.21.15.93:0/307675499 >> 172.21.15.93:0/307675499 conn(0x7ffb240fa540 msgr2=0x7ffb240fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- 172.21.15.93:0/307675499 shutdown_connections 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- 172.21.15.93:0/307675499 wait complete. 2023-07-21T22:17:22.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 Processor -- start 2023-07-21T22:17:22.676 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- start start 2023-07-21T22:17:22.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240fea20 0x7ffb24194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb24104a30 0x7ffb241952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb241a4090 con 0x7ffb240ff3d0 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.630+0000 7ffb2ae25700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb241a4200 con 0x7ffb240fea20 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb2ae25700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb241a43c0 con 0x7ffb24104a30 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45142/0 (socket says 172.21.15.93:45142) 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 -- 172.21.15.93:0/2788742349 learned_addr learned my addr 172.21.15.93:0/2788742349 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240fea20 0x7ffb24194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb293c2700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb24104a30 0x7ffb241952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 -- 172.21.15.93:0/2788742349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb24104a30 msgr2=0x7ffb241952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb24104a30 0x7ffb241952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 -- 172.21.15.93:0/2788742349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240fea20 msgr2=0x7ffb24194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240fea20 0x7ffb24194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 -- 172.21.15.93:0/2788742349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180095f0 con 0x7ffb240ff3d0 2023-07-21T22:17:22.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d980 tx=0x7ffb1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb21ffb700 1 -- 172.21.15.93:0/2788742349 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000bb60 con 0x7ffb240ff3d0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb21ffb700 1 -- 172.21.15.93:0/2788742349 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb10007640 con 0x7ffb240ff3d0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb21ffb700 1 -- 172.21.15.93:0/2788742349 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004e90 con 0x7ffb240ff3d0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb241a46a0 con 0x7ffb240ff3d0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.631+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb241a4b60 con 0x7ffb240ff3d0 2023-07-21T22:17:22.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.633+0000 7ffb21ffb700 1 -- 172.21.15.93:0/2788742349 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ffb1000bcc0 con 0x7ffb240ff3d0 2023-07-21T22:17:22.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.634+0000 7ffb21ffb700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.634+0000 7ffb21ffb700 1 -- 172.21.15.93:0/2788742349 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb100ccf10 con 0x7ffb240ff3d0 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.634+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.635+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 0x7ffb140a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b5c0 tx=0x7ffb18005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.635+0000 7ffb36faab80 1 Processor -- start 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.635+0000 7ffb36faab80 1 -- start start 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eef1b0 0x556299eef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eefac0 0x556299eea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299defcb0 con 0x556299eea930 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299e42a40 con 0x556299eef1b0 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb36faab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299e3bed0 con 0x556299eefac0 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45152/0 (socket says 172.21.15.93:45152) 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 -- 172.21.15.93:0/1610487328 learned_addr learned my addr 172.21.15.93:0/1610487328 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb23fff700 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eefac0 0x556299eea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eef1b0 0x556299eef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 -- 172.21.15.93:0/1610487328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eefac0 msgr2=0x556299eea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eefac0 0x556299eea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 -- 172.21.15.93:0/1610487328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eef1b0 msgr2=0x556299eef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eef1b0 0x556299eef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 -- 172.21.15.93:0/1610487328 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556299c69de0 con 0x556299eea930 2023-07-21T22:17:22.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb293c2700 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d20 tx=0x7ffb1c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473820a3ceee6969 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb0effd700 1 -- 172.21.15.93:0/1610487328 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018070 con 0x556299eea930 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb0effd700 1 -- 172.21.15.93:0/1610487328 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1c00f050 con 0x556299eea930 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.636+0000 7ffb0effd700 1 -- 172.21.15.93:0/1610487328 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c014410 con 0x556299eea930 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- 172.21.15.93:0/1610487328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 msgr2=0x556299eeada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d20 tx=0x7ffb1c0077e0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- 172.21.15.93:0/1610487328 shutdown_connections 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eea930 0x556299eeada0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eef1b0 0x556299eef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 --2- 172.21.15.93:0/1610487328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eefac0 0x556299eea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- 172.21.15.93:0/1610487328 >> 172.21.15.93:0/1610487328 conn(0x556299edfef0 msgr2=0x556299ee5830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- 172.21.15.93:0/1610487328 shutdown_connections 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- 172.21.15.93:0/1610487328 wait complete. 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 -- start start 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.637+0000 7ffb36faab80 1 Processor -- start 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ddbac0 con 0x556299eefac0 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ddb7c0 con 0x556299eea930 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299cbf920 con 0x556299eef1b0 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb36faab80 10 client.0 _ll_register_callbacks cb 0x556299dd9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45158/0 (socket says 172.21.15.93:45158) 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 -- 172.21.15.93:0/3931669387 learned_addr learned my addr 172.21.15.93:0/3931669387 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb23fff700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 -- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 msgr2=0x556299ef3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 -- 172.21.15.93:0/3931669387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 msgr2=0x556299ef6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 -- 172.21.15.93:0/3931669387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c009980 con 0x556299eefac0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.638+0000 7ffb293c2700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0056f0 tx=0x7ffb1c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1f87c78d9b0487 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018030 con 0x556299eefac0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb1c005420 con 0x556299eefac0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556299d954f0 con 0x556299eefac0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1c006000 con 0x556299eefac0 2023-07-21T22:17:22.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 10 client.15864 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556299c52c60 con 0x556299eefac0 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556299d954f0 con 0x556299eefac0 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb1c014670 con 0x556299eefac0 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 20 client.15864 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f04390 0x556299f067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 4 client.15864 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556299e34eb0 con 0x556299f04390 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.639+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f04390 0x556299f067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.640+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f04390 0x556299f067a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d780feb54db72a3 server_cookie=ab5629981e338ae0 in_seq=0 out_seq=0 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.640+0000 7ffb03fff700 10 client.15864 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.640+0000 7ffb03fff700 1 -- 172.21.15.93:0/3931669387 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556299e34eb0 con 0x556299f04390 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.640+0000 7ffb03fff700 10 client.15864 handle_command_reply: tid=0 2023-07-21T22:17:22.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.640+0000 7ffb36faab80 1 client.15864 shutdown 2023-07-21T22:17:22.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 msgr2=0x556299ef4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0056f0 tx=0x7ffb1c004d00 comp rx=0 tx=0).stop 2023-07-21T22:17:22.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 shutdown_connections 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f04390 0x556299f067a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef4520 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb03fff700 0 client.15864 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 --2- 172.21.15.93:0/3931669387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 >> 172.21.15.93:0/3931669387 conn(0x556299edfef0 msgr2=0x556299ee5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 shutdown_connections 2023-07-21T22:17:22.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 1 -- 172.21.15.93:0/3931669387 wait complete. 2023-07-21T22:17:22.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.641+0000 7ffb36faab80 20 client.15864 trim_cache size 0 max 16384 2023-07-21T22:17:22.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.653+0000 7ffb36faab80 1 Processor -- start 2023-07-21T22:17:22.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.653+0000 7ffb36faab80 1 -- start start 2023-07-21T22:17:22.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ef4b90 con 0x556299eefac0 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ef4880 con 0x556299eea930 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb36faab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299f00e10 con 0x556299eef1b0 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45166/0 (socket says 172.21.15.93:45166) 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 -- 172.21.15.93:0/4014822082 learned_addr learned my addr 172.21.15.93:0/4014822082 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb23fff700 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 -- 172.21.15.93:0/4014822082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 msgr2=0x556299ef3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 -- 172.21.15.93:0/4014822082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 msgr2=0x556299ef6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 -- 172.21.15.93:0/4014822082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556299c90250 con 0x556299eefac0 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb293c2700 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c014a20 tx=0x7ffb1c007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2b953271f4e73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb0f7fe700 1 -- 172.21.15.93:0/4014822082 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018070 con 0x556299eefac0 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb0f7fe700 1 -- 172.21.15.93:0/4014822082 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb1c019040 con 0x556299eefac0 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.654+0000 7ffb0f7fe700 1 -- 172.21.15.93:0/4014822082 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c01e6b0 con 0x556299eefac0 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- 172.21.15.93:0/4014822082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 msgr2=0x556299ef3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c014a20 tx=0x7ffb1c007be0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- 172.21.15.93:0/4014822082 shutdown_connections 2023-07-21T22:17:22.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eefac0 0x556299ef3bc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eea930 0x556299ef6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 --2- 172.21.15.93:0/4014822082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eef1b0 0x556299ef3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- 172.21.15.93:0/4014822082 >> 172.21.15.93:0/4014822082 conn(0x556299edfef0 msgr2=0x556299eead00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- 172.21.15.93:0/4014822082 shutdown_connections 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- 172.21.15.93:0/4014822082 wait complete. 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 -- start start 2023-07-21T22:17:22.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.655+0000 7ffb36faab80 1 Processor -- start 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eea930 0x556299efb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eefac0 0x556299ef9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ef9c40 con 0x556299eef1b0 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ef9db0 con 0x556299eefac0 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556299ef9f70 con 0x556299eea930 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 10 client.0 _ll_register_callbacks cb 0x7ffb18003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:22.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45170/0 (socket says 172.21.15.93:45170) 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 -- 172.21.15.93:0/2008555801 learned_addr learned my addr 172.21.15.93:0/2008555801 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eea930 0x556299efb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb293c2700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eefac0 0x556299ef9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 -- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eea930 msgr2=0x556299efb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eea930 0x556299efb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 -- 172.21.15.93:0/2008555801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eefac0 msgr2=0x556299ef9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:22.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eefac0 0x556299ef9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 -- 172.21.15.93:0/2008555801 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556299c52c60 con 0x556299eef1b0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb23fff700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec043a7463264768 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb03fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a21a0 con 0x556299eef1b0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ffb100a1d10 con 0x556299eef1b0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.656+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb1c00f050 con 0x556299eef1b0 2023-07-21T22:17:22.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb100a1ed0 con 0x556299eef1b0 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 10 client.15870 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556299c682b0 con 0x556299eef1b0 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb100ca020 con 0x556299eef1b0 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffb10095bd0 con 0x556299eef1b0 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 20 client.15870 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:22.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f593e0 0x556299f5b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:22.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 4 client.15870 mds_command: new command op to 24484 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:22.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556299df7050 con 0x556299f593e0 2023-07-21T22:17:22.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.657+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f593e0 0x556299f5b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:22.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb03fff700 10 client.15870 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:22.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb28bc1700 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f593e0 0x556299f5b7f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b9f0ad0fea7b0e server_cookie=dff890f8f00cc934 in_seq=0 out_seq=0 2023-07-21T22:17:22.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb03fff700 1 -- 172.21.15.93:0/2008555801 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556299df7050 con 0x556299f593e0 2023-07-21T22:17:22.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb03fff700 10 client.15870 handle_command_reply: tid=0 2023-07-21T22:17:22.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 client.15870 shutdown 2023-07-21T22:17:22.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 msgr2=0x556299efb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10099e40 comp rx=0 tx=0).stop 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 shutdown_connections 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556299f593e0 0x556299f5b7f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556299eefac0 0x556299ef9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556299eea930 0x556299efb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 --2- 172.21.15.93:0/2008555801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556299eef1b0 0x556299efb5d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb03fff700 0 client.15870 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:22.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 >> 172.21.15.93:0/2008555801 conn(0x556299edfef0 msgr2=0x556299eedd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 shutdown_connections 2023-07-21T22:17:22.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 1 -- 172.21.15.93:0/2008555801 wait complete. 2023-07-21T22:17:22.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.658+0000 7ffb36faab80 20 client.15870 trim_cache size 0 max 16384 2023-07-21T22:17:22.713 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:22.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 msgr2=0x7ffb140a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 0x7ffb140a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800b5c0 tx=0x7ffb18005fb0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 msgr2=0x7ffb24194d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000d980 tx=0x7ffb1000dd40 comp rx=0 tx=0).stop 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 shutdown_connections 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb24104a30 0x7ffb241952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb240fea20 0x7ffb24194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ffb140a13e0 0x7ffb140a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 --2- 172.21.15.93:0/2788742349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb240ff3d0 0x7ffb24194d90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:22.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 >> 172.21.15.93:0/2788742349 conn(0x7ffb240fa540 msgr2=0x7ffb240fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:22.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 shutdown_connections 2023-07-21T22:17:22.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:22.659+0000 7ffb2ae25700 1 -- 172.21.15.93:0/2788742349 wait complete. 2023-07-21T22:17:22.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:22 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3903520612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:22.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:22 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3903520612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:22.977 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-21T22:17:23.033 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-21T22:17:23.070 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:17:23.071 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:17:23.072 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-21T22:17:23.108 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:17:23.111 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:17:23.111 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-21T22:17:23.127 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-07-21T22:17:23.129 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-07-21T22:17:23.129 DEBUG:teuthology.orchestra.run.smithi093:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-07-21T22:17:23.190 INFO:tasks.workunit.client.0.smithi093.stderr:+ dbench 1 2023-07-21T22:17:23.199 INFO:tasks.workunit.client.0.smithi093.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T22:17:23.199 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:17:23.200 INFO:tasks.workunit.client.0.smithi093.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T22:17:23.205 INFO:tasks.workunit.client.0.smithi093.stdout:0 of 1 processes prepared for launch 0 sec 2023-07-21T22:17:23.305 INFO:tasks.workunit.client.0.smithi093.stdout:1 of 1 processes prepared for launch 0 sec 2023-07-21T22:17:23.305 INFO:tasks.workunit.client.0.smithi093.stdout:releasing clients 2023-07-21T22:17:23.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:23 smithi093 ceph-mon[96579]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:17:23.717 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:17:23.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:17:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:23 smithi194 ceph-mon[98288]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:17:23.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/3897309011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 msgr2=0x7fa0c810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/3897309011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c810b810 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000b0a0 tx=0x7fa0c000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/3897309011 shutdown_connections 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/3897309011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c810b810 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/3897309011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 0x7fa0c8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/3897309011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c80fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/3897309011 >> 172.21.15.93:0/3897309011 conn(0x7fa0c80fa4b0 msgr2=0x7fa0c80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/3897309011 shutdown_connections 2023-07-21T22:17:23.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.806+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/3897309011 wait complete. 2023-07-21T22:17:23.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 Processor -- start 2023-07-21T22:17:23.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 -- start start 2023-07-21T22:17:23.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c8100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 0x7fa0c8101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a85e0 con 0x7fa0c80ff380 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a8720 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0cdcaa700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81a8860 con 0x7fa0c81049e0 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56356/0 (socket says 172.21.15.93:56356) 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c8100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1371388972 learned_addr learned my addr 172.21.15.93:0/1371388972 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.807+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 0x7fa0c8101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1371388972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 msgr2=0x7fa0c8101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 0x7fa0c8101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 msgr2=0x7fa0c8100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c8100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1371388972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c0009d00 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00e970 tx=0x7fa0bc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c4ff9700 1 -- 172.21.15.93:0/1371388972 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc009790 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81a8b40 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c4ff9700 1 -- 172.21.15.93:0/1371388972 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc00c7e0 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0c4ff9700 1 -- 172.21.15.93:0/1371388972 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc003c70 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.808+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81a90a0 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.810+0000 7fa0c4ff9700 1 -- 172.21.15.93:0/1371388972 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fa0bc00cc50 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.811+0000 7fa0c4ff9700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 0x7fa0b00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.811+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 0x7fa0b00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.811+0000 7fa0c4ff9700 1 -- 172.21.15.93:0/1371388972 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0bc014070 con 0x7fa0c80fe9d0 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.811+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 0x7fa0b00a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8101dd0 tx=0x7fa0b800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 Processor -- start 2023-07-21T22:17:23.862 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 -- start start 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056a230 0x56225056a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x5622505653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225046ad30 con 0x5622505659b0 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622504bdac0 con 0x56225056ab40 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622504b6f50 con 0x56225056a230 2023-07-21T22:17:23.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x5622505653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0c7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45206/0 (socket says 172.21.15.93:45206) 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0c7fff700 1 -- 172.21.15.93:0/710976460 learned_addr learned my addr 172.21.15.93:0/710976460 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.812+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056a230 0x56225056a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 -- 172.21.15.93:0/710976460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056a230 msgr2=0x56225056a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056a230 0x56225056a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 -- 172.21.15.93:0/710976460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 msgr2=0x5622505653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x5622505653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 -- 172.21.15.93:0/710976460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622502e4e60 con 0x5622505659b0 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x5622505653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009310 tx=0x7fa0c0009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e22caa9c3ede14 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0adffb700 1 -- 172.21.15.93:0/710976460 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000e040 con 0x5622505659b0 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0adffb700 1 -- 172.21.15.93:0/710976460 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0c0003a00 con 0x5622505659b0 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.813+0000 7fa0adffb700 1 -- 172.21.15.93:0/710976460 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c001c6a0 con 0x5622505659b0 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/710976460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 msgr2=0x562250565e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009310 tx=0x7fa0c0009340 comp rx=0 tx=0).stop 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/710976460 shutdown_connections 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250565e20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056a230 0x56225056a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/710976460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x5622505653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/710976460 >> 172.21.15.93:0/710976460 conn(0x56225055af70 msgr2=0x5622505608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/710976460 shutdown_connections 2023-07-21T22:17:23.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/710976460 wait complete. 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 -- start start 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.814+0000 7fa0d9e2fb80 1 Processor -- start 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x56225056ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x56225056f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250456b40 con 0x5622505659b0 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250456840 con 0x56225056a230 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56225033a9a0 con 0x56225056ab40 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 10 client.0 _ll_register_callbacks cb 0x5622504547e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45208/0 (socket says 172.21.15.93:45208) 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 -- 172.21.15.93:0/564101455 learned_addr learned my addr 172.21.15.93:0/564101455 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x56225056f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x56225056ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 -- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 msgr2=0x56225056f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x56225056f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 -- 172.21.15.93:0/564101455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 msgr2=0x56225056ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x56225056ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 -- 172.21.15.93:0/564101455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c810c110 con 0x5622505659b0 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x56225056ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc005950 tx=0x7fa0bc003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87293f40ad4ca054 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:23.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0a23b0 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0bc0b82a0 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562250410570 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.815+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0bc0ca170 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 10 client.15882 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5622502cdce0 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc063080 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0bc0a1f20 con 0x5622505659b0 2023-07-21T22:17:23.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 20 client.15882 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56225057f400 0x562250581810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 4 client.15882 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622504aff30 con 0x56225057f400 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56225057f400 0x562250581810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56225057f400 0x562250581810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5670682d01ce8345 server_cookie=a9eb5fd99018bcdb in_seq=0 out_seq=0 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.816+0000 7fa0a6ffd700 10 client.15882 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.817+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/564101455 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622504aff30 con 0x56225057f400 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.817+0000 7fa0a6ffd700 10 client.15882 handle_command_reply: tid=0 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.817+0000 7fa0d9e2fb80 1 client.15882 shutdown 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.817+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 msgr2=0x562250571ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.817+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc005950 tx=0x7fa0bc003ec0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 shutdown_connections 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56225057f400 0x562250581810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x56225056f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x562250571ca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0a6ffd700 0 client.15882 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/564101455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x56225056ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 >> 172.21.15.93:0/564101455 conn(0x56225055af70 msgr2=0x5622505604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 shutdown_connections 2023-07-21T22:17:23.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/564101455 wait complete. 2023-07-21T22:17:23.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.818+0000 7fa0d9e2fb80 20 client.15882 trim_cache size 0 max 16384 2023-07-21T22:17:23.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 Processor -- start 2023-07-21T22:17:23.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 -- start start 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622505659b0 0x5622505765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x562250571d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250566750 con 0x56225056a230 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250569e80 con 0x56225056ab40 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622505613a0 con 0x5622505659b0 2023-07-21T22:17:23.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45218/0 (socket says 172.21.15.93:45218) 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 -- 172.21.15.93:0/2458334911 learned_addr learned my addr 172.21.15.93:0/2458334911 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x562250571d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622505659b0 0x5622505765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 -- 172.21.15.93:0/2458334911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622505659b0 msgr2=0x5622505765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622505659b0 0x5622505765a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 -- 172.21.15.93:0/2458334911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 msgr2=0x562250571d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x562250571d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 -- 172.21.15.93:0/2458334911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56225030b2d0 con 0x56225056a230 2023-07-21T22:17:23.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.835+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80fdc90 tx=0x7fa0b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a890a280bcf974bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0ae7fc700 1 -- 172.21.15.93:0/2458334911 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b802a070 con 0x56225056a230 2023-07-21T22:17:23.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0ae7fc700 1 -- 172.21.15.93:0/2458334911 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b8015070 con 0x56225056a230 2023-07-21T22:17:23.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0ae7fc700 1 -- 172.21.15.93:0/2458334911 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8024040 con 0x56225056a230 2023-07-21T22:17:23.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:23 smithi134 ceph-mon[97873]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/2458334911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 msgr2=0x5622505717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80fdc90 tx=0x7fa0b8016000 comp rx=0 tx=0).stop 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/2458334911 shutdown_connections 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056ab40 0x562250571d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5622505659b0 0x5622505765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/2458334911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56225056a230 0x5622505717d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/2458334911 >> 172.21.15.93:0/2458334911 conn(0x56225055af70 msgr2=0x562250563b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/2458334911 shutdown_connections 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.836+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/2458334911 wait complete. 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 -- start start 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 Processor -- start 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x5622505cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x562250575580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250575b50 con 0x5622505659b0 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250575cc0 con 0x56225056a230 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562250575e80 con 0x56225056ab40 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0d9e2fb80 10 client.0 _ll_register_callbacks cb 0x7fa0bc099b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0c77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45222/0 (socket says 172.21.15.93:45222) 2023-07-21T22:17:23.888 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1522563048 learned_addr learned my addr 172.21.15.93:0/1522563048 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0c7fff700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x562250575580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.837+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x5622505cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 msgr2=0x562250575580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x562250575580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1522563048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 msgr2=0x5622505cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x5622505cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.889 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 -- 172.21.15.93:0/1522563048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b8011990 con 0x5622505659b0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0c77fe700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00efd0 tx=0x7fa0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414997e908036deb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0ca990 con 0x5622505659b0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0bc0a23b0 con 0x5622505659b0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0c0003a00 con 0x5622505659b0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0bc0ca170 con 0x5622505659b0 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 10 client.15888 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:23.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5622502e3330 con 0x5622505659b0 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0c0003a00 con 0x5622505659b0 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0bc095e70 con 0x5622505659b0 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 20 client.15888 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5622505d5320 0x5622505d7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:23.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 4 client.15888 mds_command: new command op to 24484 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.838+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5622504720d0 con 0x5622505d5320 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.839+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5622505d5320 0x5622505d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.839+0000 7fa0a6ffd700 10 client.15888 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.839+0000 7fa0c6ffd700 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5622505d5320 0x5622505d7700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6776f1d1c8948b server_cookie=84a7f09bc8ad7310 in_seq=0 out_seq=0 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0a6ffd700 1 -- 172.21.15.93:0/1522563048 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5622504720d0 con 0x5622505d5320 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0a6ffd700 10 client.15888 handle_command_reply: tid=0 2023-07-21T22:17:23.892 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 client.15888 shutdown 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 msgr2=0x5622505cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00efd0 tx=0x7fa0bc004750 comp rx=0 tx=0).stop 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 shutdown_connections 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5622505d5320 0x5622505d7700 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56225056ab40 0x562250575580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0a6ffd700 0 client.15888 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5622505659b0 0x5622505cd980 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 --2- 172.21.15.93:0/1522563048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56225056a230 0x5622505cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 >> 172.21.15.93:0/1522563048 conn(0x56225055af70 msgr2=0x56225056e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 shutdown_connections 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 1 -- 172.21.15.93:0/1522563048 wait complete. 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0d9e2fb80 20 client.15888 trim_cache size 0 max 16384 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:23.894 INFO:teuthology.orchestra.run.smithi093.stdout: "return_code": 0, 2023-07-21T22:17:23.895 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_tag": "50d3dd18-848d-48fc-9269-9a6c8d64c2fe", 2023-07-21T22:17:23.895 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": "asynchronous" 2023-07-21T22:17:23.895 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 msgr2=0x7fa0b00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 0x7fa0b00a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8101dd0 tx=0x7fa0b800b410 comp rx=0 tx=0).stop 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.841+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 msgr2=0x7fa0c81005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00e970 tx=0x7fa0bc00ec80 comp rx=0 tx=0).stop 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 shutdown_connections 2023-07-21T22:17:23.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0c81049e0 0x7fa0c8101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c80fe9d0 0x7fa0c81005e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0b00a13e0 0x7fa0b00a3890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 --2- 172.21.15.93:0/1371388972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0c80ff380 0x7fa0c8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 >> 172.21.15.93:0/1371388972 conn(0x7fa0c80fa4b0 msgr2=0x7fa0c8067750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 shutdown_connections 2023-07-21T22:17:23.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.842+0000 7fa0cdcaa700 1 -- 172.21.15.93:0/1371388972 wait complete. 2023-07-21T22:17:23.898 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:17:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 -- 172.21.15.93:0/4128192590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140fea30 msgr2=0x7f99140feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 --2- 172.21.15.93:0/4128192590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140fea30 0x7f99140feea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9904009990 tx=0x7f9904009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 -- 172.21.15.93:0/4128192590 shutdown_connections 2023-07-21T22:17:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 --2- 172.21.15.93:0/4128192590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140ff3e0 0x7f991410b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 --2- 172.21.15.93:0/4128192590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 0x7f9914104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 --2- 172.21.15.93:0/4128192590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140fea30 0x7f99140feea0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.981+0000 7f991c22f700 1 -- 172.21.15.93:0/4128192590 >> 172.21.15.93:0/4128192590 conn(0x7f99140fa530 msgr2=0x7f99140fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- 172.21.15.93:0/4128192590 shutdown_connections 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- 172.21.15.93:0/4128192590 wait complete. 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 Processor -- start 2023-07-21T22:17:24.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- start start 2023-07-21T22:17:24.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140fea30 0x7f99141969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 0x7f99141a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9914197830 con 0x7f99140ff3e0 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99141a1f70 con 0x7f9914104a40 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f991c22f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99141a2130 con 0x7f99140fea30 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f99197ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f99197ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45240/0 (socket says 172.21.15.93:45240) 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f99197ca700 1 -- 172.21.15.93:0/2158771098 learned_addr learned my addr 172.21.15.93:0/2158771098 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.982+0000 7f9919fcb700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140fea30 0x7f99141969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f991a7cc700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 0x7f99141a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 -- 172.21.15.93:0/2158771098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140fea30 msgr2=0x7f99141969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140fea30 0x7f99141969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 -- 172.21.15.93:0/2158771098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 msgr2=0x7f99141a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 0x7f99141a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 -- 172.21.15.93:0/2158771098 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040095f0 con 0x7f99140ff3e0 2023-07-21T22:17:24.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f99197ca700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f9904004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f990affd700 1 -- 172.21.15.93:0/2158771098 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990401d070 con 0x7f99140ff3e0 2023-07-21T22:17:24.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f990affd700 1 -- 172.21.15.93:0/2158771098 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9904021410 con 0x7f99140ff3e0 2023-07-21T22:17:24.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99141a23b0 con 0x7f99140ff3e0 2023-07-21T22:17:24.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f990affd700 1 -- 172.21.15.93:0/2158771098 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904017650 con 0x7f99140ff3e0 2023-07-21T22:17:24.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.983+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141a2860 con 0x7f99140ff3e0 2023-07-21T22:17:24.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.986+0000 7f990affd700 1 -- 172.21.15.93:0/2158771098 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f9904021580 con 0x7f99140ff3e0 2023-07-21T22:17:24.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.986+0000 7f990affd700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 0x7f99000a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.986+0000 7f9919fcb700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 0x7f99000a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.986+0000 7f990affd700 1 -- 172.21.15.93:0/2158771098 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f99040ce230 con 0x7f99140ff3e0 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f9919fcb700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 0x7f99000a3890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99140ffb70 tx=0x7f9910009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 Processor -- start 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 -- start start 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0baca9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 0x55e0baca4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0baba9cb0 con 0x55e0baca4930 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0babfca40 con 0x55e0baca91b0 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f99283b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0babf5ed0 con 0x55e0baca9ac0 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.987+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f99197ca700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 0x55e0baca4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45250/0 (socket says 172.21.15.93:45250) 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 -- 172.21.15.93:0/2676866749 learned_addr learned my addr 172.21.15.93:0/2676866749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:24.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f99197ca700 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 0x55e0baca4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:60560/0 (socket says 172.21.15.93:60560) 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0baca9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 -- 172.21.15.93:0/2676866749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 msgr2=0x55e0baca4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 0x55e0baca4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 -- 172.21.15.93:0/2676866749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 msgr2=0x55e0baca9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0baca9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 -- 172.21.15.93:0/2676866749 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0baa23de0 con 0x55e0baca4930 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f9919fcb700 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0baca9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:24.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f991a7cc700 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f990c011680 tx=0x7f990c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ee0f5ba8aee9ee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f98fffff700 1 -- 172.21.15.93:0/2676866749 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00e040 con 0x55e0baca4930 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f98fffff700 1 -- 172.21.15.93:0/2676866749 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c00f040 con 0x55e0baca4930 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.988+0000 7f98fffff700 1 -- 172.21.15.93:0/2676866749 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004850 con 0x55e0baca4930 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- 172.21.15.93:0/2676866749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 msgr2=0x55e0baca4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f990c011680 tx=0x7f990c009920 comp rx=0 tx=0).stop 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- 172.21.15.93:0/2676866749 shutdown_connections 2023-07-21T22:17:24.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca4930 0x55e0baca4da0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0baca9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 --2- 172.21.15.93:0/2676866749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca9ac0 0x55e0baca4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- 172.21.15.93:0/2676866749 >> 172.21.15.93:0/2676866749 conn(0x55e0bac99ef0 msgr2=0x55e0bac9f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- 172.21.15.93:0/2676866749 shutdown_connections 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- 172.21.15.93:0/2676866749 wait complete. 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 -- start start 2023-07-21T22:17:24.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.989+0000 7f99283b4b80 1 Processor -- start 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bab95ac0 con 0x55e0baca9ac0 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bab957c0 con 0x55e0baca91b0 2023-07-21T22:17:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0baa79920 con 0x55e0baca4930 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99283b4b80 10 client.0 _ll_register_callbacks cb 0x55e0bab93760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99197ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45256/0 (socket says 172.21.15.93:45256) 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99197ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56416/0 (socket says 172.21.15.93:56416) 2023-07-21T22:17:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:60564/0 (socket says 172.21.15.93:60564) 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 -- 172.21.15.93:0/1707814120 learned_addr learned my addr 172.21.15.93:0/1707814120 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 -- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 msgr2=0x55e0bacb0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 -- 172.21.15.93:0/1707814120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 msgr2=0x55e0bacade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 -- 172.21.15.93:0/1707814120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9910017070 con 0x55e0baca9ac0 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f9919fcb700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f99197ca700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f991a7cc700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f990c004350 tx=0x7f990c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf3562c99725b85 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.990+0000 7f98fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00e070 con 0x55e0baca9ac0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0bab4f4f0 con 0x55e0baca9ac0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f990c003c10 con 0x55e0baca9ac0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f990c003e10 con 0x55e0baca9ac0 2023-07-21T22:17:24.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 10 client.15900 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f990c011330 con 0x55e0baca9ac0 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0bab4f4f0 con 0x55e0baca9ac0 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f990c007ce0 con 0x55e0baca9ac0 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 20 client.15900 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bacbe400 0x55e0bacc0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 4 client.15900 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:24.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0babeeeb0 con 0x55e0bacbe400 2023-07-21T22:17:24.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.991+0000 7f9919fcb700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bacbe400 0x55e0bacc0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.992+0000 7f98fd7fa700 10 client.15900 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:24.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.992+0000 7f9919fcb700 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bacbe400 0x55e0bacc0810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c723890e3f6b24cd server_cookie=4deb1a78d482e880 in_seq=0 out_seq=0 2023-07-21T22:17:24.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f98fd7fa700 1 -- 172.21.15.93:0/1707814120 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e0babeeeb0 con 0x55e0bacbe400 2023-07-21T22:17:24.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f98fd7fa700 10 client.15900 handle_command_reply: tid=0 2023-07-21T22:17:24.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 client.15900 shutdown 2023-07-21T22:17:24.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 msgr2=0x55e0bacae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f990c004350 tx=0x7f990c006000 comp rx=0 tx=0).stop 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 shutdown_connections 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bacbe400 0x55e0bacc0810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacae5a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 --2- 172.21.15.93:0/1707814120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f98fd7fa700 0 client.15900 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:24.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 >> 172.21.15.93:0/1707814120 conn(0x55e0bac99ef0 msgr2=0x55e0bac9f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 shutdown_connections 2023-07-21T22:17:24.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 1 -- 172.21.15.93:0/1707814120 wait complete. 2023-07-21T22:17:24.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:23.993+0000 7f99283b4b80 20 client.15900 trim_cache size 0 max 16384 2023-07-21T22:17:24.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.010+0000 7f99283b4b80 1 Processor -- start 2023-07-21T22:17:24.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.010+0000 7f99283b4b80 1 -- start start 2023-07-21T22:17:24.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.010+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.010+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca9ac0 0x55e0baca3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99283b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bacaebc0 con 0x55e0baca91b0 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99283b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bacae900 con 0x55e0baca9ac0 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99283b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0baca0320 con 0x55e0baca4930 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45262/0 (socket says 172.21.15.93:45262) 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 -- 172.21.15.93:0/4006034637 learned_addr learned my addr 172.21.15.93:0/4006034637 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f991a7cc700 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca9ac0 0x55e0baca3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 -- 172.21.15.93:0/4006034637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 msgr2=0x55e0bacb0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 -- 172.21.15.93:0/4006034637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca9ac0 msgr2=0x55e0baca3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca9ac0 0x55e0baca3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 -- 172.21.15.93:0/4006034637 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0baa0cc60 con 0x55e0baca91b0 2023-07-21T22:17:24.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f99197ca700 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9904097b80 tx=0x7f9904009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af0d95188d5f088 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f9908ff9700 1 -- 172.21.15.93:0/4006034637 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990409af30 con 0x55e0baca91b0 2023-07-21T22:17:24.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f9908ff9700 1 -- 172.21.15.93:0/4006034637 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990409aca0 con 0x55e0baca91b0 2023-07-21T22:17:24.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.011+0000 7f9908ff9700 1 -- 172.21.15.93:0/4006034637 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990409a8b0 con 0x55e0baca91b0 2023-07-21T22:17:24.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- 172.21.15.93:0/4006034637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 msgr2=0x55e0baca3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9904097b80 tx=0x7f9904009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- 172.21.15.93:0/4006034637 shutdown_connections 2023-07-21T22:17:24.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca9ac0 0x55e0baca3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 --2- 172.21.15.93:0/4006034637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca91b0 0x55e0baca3980 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- 172.21.15.93:0/4006034637 >> 172.21.15.93:0/4006034637 conn(0x55e0bac99ef0 msgr2=0x55e0baca4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- 172.21.15.93:0/4006034637 shutdown_connections 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- 172.21.15.93:0/4006034637 wait complete. 2023-07-21T22:17:24.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 -- start start 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.012+0000 7f99283b4b80 1 Processor -- start 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacb35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bacb4040 con 0x55e0baca9ac0 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bacb41b0 con 0x55e0baca91b0 2023-07-21T22:17:24.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0bacb4370 con 0x55e0baca4930 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99283b4b80 10 client.0 _ll_register_callbacks cb 0x7f9910011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacb35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f991a7cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45270/0 (socket says 172.21.15.93:45270) 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f991a7cc700 1 -- 172.21.15.93:0/579421690 learned_addr learned my addr 172.21.15.93:0/579421690 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:24.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f9919fcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:60578/0 (socket says 172.21.15.93:60578) 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 -- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 msgr2=0x55e0bacb5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 -- 172.21.15.93:0/579421690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 msgr2=0x55e0bacb3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 -- 172.21.15.93:0/579421690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99141a2860 con 0x55e0baca91b0 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f991a7cc700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f99197ca700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacb35c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9904097b80 tx=0x7f99040a32c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d3b056446e9810 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:24.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.013+0000 7f98fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990409af30 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f990409b160 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f990c00f040 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99040cb790 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 10 client.25493 fetch_fsmap learned FSMap version 26 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55e0baa4a250 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99040cb020 con 0x55e0baca91b0 2023-07-21T22:17:24.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99040cb190 con 0x55e0baca91b0 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 10 client.25493 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 20 client.25493 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bad175a0 0x55e0bad19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 4 client.25493 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0babb1050 con 0x55e0bad175a0 2023-07-21T22:17:24.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.014+0000 7f9919fcb700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bad175a0 0x55e0bad19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f98fd7fa700 10 client.25493 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f9919fcb700 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bad175a0 0x55e0bad19980 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7d6d41952b6f84 server_cookie=85f893cced7eb3a in_seq=0 out_seq=0 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f98fd7fa700 1 -- 172.21.15.93:0/579421690 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e0babb1050 con 0x55e0bad175a0 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f98fd7fa700 10 client.25493 handle_command_reply: tid=0 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 client.25493 shutdown 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 msgr2=0x55e0bacb35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacb35c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9904097b80 tx=0x7f99040a32c0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 shutdown_connections 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e0bad175a0 0x55e0bad19980 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e0baca9ac0 0x55e0bacb3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f98fd7fa700 0 client.25493 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e0baca4930 0x55e0bacb5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 --2- 172.21.15.93:0/579421690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0baca91b0 0x55e0bacb35c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 >> 172.21.15.93:0/579421690 conn(0x55e0bac99ef0 msgr2=0x55e0baca3980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 shutdown_connections 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 1 -- 172.21.15.93:0/579421690 wait complete. 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.015+0000 7f99283b4b80 20 client.25493 trim_cache size 0 max 16384 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:17:24.070 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": { 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: "50d3dd18-848d-48fc-9269-9a6c8d64c2fe": { 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/", 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: "tag": "50d3dd18-848d-48fc-9269-9a6c8d64c2fe", 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: "options": "recursive,force" 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:17:24.071 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:24.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 msgr2=0x7f99000a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 0x7f99000a3890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99140ffb70 tx=0x7f9910009260 comp rx=0 tx=0).stop 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 msgr2=0x7f9914196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9904000c00 tx=0x7f9904004790 comp rx=0 tx=0).stop 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 shutdown_connections 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9914104a40 0x7f99141a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99140fea30 0x7f99141969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f99000a13e0 0x7f99000a3890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 --2- 172.21.15.93:0/2158771098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99140ff3e0 0x7f9914196ef0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:24.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 >> 172.21.15.93:0/2158771098 conn(0x7f99140fa530 msgr2=0x7f99141025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:24.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 shutdown_connections 2023-07-21T22:17:24.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:24.016+0000 7f991c22f700 1 -- 172.21.15.93:0/2158771098 wait complete. 2023-07-21T22:17:24.074 INFO:tasks.cephfs.filesystem:scrub status for tag:50d3dd18-848d-48fc-9269-9a6c8d64c2fe - {'path': '/', 'tag': '50d3dd18-848d-48fc-9269-9a6c8d64c2fe', 'options': 'recursive,force'} 2023-07-21T22:17:24.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 2596 29.00 MB/sec warmup 1 sec latency 30.228 ms 2023-07-21T22:17:25.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 4960 22.57 MB/sec warmup 2 sec latency 11.191 ms 2023-07-21T22:17:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:25 smithi093 ceph-mon[96579]: scrub queued for path: / 2023-07-21T22:17:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:25 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:25 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:25 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:25 smithi093 ceph-mon[96579]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:25 smithi194 ceph-mon[98288]: scrub queued for path: / 2023-07-21T22:17:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:25 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:25 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:25.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:25 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:25 smithi194 ceph-mon[98288]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:25.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:25 smithi134 ceph-mon[97873]: scrub queued for path: / 2023-07-21T22:17:25.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:25 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:25 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:25.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:25 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:25.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:25 smithi134 ceph-mon[97873]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:26.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 7697 19.74 MB/sec warmup 3 sec latency 6.821 ms 2023-07-21T22:17:27.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 10882 19.50 MB/sec warmup 4 sec latency 8.443 ms 2023-07-21T22:17:27.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:27 smithi093 ceph-mon[96579]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:27.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:27 smithi194 ceph-mon[98288]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:27.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:27 smithi134 ceph-mon[97873]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-21T22:17:28.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 13137 17.22 MB/sec warmup 5 sec latency 57.717 ms 2023-07-21T22:17:29.305 INFO:tasks.workunit.client.0.smithi093.stdout: 1 15433 16.59 MB/sec warmup 6 sec latency 11.540 ms 2023-07-21T22:17:29.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:29 smithi093 ceph-mon[96579]: pgmap v34: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 470 KiB/s wr, 71 op/s 2023-07-21T22:17:29.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:29 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:17:29.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:29 smithi194 ceph-mon[98288]: pgmap v34: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 470 KiB/s wr, 71 op/s 2023-07-21T22:17:29.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:29 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:17:29.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:29 smithi134 ceph-mon[97873]: pgmap v34: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 470 KiB/s wr, 71 op/s 2023-07-21T22:17:29.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:29 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:17:30.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 17980 16.61 MB/sec warmup 7 sec latency 207.047 ms 2023-07-21T22:17:30.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:30 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:30.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:30 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:30.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:30 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:31.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 17980 14.53 MB/sec warmup 8 sec latency 1207.080 ms 2023-07-21T22:17:31.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:31 smithi093 ceph-mon[96579]: pgmap v35: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 470 KiB/s wr, 67 op/s 2023-07-21T22:17:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:31 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:31.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:31 smithi194 ceph-mon[98288]: pgmap v35: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 470 KiB/s wr, 67 op/s 2023-07-21T22:17:31.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:31 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:31.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:31 smithi134 ceph-mon[97873]: pgmap v35: 97 pgs: 97 active+clean; 6.2 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 470 KiB/s wr, 67 op/s 2023-07-21T22:17:31.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:31 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:32.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 17980 12.92 MB/sec warmup 9 sec latency 2207.197 ms 2023-07-21T22:17:32.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:32.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:32.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:17:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:32.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:17:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:32.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:32.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:17:33.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 17980 11.62 MB/sec warmup 10 sec latency 3207.317 ms 2023-07-21T22:17:33.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:33 smithi093 ceph-mon[96579]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 874 KiB/s wr, 122 op/s 2023-07-21T22:17:33.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:33 smithi194 ceph-mon[98288]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 874 KiB/s wr, 122 op/s 2023-07-21T22:17:33.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:33 smithi134 ceph-mon[97873]: pgmap v36: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 874 KiB/s wr, 122 op/s 2023-07-21T22:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 -- 172.21.15.93:0/1580685331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec10a110 msgr2=0x7f15ec10c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 --2- 172.21.15.93:0/1580685331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec10a110 0x7f15ec10c570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f15e400b0a0 tx=0x7f15e400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 -- 172.21.15.93:0/1580685331 shutdown_connections 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 --2- 172.21.15.93:0/1580685331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec10a110 0x7f15ec10c570 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 --2- 172.21.15.93:0/1580685331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 0x7f15ec1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 --2- 172.21.15.93:0/1580685331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec1016e0 0x7f15ec109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.225+0000 7f15f4317700 1 -- 172.21.15.93:0/1580685331 >> 172.21.15.93:0/1580685331 conn(0x7f15ec0769b0 msgr2=0x7f15ec076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- 172.21.15.93:0/1580685331 shutdown_connections 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- 172.21.15.93:0/1580685331 wait complete. 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 Processor -- start 2023-07-21T22:17:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- start start 2023-07-21T22:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 0x7f15ec19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec10a110 0x7f15ec195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec1961b0 con 0x7f15ec1016e0 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec196320 con 0x7f15ec100dd0 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f4317700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec1964e0 con 0x7f15ec10a110 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f18b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f18b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45292/0 (socket says 172.21.15.93:45292) 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.226+0000 7f15f18b2700 1 -- 172.21.15.93:0/3237458734 learned_addr learned my addr 172.21.15.93:0/3237458734 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f20b3700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 0x7f15ec19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f28b4700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec10a110 0x7f15ec195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 -- 172.21.15.93:0/3237458734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec10a110 msgr2=0x7f15ec195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec10a110 0x7f15ec195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 -- 172.21.15.93:0/3237458734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 msgr2=0x7f15ec19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 0x7f15ec19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 -- 172.21.15.93:0/3237458734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4009d00 con 0x7f15ec1016e0 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f18b2700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f15e800e9a0 tx=0x7f15e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15df7fe700 1 -- 172.21.15.93:0/3237458734 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8009790 con 0x7f15ec1016e0 2023-07-21T22:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15df7fe700 1 -- 172.21.15.93:0/3237458734 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e8010430 con 0x7f15ec1016e0 2023-07-21T22:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec1967c0 con 0x7f15ec1016e0 2023-07-21T22:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15df7fe700 1 -- 172.21.15.93:0/3237458734 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8018680 con 0x7f15ec1016e0 2023-07-21T22:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.227+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec1a2cd0 con 0x7f15ec1016e0 2023-07-21T22:17:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.229+0000 7f15df7fe700 1 -- 172.21.15.93:0/3237458734 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f15e80105a0 con 0x7f15ec1016e0 2023-07-21T22:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.230+0000 7f15df7fe700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 0x7f15d80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.230+0000 7f15df7fe700 1 -- 172.21.15.93:0/3237458734 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f15e8014070 con 0x7f15ec1016e0 2023-07-21T22:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.230+0000 7f15f20b3700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 0x7f15d80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.230+0000 7f15f20b3700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 0x7f15d80a3910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007be0 tx=0x7f15e00073d0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 Processor -- start 2023-07-21T22:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 -- start start 2023-07-21T22:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5210 0x5580c22cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5660 0x5580c22cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c21d4c50 con 0x5580c212b6b0 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22279e0 con 0x5580c22d5660 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.231+0000 7f160049cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c2220e70 con 0x5580c22d5210 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45298/0 (socket says 172.21.15.93:45298) 2023-07-21T22:17:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 -- 172.21.15.93:0/4036169391 learned_addr learned my addr 172.21.15.93:0/4036169391 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f28b4700 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5660 0x5580c22cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f18b2700 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5210 0x5580c22cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 -- 172.21.15.93:0/4036169391 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5210 msgr2=0x5580c22cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5210 0x5580c22cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 -- 172.21.15.93:0/4036169391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5660 msgr2=0x5580c22cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5660 0x5580c22cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 -- 172.21.15.93:0/4036169391 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c204ed80 con 0x5580c212b6b0 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f15e0010e80 tx=0x7f15e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a71cf80e21661a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15dcdac700 1 -- 172.21.15.93:0/4036169391 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0010eb0 con 0x5580c212b6b0 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15dcdac700 1 -- 172.21.15.93:0/4036169391 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e0003b40 con 0x5580c212b6b0 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.232+0000 7f15dcdac700 1 -- 172.21.15.93:0/4036169391 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0011bc0 con 0x5580c212b6b0 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- 172.21.15.93:0/4036169391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 msgr2=0x5580c22d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f15e0010e80 tx=0x7f15e0013000 comp rx=0 tx=0).stop 2023-07-21T22:17:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- 172.21.15.93:0/4036169391 shutdown_connections 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5660 0x5580c22cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c212b6b0 0x5580c22d4cd0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 --2- 172.21.15.93:0/4036169391 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5210 0x5580c22cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- 172.21.15.93:0/4036169391 >> 172.21.15.93:0/4036169391 conn(0x5580c22c4ae0 msgr2=0x5580c22ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- 172.21.15.93:0/4036169391 shutdown_connections 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- 172.21.15.93:0/4036169391 wait complete. 2023-07-21T22:17:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.233+0000 7f160049cb80 1 -- start start 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 Processor -- start 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22db620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5660 0x5580c22d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c21c0a60 con 0x5580c22d5210 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c21c0760 con 0x5580c212b6b0 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c20a48c0 con 0x5580c22d5660 2023-07-21T22:17:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f160049cb80 10 client.0 _ll_register_callbacks cb 0x5580c21be700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45306/0 (socket says 172.21.15.93:45306) 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 -- 172.21.15.93:0/1784663693 learned_addr learned my addr 172.21.15.93:0/1784663693 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f28b4700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5660 0x5580c22d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f20b3700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 -- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5660 msgr2=0x5580c22d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5660 0x5580c22d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 -- 172.21.15.93:0/1784663693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 msgr2=0x5580c22db620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22db620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 -- 172.21.15.93:0/1784663693 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0010b60 con 0x5580c22d5210 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15f18b2700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f15e8099c70 tx=0x7f15e809a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9549260b95c6c46 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.234+0000 7f15d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80a2130 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f15e809dfa0 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580c217a490 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e80a23c0 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 10 client.15918 fetch_fsmap learned FSMap version 27 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580c2037c00 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580c217a490 con 0x5580c22d5210 2023-07-21T22:17:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 27) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f15e8098ed0 con 0x5580c22d5210 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 20 client.15918 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22e8c60 0x5580c22eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 4 client.15918 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580c2219e50 con 0x5580c22e8c60 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.235+0000 7f15f20b3700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22e8c60 0x5580c22eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.236+0000 7f15d5ffb700 10 client.15918 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.236+0000 7f15f20b3700 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22e8c60 0x5580c22eb070 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ccccd1020241e0f server_cookie=4a1efb8547ab1612 in_seq=0 out_seq=0 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f15d5ffb700 1 -- 172.21.15.93:0/1784663693 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580c2219e50 con 0x5580c22e8c60 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f15d5ffb700 10 client.15918 handle_command_reply: tid=0 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 client.15918 shutdown 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 msgr2=0x5580c22d8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f15e8099c70 tx=0x7f15e809a4b0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 shutdown_connections 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22e8c60 0x5580c22eb070 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22d5660 0x5580c22d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22db620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f15d5ffb700 0 client.15918 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 --2- 172.21.15.93:0/1784663693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22d8830 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 >> 172.21.15.93:0/1784663693 conn(0x5580c22c4ae0 msgr2=0x5580c22c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 shutdown_connections 2023-07-21T22:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 1 -- 172.21.15.93:0/1784663693 wait complete. 2023-07-21T22:17:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.237+0000 7f160049cb80 20 client.15918 trim_cache size 0 max 16384 2023-07-21T22:17:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 Processor -- start 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 -- start start 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 0x5580c22d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5210 0x5580c22db150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22e55e0 con 0x5580c22c4ae0 2023-07-21T22:17:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22cacb0 con 0x5580c22d5210 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f160049cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22d4920 con 0x5580c212b6b0 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45312/0 (socket says 172.21.15.93:45312) 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 -- 172.21.15.93:0/2583400602 learned_addr learned my addr 172.21.15.93:0/2583400602 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f18b2700 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 0x5580c22d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 -- 172.21.15.93:0/2583400602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 msgr2=0x5580c22d4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 0x5580c22d4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 -- 172.21.15.93:0/2583400602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5210 msgr2=0x5580c22db150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5210 0x5580c22db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 -- 172.21.15.93:0/2583400602 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c20751f0 con 0x5580c22c4ae0 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f18b2700 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 0x5580c22d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15f20b3700 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f15e00079c0 tx=0x7f15e0011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f62bc82f903e57d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.252+0000 7f15dd5ad700 1 -- 172.21.15.93:0/2583400602 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0003680 con 0x5580c22c4ae0 2023-07-21T22:17:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f15dd5ad700 1 -- 172.21.15.93:0/2583400602 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e00037e0 con 0x5580c22c4ae0 2023-07-21T22:17:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f15dd5ad700 1 -- 172.21.15.93:0/2583400602 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005630 con 0x5580c22c4ae0 2023-07-21T22:17:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- 172.21.15.93:0/2583400602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 msgr2=0x5580c22dfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f15e00079c0 tx=0x7f15e0011520 comp rx=0 tx=0).stop 2023-07-21T22:17:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- 172.21.15.93:0/2583400602 shutdown_connections 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c22d5210 0x5580c22db150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22c4ae0 0x5580c22dfc40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 --2- 172.21.15.93:0/2583400602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c212b6b0 0x5580c22d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- 172.21.15.93:0/2583400602 >> 172.21.15.93:0/2583400602 conn(0x5580c22e5210 msgr2=0x5580c22d3910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- 172.21.15.93:0/2583400602 shutdown_connections 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- 172.21.15.93:0/2583400602 wait complete. 2023-07-21T22:17:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 -- start start 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.253+0000 7f160049cb80 1 Processor -- start 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 0x5580c22ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22de6e0 con 0x5580c22d5210 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22de820 con 0x5580c212b6b0 2023-07-21T22:17:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c22de9e0 con 0x5580c22c4ae0 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 10 client.0 _ll_register_callbacks cb 0x7f15e8099ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45322/0 (socket says 172.21.15.93:45322) 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 -- 172.21.15.93:0/4078013402 learned_addr learned my addr 172.21.15.93:0/4078013402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f18b2700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 0x5580c22ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 -- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 msgr2=0x5580c22ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 0x5580c22ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 -- 172.21.15.93:0/4078013402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 msgr2=0x5580c22df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 -- 172.21.15.93:0/4078013402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580c204ed80 con 0x5580c22d5210 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f18b2700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 0x5580c22ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15f28b4700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009a20 tx=0x7f15e4004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e39c4f6ac9a1ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400e040 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f15e40076e0 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.254+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15e0003b40 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e40078a0 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 10 client.15924 fetch_fsmap learned FSMap version 27 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5580c204d250 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e0003b40 con 0x5580c22d5210 2023-07-21T22:17:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 27) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f15e400b0d0 con 0x5580c22d5210 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 20 client.15924 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22d5660 0x5580c2344480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 4 client.15924 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580c21dbff0 con 0x5580c22d5660 2023-07-21T22:17:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.255+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22d5660 0x5580c2344480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.256+0000 7f15d5ffb700 10 client.15924 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.256+0000 7f15f20b3700 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22d5660 0x5580c2344480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d97a2b5264a8c2 server_cookie=ac482772b90e818 in_seq=0 out_seq=0 2023-07-21T22:17:34.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 17980 10.57 MB/sec warmup 11 sec latency 4207.348 ms 2023-07-21T22:17:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f15d5ffb700 1 -- 172.21.15.93:0/4078013402 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580c21dbff0 con 0x5580c22d5660 2023-07-21T22:17:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f15d5ffb700 10 client.15924 handle_command_reply: tid=0 2023-07-21T22:17:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 client.15924 shutdown 2023-07-21T22:17:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 msgr2=0x5580c22de160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009a20 tx=0x7f15e4004cf0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 shutdown_connections 2023-07-21T22:17:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5580c22d5660 0x5580c2344480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5580c22d5210 0x5580c22de160 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f15d5ffb700 0 client.15924 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5580c212b6b0 0x5580c22df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 --2- 172.21.15.93:0/4078013402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c22c4ae0 0x5580c22ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 >> 172.21.15.93:0/4078013402 conn(0x5580c22e5210 msgr2=0x5580c22d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 shutdown_connections 2023-07-21T22:17:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 1 -- 172.21.15.93:0/4078013402 wait complete. 2023-07-21T22:17:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.368+0000 7f160049cb80 20 client.15924 trim_cache size 0 max 16384 2023-07-21T22:17:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:17:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:17:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:17:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:17:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 msgr2=0x7f15d80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 0x7f15d80a3910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007be0 tx=0x7f15e00073d0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 msgr2=0x7f15ec19c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f15e800e9a0 tx=0x7f15e800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 shutdown_connections 2023-07-21T22:17:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ec10a110 0x7f15ec195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15ec100dd0 0x7f15ec19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15d80a1460 0x7f15d80a3910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 --2- 172.21.15.93:0/3237458734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15ec1016e0 0x7f15ec19c220 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 >> 172.21.15.93:0/3237458734 conn(0x7f15ec0769b0 msgr2=0x7f15ec101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 shutdown_connections 2023-07-21T22:17:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:34.369+0000 7f15f4317700 1 -- 172.21.15.93:0/3237458734 wait complete. 2023-07-21T22:17:35.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18267 9.71 MB/sec warmup 12 sec latency 4880.886 ms 2023-07-21T22:17:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:35 smithi093 ceph-mon[96579]: pgmap v37: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:35 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:35.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:35 smithi194 ceph-mon[98288]: pgmap v37: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:35.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:35 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:35 smithi134 ceph-mon[97873]: pgmap v37: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:35 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:36.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18466 8.98 MB/sec warmup 13 sec latency 856.473 ms 2023-07-21T22:17:36.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:36.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:36.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:37.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18466 8.34 MB/sec warmup 14 sec latency 1856.556 ms 2023-07-21T22:17:37.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:37 smithi093 ceph-mon[96579]: pgmap v38: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:37.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:37 smithi194 ceph-mon[98288]: pgmap v38: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:37.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:37 smithi134 ceph-mon[97873]: pgmap v38: 97 pgs: 97 active+clean; 10 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 873 KiB/s wr, 117 op/s 2023-07-21T22:17:38.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18466 7.79 MB/sec warmup 15 sec latency 2856.659 ms 2023-07-21T22:17:39.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18466 7.30 MB/sec warmup 16 sec latency 3856.708 ms 2023-07-21T22:17:39.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:39 smithi093 ceph-mon[96579]: pgmap v39: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 979 KiB/s wr, 215 op/s 2023-07-21T22:17:39.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:39 smithi194 ceph-mon[98288]: pgmap v39: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 979 KiB/s wr, 215 op/s 2023-07-21T22:17:39.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:39 smithi134 ceph-mon[97873]: pgmap v39: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 979 KiB/s wr, 215 op/s 2023-07-21T22:17:40.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18657 6.90 MB/sec warmup 17 sec latency 4528.357 ms 2023-07-21T22:17:41.306 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18657 6.52 MB/sec warmup 18 sec latency 1206.522 ms 2023-07-21T22:17:41.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:41 smithi194 ceph-mon[98288]: pgmap v40: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 508 KiB/s wr, 152 op/s 2023-07-21T22:17:41.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:41 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:41.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:41 smithi134 ceph-mon[97873]: pgmap v40: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 508 KiB/s wr, 152 op/s 2023-07-21T22:17:41.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:41 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:41.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:41 smithi093 ceph-mon[96579]: pgmap v40: 97 pgs: 97 active+clean; 6.4 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 508 KiB/s wr, 152 op/s 2023-07-21T22:17:41.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:41 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:17:42.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18657 6.17 MB/sec warmup 19 sec latency 2206.628 ms 2023-07-21T22:17:43.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18657 5.86 MB/sec warmup 20 sec latency 3206.734 ms 2023-07-21T22:17:43.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:43 smithi194 ceph-mon[98288]: pgmap v41: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 546 KiB/s wr, 163 op/s 2023-07-21T22:17:43.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:43 smithi134 ceph-mon[97873]: pgmap v41: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 546 KiB/s wr, 163 op/s 2023-07-21T22:17:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:43 smithi093 ceph-mon[96579]: pgmap v41: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 546 KiB/s wr, 163 op/s 2023-07-21T22:17:44.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18722 5.59 MB/sec warmup 21 sec latency 3514.960 ms 2023-07-21T22:17:44.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:44 smithi194 ceph-mon[98288]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:44.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:44 smithi134 ceph-mon[97873]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:44.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:44 smithi093 ceph-mon[96579]: pgmap v42: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:45.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18723 5.34 MB/sec warmup 22 sec latency 1358.573 ms 2023-07-21T22:17:46.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18723 5.10 MB/sec warmup 23 sec latency 1305.170 ms 2023-07-21T22:17:47.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18723 4.89 MB/sec warmup 24 sec latency 2305.308 ms 2023-07-21T22:17:47.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:47 smithi134 ceph-mon[97873]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:47.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:47 smithi093 ceph-mon[96579]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 -- 172.21.15.93:0/3625174464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc0ffb50 msgr2=0x7febfc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 --2- 172.21.15.93:0/3625174464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc0ffb50 0x7febfc0fffc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7febf8009990 tx=0x7febf8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 -- 172.21.15.93:0/3625174464 shutdown_connections 2023-07-21T22:17:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 --2- 172.21.15.93:0/3625174464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc100500 0x7febfc10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 --2- 172.21.15.93:0/3625174464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc105b80 0x7febfc105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 --2- 172.21.15.93:0/3625174464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc0ffb50 0x7febfc0fffc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 -- 172.21.15.93:0/3625174464 >> 172.21.15.93:0/3625174464 conn(0x7febfc0fb630 msgr2=0x7febfc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 -- 172.21.15.93:0/3625174464 shutdown_connections 2023-07-21T22:17:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 -- 172.21.15.93:0/3625174464 wait complete. 2023-07-21T22:17:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.487+0000 7fec040e1700 1 Processor -- start 2023-07-21T22:17:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 -- start start 2023-07-21T22:17:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc0ffb50 0x7febfc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 0x7febfc195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1960b0 con 0x7febfc100500 2023-07-21T22:17:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc196220 con 0x7febfc105b80 2023-07-21T22:17:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec040e1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc1963e0 con 0x7febfc0ffb50 2023-07-21T22:17:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45336/0 (socket says 172.21.15.93:45336) 2023-07-21T22:17:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 -- 172.21.15.93:0/704103339 learned_addr learned my addr 172.21.15.93:0/704103339 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec01e7d700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc0ffb50 0x7febfc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0267e700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 0x7febfc195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 -- 172.21.15.93:0/704103339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc0ffb50 msgr2=0x7febfc19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc0ffb50 0x7febfc19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 -- 172.21.15.93:0/704103339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 msgr2=0x7febfc195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 0x7febfc195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 -- 172.21.15.93:0/704103339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80095f0 con 0x7febfc100500 2023-07-21T22:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0267e700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 0x7febfc195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7fec0167c700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7febf80095c0 tx=0x7febf800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.488+0000 7febf2ffd700 1 -- 172.21.15.93:0/704103339 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf801d070 con 0x7febfc100500 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.489+0000 7febf2ffd700 1 -- 172.21.15.93:0/704103339 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf800fc00 con 0x7febfc100500 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.489+0000 7febf2ffd700 1 -- 172.21.15.93:0/704103339 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80173f0 con 0x7febfc100500 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.489+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc196660 con 0x7febfc100500 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.489+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc1a2bd0 con 0x7febfc100500 2023-07-21T22:17:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.491+0000 7febf2ffd700 1 -- 172.21.15.93:0/704103339 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7febf8022470 con 0x7febfc100500 2023-07-21T22:17:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.491+0000 7febf2ffd700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 0x7febe80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.491+0000 7febf2ffd700 1 -- 172.21.15.93:0/704103339 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7febf80ce1e0 con 0x7febfc100500 2023-07-21T22:17:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.491+0000 7fec01e7d700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 0x7febe80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec01e7d700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 0x7febe80a3890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7febec00dad0 tx=0x7febec00d4b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 Processor -- start 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 -- start start 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0a620 0x55636ee04350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0aa70 0x55636ee048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ed05c50 con 0x55636ec5c6b0 2023-07-21T22:17:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ed589e0 con 0x55636ee0a620 2023-07-21T22:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.492+0000 7fec10266b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ed51e70 con 0x55636ee0aa70 2023-07-21T22:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45342/0 (socket says 172.21.15.93:45342) 2023-07-21T22:17:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 -- 172.21.15.93:0/2540933911 learned_addr learned my addr 172.21.15.93:0/2540933911 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec0167c700 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0a620 0x55636ee04350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec0267e700 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0aa70 0x55636ee048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 -- 172.21.15.93:0/2540933911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0aa70 msgr2=0x55636ee048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0aa70 0x55636ee048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 -- 172.21.15.93:0/2540933911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0a620 msgr2=0x55636ee04350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0a620 0x55636ee04350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 -- 172.21.15.93:0/2540933911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55636eb7fd80 con 0x55636ec5c6b0 2023-07-21T22:17:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7fec01e7d700 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7febec005c50 tx=0x7febec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f89a8d58f31b875 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7febe7fff700 1 -- 172.21.15.93:0/2540933911 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec011b90 con 0x55636ec5c6b0 2023-07-21T22:17:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7febe7fff700 1 -- 172.21.15.93:0/2540933911 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febec011cf0 con 0x55636ec5c6b0 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.493+0000 7febe7fff700 1 -- 172.21.15.93:0/2540933911 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec01d5f0 con 0x55636ec5c6b0 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- 172.21.15.93:0/2540933911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 msgr2=0x55636ee0a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7febec005c50 tx=0x7febec012000 comp rx=0 tx=0).stop 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- 172.21.15.93:0/2540933911 shutdown_connections 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0aa70 0x55636ee048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee0a0e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 --2- 172.21.15.93:0/2540933911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0a620 0x55636ee04350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- 172.21.15.93:0/2540933911 >> 172.21.15.93:0/2540933911 conn(0x55636edf5ae0 msgr2=0x55636edfb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- 172.21.15.93:0/2540933911 shutdown_connections 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- 172.21.15.93:0/2540933911 wait complete. 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.494+0000 7fec10266b80 1 -- start start 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 Processor -- start 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0aa70 0x55636ee0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ecf1a60 con 0x55636ec5c6b0 2023-07-21T22:17:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ecf1760 con 0x55636ee0aa70 2023-07-21T22:17:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ebd58c0 con 0x55636ee0a620 2023-07-21T22:17:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec10266b80 10 client.0 _ll_register_callbacks cb 0x55636ecef700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45348/0 (socket says 172.21.15.93:45348) 2023-07-21T22:17:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 -- 172.21.15.93:0/831292102 learned_addr learned my addr 172.21.15.93:0/831292102 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec0167c700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec0267e700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0aa70 0x55636ee0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 -- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 msgr2=0x55636ee0db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 -- 172.21.15.93:0/831292102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0aa70 msgr2=0x55636ee0e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0aa70 0x55636ee0e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 -- 172.21.15.93:0/831292102 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec00f930 con 0x55636ec5c6b0 2023-07-21T22:17:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7fec01e7d700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7febec005c20 tx=0x7febec011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474f136c93cd729b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.495+0000 7febe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec02e070 con 0x55636ec5c6b0 2023-07-21T22:17:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7febec01e660 con 0x55636ec5c6b0 2023-07-21T22:17:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55636ecab490 con 0x55636ec5c6b0 2023-07-21T22:17:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7febec01e860 con 0x55636ec5c6b0 2023-07-21T22:17:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 10 client.15936 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55636eb68c00 con 0x55636ec5c6b0 2023-07-21T22:17:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55636ecab490 con 0x55636ec5c6b0 2023-07-21T22:17:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7febec004b80 con 0x55636ec5c6b0 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 20 client.15936 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee1de10 0x55636ee20220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 4 client.15936 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55636ed4ae50 con 0x55636ee1de10 2023-07-21T22:17:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.496+0000 7fec0167c700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee1de10 0x55636ee20220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.497+0000 7febe57fa700 10 client.15936 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.497+0000 7fec0167c700 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee1de10 0x55636ee20220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a5e5c046896732 server_cookie=c3ca369b9268b4c5 in_seq=0 out_seq=0 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.497+0000 7febe57fa700 1 -- 172.21.15.93:0/831292102 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55636ed4ae50 con 0x55636ee1de10 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.497+0000 7febe57fa700 10 client.15936 handle_command_reply: tid=0 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 client.15936 shutdown 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 msgr2=0x55636ee10920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7febec005c20 tx=0x7febec011c70 comp rx=0 tx=0).stop 2023-07-21T22:17:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 shutdown_connections 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee1de10 0x55636ee20220 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ee0aa70 0x55636ee0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7febe57fa700 0 client.15936 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ec5c6b0 0x55636ee10920 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 --2- 172.21.15.93:0/831292102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 >> 172.21.15.93:0/831292102 conn(0x55636edf5ae0 msgr2=0x55636edfa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 shutdown_connections 2023-07-21T22:17:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 1 -- 172.21.15.93:0/831292102 wait complete. 2023-07-21T22:17:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.498+0000 7fec10266b80 20 client.15936 trim_cache size 0 max 16384 2023-07-21T22:17:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.512+0000 7fec10266b80 1 Processor -- start 2023-07-21T22:17:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.512+0000 7fec10266b80 1 -- start start 2023-07-21T22:17:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee0a470 con 0x55636ee0aa70 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee05330 con 0x55636ec5c6b0 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec10266b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee0e6b0 con 0x55636ee0a620 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45360/0 (socket says 172.21.15.93:45360) 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 -- 172.21.15.93:0/2273510404 learned_addr learned my addr 172.21.15.93:0/2273510404 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0167c700 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec01e7d700 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 -- 172.21.15.93:0/2273510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 msgr2=0x55636ee10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 -- 172.21.15.93:0/2273510404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 msgr2=0x55636ee15040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 -- 172.21.15.93:0/2273510404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55636eba61f0 con 0x55636ee0aa70 2023-07-21T22:17:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7fec0267e700 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c90 tx=0x7febf4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32044423241a51b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7febf0ff9700 1 -- 172.21.15.93:0/2273510404 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4007cb0 con 0x55636ee0aa70 2023-07-21T22:17:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7febf0ff9700 1 -- 172.21.15.93:0/2273510404 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf400f030 con 0x55636ee0aa70 2023-07-21T22:17:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.513+0000 7febf0ff9700 1 -- 172.21.15.93:0/2273510404 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4014640 con 0x55636ee0aa70 2023-07-21T22:17:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- 172.21.15.93:0/2273510404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 msgr2=0x55636edfacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c90 tx=0x7febf4009040 comp rx=0 tx=0).stop 2023-07-21T22:17:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- 172.21.15.93:0/2273510404 shutdown_connections 2023-07-21T22:17:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636edfacd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 --2- 172.21.15.93:0/2273510404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- 172.21.15.93:0/2273510404 >> 172.21.15.93:0/2273510404 conn(0x55636edf5ae0 msgr2=0x55636ee04350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- 172.21.15.93:0/2273510404 shutdown_connections 2023-07-21T22:17:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- 172.21.15.93:0/2273510404 wait complete. 2023-07-21T22:17:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 -- start start 2023-07-21T22:17:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.514+0000 7fec10266b80 1 Processor -- start 2023-07-21T22:17:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee0cc80 con 0x55636ee0aa70 2023-07-21T22:17:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee0cdf0 con 0x55636ec5c6b0 2023-07-21T22:17:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636ee0cfb0 con 0x55636ee0a620 2023-07-21T22:17:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec10266b80 10 client.0 _ll_register_callbacks cb 0x7febec00fb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45366/0 (socket says 172.21.15.93:45366) 2023-07-21T22:17:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 -- 172.21.15.93:0/3901912326 learned_addr learned my addr 172.21.15.93:0/3901912326 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0167c700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec01e7d700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 -- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 msgr2=0x55636ee0c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 -- 172.21.15.93:0/3901912326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 msgr2=0x55636ee15050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 -- 172.21.15.93:0/3901912326 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf40098c0 con 0x55636ee0aa70 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7fec0267e700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c60 tx=0x7febf4015d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92780b1dc49a847 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.515+0000 7febe57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf40156f0 con 0x55636ee0aa70 2023-07-21T22:17:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7febf4015850 con 0x55636ee0aa70 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febec011cf0 con 0x55636ee0aa70 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7febf4005040 con 0x55636ee0aa70 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 10 client.15942 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55636eb7e250 con 0x55636ee0aa70 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febec011cf0 con 0x55636ee0aa70 2023-07-21T22:17:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7febf4014070 con 0x55636ee0aa70 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 20 client.15942 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee730f0 0x55636ee75500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 4 client.15942 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55636ed0cff0 con 0x55636ee730f0 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.516+0000 7fec01e7d700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee730f0 0x55636ee75500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.517+0000 7febe57fa700 10 client.15942 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.517+0000 7fec01e7d700 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee730f0 0x55636ee75500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7530106d784e889a server_cookie=e3c8407cd0e61545 in_seq=0 out_seq=0 2023-07-21T22:17:47.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:47 smithi194 ceph-mon[98288]: pgmap v43: 97 pgs: 97 active+clean; 6.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 143 KiB/s wr, 108 op/s 2023-07-21T22:17:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.548+0000 7febe57fa700 1 -- 172.21.15.93:0/3901912326 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55636ed0cff0 con 0x55636ee730f0 2023-07-21T22:17:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.548+0000 7febe57fa700 10 client.15942 handle_command_reply: tid=0 2023-07-21T22:17:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.548+0000 7fec10266b80 1 client.15942 shutdown 2023-07-21T22:17:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 msgr2=0x55636ee0c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7febf4009c60 tx=0x7febf4015d10 comp rx=0 tx=0).stop 2023-07-21T22:17:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 shutdown_connections 2023-07-21T22:17:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55636ee730f0 0x55636ee75500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55636ee0aa70 0x55636ee0c740 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55636ec5c6b0 0x55636ee15050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7febe57fa700 0 client.15942 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 --2- 172.21.15.93:0/3901912326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55636ee0a620 0x55636ee0c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 >> 172.21.15.93:0/3901912326 conn(0x55636edf5ae0 msgr2=0x55636edfac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 shutdown_connections 2023-07-21T22:17:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 1 -- 172.21.15.93:0/3901912326 wait complete. 2023-07-21T22:17:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec10266b80 20 client.15942 trim_cache size 0 max 16384 2023-07-21T22:17:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:17:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:17:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:17:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:17:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 msgr2=0x7febe80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 0x7febe80a3890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7febec00dad0 tx=0x7febec00d4b0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 msgr2=0x7febfc19c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7febf80095c0 tx=0x7febf800f710 comp rx=0 tx=0).stop 2023-07-21T22:17:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.549+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 shutdown_connections 2023-07-21T22:17:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febfc105b80 0x7febfc195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febfc0ffb50 0x7febfc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7febe80a13e0 0x7febe80a3890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 --2- 172.21.15.93:0/704103339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febfc100500 0x7febfc19c120 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 >> 172.21.15.93:0/704103339 conn(0x7febfc0fb630 msgr2=0x7febfc103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 shutdown_connections 2023-07-21T22:17:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:17:47.550+0000 7fec040e1700 1 -- 172.21.15.93:0/704103339 wait complete. 2023-07-21T22:17:48.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18723 4.70 MB/sec warmup 25 sec latency 3305.425 ms 2023-07-21T22:17:49.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 18723 4.52 MB/sec warmup 26 sec latency 4305.539 ms 2023-07-21T22:17:49.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:49 smithi134 ceph-mon[97873]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 166 op/s 2023-07-21T22:17:49.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:49 smithi093 ceph-mon[96579]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 166 op/s 2023-07-21T22:17:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:49 smithi194 ceph-mon[98288]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 3.0 MiB/s wr, 166 op/s 2023-07-21T22:17:50.307 INFO:tasks.workunit.client.0.smithi093.stdout: 1 19560 4.52 MB/sec warmup 27 sec latency 4973.841 ms 2023-07-21T22:17:51.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 20792 4.61 MB/sec warmup 28 sec latency 568.511 ms 2023-07-21T22:17:51.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:51 smithi134 ceph-mon[97873]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2023-07-21T22:17:51.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:51 smithi093 ceph-mon[96579]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2023-07-21T22:17:51.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:51 smithi194 ceph-mon[98288]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2023-07-21T22:17:52.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 20792 4.45 MB/sec warmup 29 sec latency 1568.543 ms 2023-07-21T22:17:53.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 21647 4.51 MB/sec warmup 30 sec latency 2091.276 ms 2023-07-21T22:17:53.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:53 smithi134 ceph-mon[97873]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.0 MiB/s wr, 96 op/s 2023-07-21T22:17:53.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:53 smithi093 ceph-mon[96579]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.0 MiB/s wr, 96 op/s 2023-07-21T22:17:53.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:53 smithi194 ceph-mon[98288]: pgmap v46: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 3.0 MiB/s wr, 96 op/s 2023-07-21T22:17:54.076 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:17:54.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1533544752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea81015b0 msgr2=0x7f5ea8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1533544752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea81015b0 0x7f5ea8109aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009990 tx=0x7f5ea4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1533544752 shutdown_connections 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1533544752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea810a070 0x7f5ea810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1533544752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 0x7f5ea8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1533544752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea81015b0 0x7f5ea8109aa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1533544752 >> 172.21.15.93:0/1533544752 conn(0x7f5ea80fa4b0 msgr2=0x7f5ea80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1533544752 shutdown_connections 2023-07-21T22:17:54.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.163+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1533544752 wait complete. 2023-07-21T22:17:54.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 Processor -- start 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 -- start start 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 0x7f5ea819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 0x7f5ea819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8196210 con 0x7f5ea810a070 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8196380 con 0x7f5ea8100ca0 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eaf9b1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8196540 con 0x7f5ea81015b0 2023-07-21T22:17:54.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45380/0 (socket says 172.21.15.93:45380) 2023-07-21T22:17:54.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 -- 172.21.15.93:0/1389324593 learned_addr learned my addr 172.21.15.93:0/1389324593 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 0x7f5ea819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5ead74d700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 0x7f5ea819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 -- 172.21.15.93:0/1389324593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 msgr2=0x7f5ea819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 0x7f5ea819bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 -- 172.21.15.93:0/1389324593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 msgr2=0x7f5ea819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 0x7f5ea819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eadf4e700 1 -- 172.21.15.93:0/1389324593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40095f0 con 0x7f5ea810a070 2023-07-21T22:17:54.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.164+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 0x7f5ea819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0007ee0 tx=0x7f5ea000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5e9e7fc700 1 -- 172.21.15.93:0/1389324593 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000ddd0 con 0x7f5ea810a070 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5e9e7fc700 1 -- 172.21.15.93:0/1389324593 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea000f040 con 0x7f5ea810a070 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea8196820 con 0x7f5ea810a070 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5e9e7fc700 1 -- 172.21.15.93:0/1389324593 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0015400 con 0x7f5ea810a070 2023-07-21T22:17:54.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.165+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81a29e0 con 0x7f5ea810a070 2023-07-21T22:17:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.167+0000 7f5e9e7fc700 1 -- 172.21.15.93:0/1389324593 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f5ea0004020 con 0x7f5ea810a070 2023-07-21T22:17:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.168+0000 7f5e9e7fc700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 0x7f5e940a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.168+0000 7f5e9e7fc700 1 -- 172.21.15.93:0/1389324593 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5ea00cc8e0 con 0x7f5ea810a070 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.168+0000 7f5ead74d700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 0x7f5e940a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.168+0000 7f5ead74d700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 0x7f5e940a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005ea0 tx=0x7f5e98005e30 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 Processor -- start 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 -- start start 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce403580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 0x5637ce3fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce303cb0 con 0x5637ce3fe930 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce356a40 con 0x5637ce4031b0 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce34fed0 con 0x5637ce403ac0 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45388/0 (socket says 172.21.15.93:45388) 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3523928480 learned_addr learned my addr 172.21.15.93:0/3523928480 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 0x5637ce3fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5ead74d700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce403580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3523928480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 msgr2=0x5637ce3fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 0x5637ce3fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3523928480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 msgr2=0x5637ce403580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce403580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3523928480 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ce17dde0 con 0x5637ce3fe930 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.169+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 0x5637ce3fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00c9fe0 tx=0x7f5ea00954c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64ebc5c3d3f00b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5e937fe700 1 -- 172.21.15.93:0/3523928480 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0099780 con 0x5637ce3fe930 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5e937fe700 1 -- 172.21.15.93:0/3523928480 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea009d7b0 con 0x5637ce3fe930 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5e937fe700 1 -- 172.21.15.93:0/3523928480 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0098c70 con 0x5637ce3fe930 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3523928480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 msgr2=0x5637ce3feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00c9fe0 tx=0x7f5ea00954c0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3523928480 shutdown_connections 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce3feda0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce403580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3523928480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce403ac0 0x5637ce3fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3523928480 >> 172.21.15.93:0/3523928480 conn(0x5637ce3f3ef0 msgr2=0x5637ce3f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3523928480 shutdown_connections 2023-07-21T22:17:54.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.170+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3523928480 wait complete. 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 -- start start 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 Processor -- start 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce407e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce2efac0 con 0x5637ce403ac0 2023-07-21T22:17:54.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce2ef7c0 con 0x5637ce4031b0 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce1d3920 con 0x5637ce3fe930 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5ebbb36b80 10 client.0 _ll_register_callbacks cb 0x5637ce2ed760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5eadf4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45394/0 (socket says 172.21.15.93:45394) 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3125425887 learned_addr learned my addr 172.21.15.93:0/3125425887 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.171+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce407e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ead74d700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 msgr2=0x5637ce40abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3125425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 msgr2=0x5637ce407e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce407e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 -- 172.21.15.93:0/3125425887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81a29e0 con 0x5637ce403ac0 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ead74d700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0099480 tx=0x7f5ea009ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e644738296e8e98e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00a1bd0 con 0x5637ce403ac0 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637ce2a94f0 con 0x5637ce403ac0 2023-07-21T22:17:54.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5ea0099780 con 0x5637ce403ac0 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea0098e80 con 0x5637ce403ac0 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 10 client.15954 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5637ce166c60 con 0x5637ce403ac0 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637ce2a94f0 con 0x5637ce403ac0 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5ea00a1740 con 0x5637ce403ac0 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:54.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 20 client.15954 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce4181b0 0x5637ce41a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.172+0000 7f5ebbb36b80 4 client.15954 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.173+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637ce348eb0 con 0x5637ce4181b0 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.173+0000 7f5ead74d700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce4181b0 0x5637ce41a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.173+0000 7f5ead74d700 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce4181b0 0x5637ce41a5c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab0ce4750acb1cc server_cookie=ddffd0b413c225e2 in_seq=0 out_seq=0 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.173+0000 7f5e90ff9700 10 client.15954 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5e90ff9700 1 -- 172.21.15.93:0/3125425887 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637ce348eb0 con 0x5637ce4181b0 2023-07-21T22:17:54.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5e90ff9700 10 client.15954 handle_command_reply: tid=0 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 client.15954 shutdown 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 msgr2=0x5637ce4085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0099480 tx=0x7f5ea009ae10 comp rx=0 tx=0).stop 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 shutdown_connections 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce4181b0 0x5637ce41a5c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce403ac0 0x5637ce4085a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5e90ff9700 0 client.15954 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/3125425887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce407e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 >> 172.21.15.93:0/3125425887 conn(0x5637ce3f3ef0 msgr2=0x5637ce3f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 shutdown_connections 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/3125425887 wait complete. 2023-07-21T22:17:54.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.174+0000 7f5ebbb36b80 20 client.15954 trim_cache size 0 max 16384 2023-07-21T22:17:54.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 Processor -- start 2023-07-21T22:17:54.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 -- start start 2023-07-21T22:17:54.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce3fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce3f39b0 con 0x5637ce3f3ef0 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce414ed0 con 0x5637ce4031b0 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce3f5900 con 0x5637ce3fe930 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45398/0 (socket says 172.21.15.93:45398) 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 -- 172.21.15.93:0/68019327 learned_addr learned my addr 172.21.15.93:0/68019327 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce3fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 -- 172.21.15.93:0/68019327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 msgr2=0x5637ce40a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 -- 172.21.15.93:0/68019327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 msgr2=0x5637ce3fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce3fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 -- 172.21.15.93:0/68019327 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637ce1a4250 con 0x5637ce3f3ef0 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce3fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.191+0000 7f5ead74d700 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5637ce41aeb0 tx=0x7f5e9800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a489a32c01893c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5e93fff700 1 -- 172.21.15.93:0/68019327 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9801fd30 con 0x5637ce3f3ef0 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5e93fff700 1 -- 172.21.15.93:0/68019327 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e98025040 con 0x5637ce3f3ef0 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5e93fff700 1 -- 172.21.15.93:0/68019327 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98025dc0 con 0x5637ce3f3ef0 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/68019327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 msgr2=0x5637ce40f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5637ce41aeb0 tx=0x7f5e9800af90 comp rx=0 tx=0).stop 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/68019327 shutdown_connections 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce3fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3f3ef0 0x5637ce40f580 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/68019327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3fe930 0x5637ce40a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/68019327 >> 172.21.15.93:0/68019327 conn(0x5637ce408710 msgr2=0x5637ce3fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/68019327 shutdown_connections 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/68019327 wait complete. 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.192+0000 7f5ebbb36b80 1 -- start start 2023-07-21T22:17:54.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 Processor -- start 2023-07-21T22:17:54.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3f3ef0 0x5637ce465830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce45ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce4605c0 con 0x5637ce3fe930 2023-07-21T22:17:54.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce460730 con 0x5637ce4031b0 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637ce4608f0 con 0x5637ce3f3ef0 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ebbb36b80 10 client.0 _ll_register_callbacks cb 0x7f5ea0095ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45406/0 (socket says 172.21.15.93:45406) 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 -- 172.21.15.93:0/866848395 learned_addr learned my addr 172.21.15.93:0/866848395 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5ead74d700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3f3ef0 0x5637ce465830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.245 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eadf4e700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce45ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 -- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3f3ef0 msgr2=0x5637ce465830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3f3ef0 0x5637ce465830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 -- 172.21.15.93:0/866848395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 msgr2=0x5637ce45ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce45ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 -- 172.21.15.93:0/866848395 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9800eca0 con 0x5637ce3fe930 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5eacf4c700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009990 tx=0x7f5ea4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5598177911fb125f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.246 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.193+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4003aa0 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5ea4003c00 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea009d7b0 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea4003dc0 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 10 client.15960 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5637ce17c2b0 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea009d7b0 con 0x5637ce3fe930 2023-07-21T22:17:54.247 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5ea4025070 con 0x5637ce3fe930 2023-07-21T22:17:54.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:54.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 20 client.15960 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce403ac0 0x5637ce46f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.248 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 4 client.15960 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637ce30b050 con 0x5637ce403ac0 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.194+0000 7f5ead74d700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce403ac0 0x5637ce46f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5e90ff9700 10 client.15960 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ead74d700 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce403ac0 0x5637ce46f7e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09374b3bf5b3e78 server_cookie=ba6d7b48e9beca85 in_seq=0 out_seq=0 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5e90ff9700 1 -- 172.21.15.93:0/866848395 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5637ce30b050 con 0x5637ce403ac0 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5e90ff9700 10 client.15960 handle_command_reply: tid=0 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 client.15960 shutdown 2023-07-21T22:17:54.249 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 msgr2=0x5637ce465d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009990 tx=0x7f5ea4009f90 comp rx=0 tx=0).stop 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 shutdown_connections 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5637ce403ac0 0x5637ce46f7e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637ce4031b0 0x5637ce45ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637ce3f3ef0 0x5637ce465830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5e90ff9700 0 client.15960 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:54.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 --2- 172.21.15.93:0/866848395 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5637ce3fe930 0x5637ce465d70 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 >> 172.21.15.93:0/866848395 conn(0x5637ce408710 msgr2=0x5637ce3fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 shutdown_connections 2023-07-21T22:17:54.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 1 -- 172.21.15.93:0/866848395 wait complete. 2023-07-21T22:17:54.251 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.195+0000 7f5ebbb36b80 20 client.15960 trim_cache size 0 max 16384 2023-07-21T22:17:54.251 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:54.252 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "no active scrubs running", 2023-07-21T22:17:54.252 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": {} 2023-07-21T22:17:54.252 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:54.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 msgr2=0x7f5e940a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 0x7f5e940a3890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005ea0 tx=0x7f5e98005e30 comp rx=0 tx=0).stop 2023-07-21T22:17:54.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 msgr2=0x7f5ea8195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0007ee0 tx=0x7f5ea000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 shutdown_connections 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5e940a13e0 0x7f5e940a3890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ea810a070 0x7f5ea8195a40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea8100ca0 0x7f5ea819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 --2- 172.21.15.93:0/1389324593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ea81015b0 0x7f5ea819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 >> 172.21.15.93:0/1389324593 conn(0x7f5ea80fa4b0 msgr2=0x7f5ea810c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 shutdown_connections 2023-07-21T22:17:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.196+0000 7f5eaf9b1700 1 -- 172.21.15.93:0/1389324593 wait complete. 2023-07-21T22:17:54.255 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:17:54.255 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:17:54.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 21991 4.38 MB/sec warmup 31 sec latency 636.885 ms 2023-07-21T22:17:54.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 -- 172.21.15.93:0/2193734940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 msgr2=0x7fb308104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 --2- 172.21.15.93:0/2193734940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb308104f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009990 tx=0x7fb2fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 -- 172.21.15.93:0/2193734940 shutdown_connections 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 --2- 172.21.15.93:0/2193734940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 0x7fb30810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 --2- 172.21.15.93:0/2193734940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb308104f50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 --2- 172.21.15.93:0/2193734940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 0x7fb3080ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 -- 172.21.15.93:0/2193734940 >> 172.21.15.93:0/2193734940 conn(0x7fb308074e00 msgr2=0x7fb308075200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 -- 172.21.15.93:0/2193734940 shutdown_connections 2023-07-21T22:17:54.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.334+0000 7fb30c952700 1 -- 172.21.15.93:0/2193734940 wait complete. 2023-07-21T22:17:54.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 Processor -- start 2023-07-21T22:17:54.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 -- start start 2023-07-21T22:17:54.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 0x7fb308196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 0x7fb308196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081978a0 con 0x7fb308104b80 2023-07-21T22:17:54.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a2010 con 0x7fb3080ff0c0 2023-07-21T22:17:54.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30c952700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a2180 con 0x7fb3080ffa70 2023-07-21T22:17:54.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45422/0 (socket says 172.21.15.93:45422) 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 -- 172.21.15.93:0/3373111365 learned_addr learned my addr 172.21.15.93:0/3373111365 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb305d9b700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 0x7fb308196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb30659c700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 0x7fb308196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 -- 172.21.15.93:0/3373111365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 msgr2=0x7fb308196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 0x7fb308196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 -- 172.21.15.93:0/3373111365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 msgr2=0x7fb308196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 0x7fb308196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.335+0000 7fb306d9d700 1 -- 172.21.15.93:0/3373111365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0095f0 con 0x7fb308104b80 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb306d9d700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb300007ee0 tx=0x7fb30000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300017070 con 0x7fb308104b80 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3081a2460 con 0x7fb308104b80 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb30000f040 con 0x7fb308104b80 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300005580 con 0x7fb308104b80 2023-07-21T22:17:54.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.336+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3081a2920 con 0x7fb308104b80 2023-07-21T22:17:54.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.337+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308066a70 con 0x7fb308104b80 2023-07-21T22:17:54.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.338+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fb30000db20 con 0x7fb308104b80 2023-07-21T22:17:54.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.338+0000 7fb2f77fe700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 0x7fb2f00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.338+0000 7fb30659c700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 0x7fb2f00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.338+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb3000ccd40 con 0x7fb308104b80 2023-07-21T22:17:54.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.339+0000 7fb30659c700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 0x7fb2f00a38e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b5c0 tx=0x7fb2fc005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.342+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb300095ec0 con 0x7fb308104b80 2023-07-21T22:17:54.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.576+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3080fe5a0 con 0x7fb308104b80 2023-07-21T22:17:54.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.577+0000 7fb2f77fe700 1 -- 172.21.15.93:0/3373111365 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10867 (secure 0 0 0) 0x7fb3000994e0 con 0x7fb308104b80 2023-07-21T22:17:54.622 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:17:54.622 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:17:40.392501+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:17:54.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 msgr2=0x7fb2f00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 0x7fb2f00a38e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b5c0 tx=0x7fb2fc005fd0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 msgr2=0x7fb3081a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb300007ee0 tx=0x7fb30000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 shutdown_connections 2023-07-21T22:17:54.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2f00a1430 0x7fb2f00a38e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb308104b80 0x7fb3081a1ad0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3080ff0c0 0x7fb308196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 --2- 172.21.15.93:0/3373111365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3080ffa70 0x7fb308196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 >> 172.21.15.93:0/3373111365 conn(0x7fb308074e00 msgr2=0x7fb3080fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 shutdown_connections 2023-07-21T22:17:54.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.579+0000 7fb30c952700 1 -- 172.21.15.93:0/3373111365 wait complete. 2023-07-21T22:17:54.627 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 30 2023-07-21T22:17:54.636 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:17:54.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 -- 172.21.15.93:0/3593341088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24980ff580 msgr2=0x7f24980ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 --2- 172.21.15.93:0/3593341088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24980ff580 0x7f24980ff950 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f24880099c0 tx=0x7f2488009cd0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 -- 172.21.15.93:0/3593341088 shutdown_connections 2023-07-21T22:17:54.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 --2- 172.21.15.93:0/3593341088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2498103ce0 0x7f249810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 --2- 172.21.15.93:0/3593341088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24980ff580 0x7f24980ff950 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 --2- 172.21.15.93:0/3593341088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.714+0000 7f249ffe9700 1 -- 172.21.15.93:0/3593341088 >> 172.21.15.93:0/3593341088 conn(0x7f2498076200 msgr2=0x7f2498076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- 172.21.15.93:0/3593341088 shutdown_connections 2023-07-21T22:17:54.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- 172.21.15.93:0/3593341088 wait complete. 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 Processor -- start 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- start start 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24980ff580 0x7f2498196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981a1f00 con 0x7f2498103ce0 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981a2040 con 0x7f24980ff580 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249ffe9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981a2180 con 0x7f24980fff20 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249e586700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249e586700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45436/0 (socket says 172.21.15.93:45436) 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.715+0000 7f249e586700 1 -- 172.21.15.93:0/2669821909 learned_addr learned my addr 172.21.15.93:0/2669821909 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249dd85700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24980ff580 0x7f2498196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249d584700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 -- 172.21.15.93:0/2669821909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 msgr2=0x7f2498196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 -- 172.21.15.93:0/2669821909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24980ff580 msgr2=0x7f2498196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24980ff580 0x7f2498196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 -- 172.21.15.93:0/2669821909 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880095f0 con 0x7f2498103ce0 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249d584700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:54.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249e586700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2490007d70 tx=0x7f249000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f248effd700 1 -- 172.21.15.93:0/2669821909 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000ad40 con 0x7f2498103ce0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f248effd700 1 -- 172.21.15.93:0/2669821909 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24900128e0 con 0x7f2498103ce0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24981a23e0 con 0x7f2498103ce0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.716+0000 7f248effd700 1 -- 172.21.15.93:0/2669821909 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490011780 con 0x7f2498103ce0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.717+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24981a2910 con 0x7f2498103ce0 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.718+0000 7f248effd700 1 -- 172.21.15.93:0/2669821909 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f2490012a50 con 0x7f2498103ce0 2023-07-21T22:17:54.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.719+0000 7f248effd700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 0x7f24840a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.719+0000 7f249dd85700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 0x7f24840a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.719+0000 7f248effd700 1 -- 172.21.15.93:0/2669821909 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f24900cd140 con 0x7f2498103ce0 2023-07-21T22:17:54.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.719+0000 7f249dd85700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 0x7f24840a3910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f2488005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 Processor -- start 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 -- start start 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x557122779580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x557122774da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122679cb0 con 0x557122779ac0 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571226cca40 con 0x5571227791b0 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571226c5ed0 con 0x557122774930 2023-07-21T22:17:54.771 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.720+0000 7f249d584700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249dd85700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x557122779580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45442/0 (socket says 172.21.15.93:45442) 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 -- 172.21.15.93:0/1179588914 learned_addr learned my addr 172.21.15.93:0/1179588914 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249e586700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x557122774da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 -- 172.21.15.93:0/1179588914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 msgr2=0x557122774da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x557122774da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.772 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 -- 172.21.15.93:0/1179588914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 msgr2=0x557122779580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x557122779580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 -- 172.21.15.93:0/1179588914 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571224f3de0 con 0x557122779ac0 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249e586700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x557122774da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249dd85700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x557122779580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f249d584700 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f249400bc40 tx=0x7f249400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1caeae43b09710 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f2483fff700 1 -- 172.21.15.93:0/1179588914 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400dd60 con 0x557122779ac0 2023-07-21T22:17:54.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f2483fff700 1 -- 172.21.15.93:0/1179588914 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24940178e0 con 0x557122779ac0 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.721+0000 7f2483fff700 1 -- 172.21.15.93:0/1179588914 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400f920 con 0x557122779ac0 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1179588914 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 msgr2=0x557122774350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f249400bc40 tx=0x7f249400bf50 comp rx=0 tx=0).stop 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1179588914 shutdown_connections 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x557122774da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x557122779580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1179588914 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122779ac0 0x557122774350 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.774 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1179588914 >> 172.21.15.93:0/1179588914 conn(0x557122769ef0 msgr2=0x55712276f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1179588914 shutdown_connections 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1179588914 wait complete. 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 -- start start 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.722+0000 7f24ac16eb80 1 Processor -- start 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.775 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122779ac0 0x55712277e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122665ac0 con 0x557122774930 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571226657c0 con 0x5571227791b0 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122549920 con 0x557122779ac0 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24ac16eb80 10 client.0 _ll_register_callbacks cb 0x557122663760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45446/0 (socket says 172.21.15.93:45446) 2023-07-21T22:17:54.776 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 -- 172.21.15.93:0/3956061800 learned_addr learned my addr 172.21.15.93:0/3956061800 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249e586700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122779ac0 0x55712277e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249d584700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 -- 172.21.15.93:0/3956061800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122779ac0 msgr2=0x55712277e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122779ac0 0x55712277e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 -- 172.21.15.93:0/3956061800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 msgr2=0x55712277dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 -- 172.21.15.93:0/3956061800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249810c130 con 0x557122774930 2023-07-21T22:17:54.777 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249d584700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f249dd85700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f24880099c0 tx=0x7f248801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6aa179537446c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.723+0000 7f24817fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248802b050 con 0x557122774930 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2488003f20 con 0x557122774930 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55712261f4f0 con 0x557122774930 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2488005710 con 0x557122774930 2023-07-21T22:17:54.778 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 10 client.15978 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f249400b870 con 0x557122774930 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55712261f4f0 con 0x557122774930 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f248800f4d0 con 0x557122774930 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 10 client.15978 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 10 client.15978 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 20 client.15978 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55712278e520 0x557122790930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 4 client.15978 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571226beeb0 con 0x55712278e520 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.724+0000 7f249d584700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55712278e520 0x557122790930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24817fa700 10 client.15978 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f249d584700 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55712278e520 0x557122790930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffab85fb2d284597 server_cookie=e786292913dba079 in_seq=0 out_seq=0 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24817fa700 1 -- 172.21.15.93:0/3956061800 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571226beeb0 con 0x55712278e520 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24817fa700 10 client.15978 handle_command_reply: tid=0 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 client.15978 shutdown 2023-07-21T22:17:54.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 msgr2=0x557122780b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f24880099c0 tx=0x7f248801a050 comp rx=0 tx=0).stop 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 shutdown_connections 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55712278e520 0x557122790930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122779ac0 0x55712277e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24817fa700 0 client.15978 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x557122780b90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3956061800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 >> 172.21.15.93:0/3956061800 conn(0x557122769ef0 msgr2=0x55712276f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 shutdown_connections 2023-07-21T22:17:54.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.725+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3956061800 wait complete. 2023-07-21T22:17:54.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.726+0000 7f24ac16eb80 20 client.15978 trim_cache size 0 max 16384 2023-07-21T22:17:54.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 Processor -- start 2023-07-21T22:17:54.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 -- start start 2023-07-21T22:17:54.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x55712277d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712276c010 con 0x557122769ef0 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712277e9b0 con 0x5571227791b0 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.738+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712277c340 con 0x557122774930 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45452/0 (socket says 172.21.15.93:45452) 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 -- 172.21.15.93:0/3748748675 learned_addr learned my addr 172.21.15.93:0/3748748675 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249d584700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x55712277d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249e586700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 -- 172.21.15.93:0/3748748675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 msgr2=0x55712277d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x55712277d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 -- 172.21.15.93:0/3748748675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 msgr2=0x55712277db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 -- 172.21.15.93:0/3748748675 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571224dcc60 con 0x557122769ef0 2023-07-21T22:17:54.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249d584700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x55712277d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.787 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249e586700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:54.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f249dd85700 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2488005a60 tx=0x7f2488005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c65f5d9a24966dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f248cff9700 1 -- 172.21.15.93:0/3748748675 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248802b040 con 0x557122769ef0 2023-07-21T22:17:54.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f248cff9700 1 -- 172.21.15.93:0/3748748675 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248802c040 con 0x557122769ef0 2023-07-21T22:17:54.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.739+0000 7f248cff9700 1 -- 172.21.15.93:0/3748748675 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880048a0 con 0x557122769ef0 2023-07-21T22:17:54.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3748748675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 msgr2=0x5571227806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2488005a60 tx=0x7f2488005220 comp rx=0 tx=0).stop 2023-07-21T22:17:54.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3748748675 shutdown_connections 2023-07-21T22:17:54.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571227791b0 0x55712277db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122769ef0 0x5571227806c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/3748748675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557122774930 0x55712277d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3748748675 >> 172.21.15.93:0/3748748675 conn(0x55712276e2c0 msgr2=0x5571227729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3748748675 shutdown_connections 2023-07-21T22:17:54.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- 172.21.15.93:0/3748748675 wait complete. 2023-07-21T22:17:54.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- start start 2023-07-21T22:17:54.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 Processor -- start 2023-07-21T22:17:54.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557122769ef0 0x5571227dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571227791b0 0x5571227835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122783cd0 con 0x557122774930 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122783e40 con 0x557122769ef0 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.740+0000 7f24ac16eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557122784000 con 0x5571227791b0 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24ac16eb80 10 client.0 _ll_register_callbacks cb 0x7f248800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45460/0 (socket says 172.21.15.93:45460) 2023-07-21T22:17:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 -- 172.21.15.93:0/1200023605 learned_addr learned my addr 172.21.15.93:0/1200023605 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249dd85700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557122769ef0 0x5571227dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249e586700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571227791b0 0x5571227835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 -- 172.21.15.93:0/1200023605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571227791b0 msgr2=0x5571227835f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571227791b0 0x5571227835f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 -- 172.21.15.93:0/1200023605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557122769ef0 msgr2=0x5571227dfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557122769ef0 0x5571227dfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 -- 172.21.15.93:0/1200023605 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249400b870 con 0x557122774930 2023-07-21T22:17:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f249d584700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f249400bc40 tx=0x7f249400bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861ec11e01b881ca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24817fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494027040 con 0x557122774930 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2494013070 con 0x557122774930 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24940178e0 con 0x557122774930 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2494017420 con 0x557122774930 2023-07-21T22:17:54.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24ac16eb80 10 client.15984 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.741+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55712251a250 con 0x557122774930 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249403a080 con 0x557122774930 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f249402a030 con 0x557122774930 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 10 client.15984 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 20 client.15984 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x557122779ac0 0x5571227e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 4 client.15984 mds_command: new command op to 14589 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557122681050 con 0x557122779ac0 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f249dd85700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x557122779ac0 0x5571227e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24817fa700 10 client.15984 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f249dd85700 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x557122779ac0 0x5571227e9a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff6dd06c07699d02 server_cookie=73927fbedfb96901 in_seq=0 out_seq=0 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24817fa700 1 -- 172.21.15.93:0/1200023605 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557122681050 con 0x557122779ac0 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24817fa700 10 client.15984 handle_command_reply: tid=0 2023-07-21T22:17:54.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 1 client.15984 shutdown 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 msgr2=0x5571227e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.742+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f249400bc40 tx=0x7f249400bbd0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 shutdown_connections 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x557122779ac0 0x5571227e9a50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5571227791b0 0x5571227835f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24817fa700 0 client.15984 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:17:54.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557122769ef0 0x5571227dfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 --2- 172.21.15.93:0/1200023605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557122774930 0x5571227e0160 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 >> 172.21.15.93:0/1200023605 conn(0x55712276e2c0 msgr2=0x5571227738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 shutdown_connections 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 1 -- 172.21.15.93:0/1200023605 wait complete. 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f24ac16eb80 20 client.15984 trim_cache size 0 max 16384 2023-07-21T22:17:54.802 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:54.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 msgr2=0x7f24840a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 0x7f24840a3910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2488000c00 tx=0x7f2488005fb0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 msgr2=0x7f24981a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2490007d70 tx=0x7f249000a340 comp rx=0 tx=0).stop 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 shutdown_connections 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24840a1460 0x7f24840a3910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2498103ce0 0x7f24981a19c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24980ff580 0x7f2498196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 --2- 172.21.15.93:0/2669821909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24980fff20 0x7f2498196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 >> 172.21.15.93:0/2669821909 conn(0x7f2498076200 msgr2=0x7f24980fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 shutdown_connections 2023-07-21T22:17:54.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.743+0000 7f249ffe9700 1 -- 172.21.15.93:0/2669821909 wait complete. 2023-07-21T22:17:54.805 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:17:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 -- 172.21.15.93:0/2832106801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00fe200 msgr2=0x7ff6e00fe650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2832106801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00fe200 0x7ff6e00fe650 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009990 tx=0x7ff6d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 -- 172.21.15.93:0/2832106801 shutdown_connections 2023-07-21T22:17:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2832106801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00feb90 0x7ff6e010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2832106801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 0x7ff6e0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2832106801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00fe200 0x7ff6e00fe650 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 -- 172.21.15.93:0/2832106801 >> 172.21.15.93:0/2832106801 conn(0x7ff6e0076200 msgr2=0x7ff6e0076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 -- 172.21.15.93:0/2832106801 shutdown_connections 2023-07-21T22:17:54.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 -- 172.21.15.93:0/2832106801 wait complete. 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.883+0000 7ff6e7557700 1 Processor -- start 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 -- start start 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00fe200 0x7ff6e019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 0x7ff6e0195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.932 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01960f0 con 0x7ff6e00feb90 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0196260 con 0x7ff6e0105c00 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e7557700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0196420 con 0x7ff6e00fe200 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45474/0 (socket says 172.21.15.93:45474) 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 -- 172.21.15.93:0/2074994837 learned_addr learned my addr 172.21.15.93:0/2074994837 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e52f3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00fe200 0x7ff6e019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.933 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 0x7ff6e0195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 -- 172.21.15.93:0/2074994837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00fe200 msgr2=0x7ff6e019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00fe200 0x7ff6e019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 -- 172.21.15.93:0/2074994837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 msgr2=0x7ff6e0195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 0x7ff6e0195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.884+0000 7ff6e4af2700 1 -- 172.21.15.93:0/2074994837 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d00095f0 con 0x7ff6e00feb90 2023-07-21T22:17:54.934 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6d67fc700 1 -- 172.21.15.93:0/2074994837 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001d070 con 0x7ff6e00feb90 2023-07-21T22:17:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6d67fc700 1 -- 172.21.15.93:0/2074994837 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d0021410 con 0x7ff6e00feb90 2023-07-21T22:17:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e01966a0 con 0x7ff6e00feb90 2023-07-21T22:17:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6d67fc700 1 -- 172.21.15.93:0/2074994837 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0017650 con 0x7ff6e00feb90 2023-07-21T22:17:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.885+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a2a00 con 0x7ff6e00feb90 2023-07-21T22:17:54.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.888+0000 7ff6d67fc700 1 -- 172.21.15.93:0/2074994837 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff6d00217f0 con 0x7ff6e00feb90 2023-07-21T22:17:54.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 Processor -- start 2023-07-21T22:17:54.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6d67fc700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 0x7ff6cc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 -- start start 2023-07-21T22:17:54.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6d67fc700 1 -- 172.21.15.93:0/2074994837 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff6d00ce340 con 0x7ff6e00feb90 2023-07-21T22:17:54.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 0x7ff6cc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921f00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.941 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921e05cb0 con 0x55b921f05ac0 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921e58a40 con 0x55b921f051b0 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921e51ed0 con 0x55b921f00930 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6e52f3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6e4af2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921f00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6e52f3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56642/0 (socket says 172.21.15.93:56642) 2023-07-21T22:17:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.889+0000 7ff6e52f3700 1 -- 172.21.15.93:0/1090313370 learned_addr learned my addr 172.21.15.93:0/1090313370 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 -- 172.21.15.93:0/1090313370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 msgr2=0x55b921f00da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f00da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 -- 172.21.15.93:0/1090313370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 msgr2=0x55b921f00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921f00350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 0x7ff6cc0a3910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6e019c710 tx=0x7ff6d8009920 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.943 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 -- 172.21.15.93:0/1090313370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b921c7fde0 con 0x55b921f051b0 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921f00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00bcf0 tx=0x7ff6dc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a530c1b5333a53f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6cb7fe700 1 -- 172.21.15.93:0/1090313370 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc010c50 con 0x55b921f051b0 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6cb7fe700 1 -- 172.21.15.93:0/1090313370 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6dc015070 con 0x55b921f051b0 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.890+0000 7ff6cb7fe700 1 -- 172.21.15.93:0/1090313370 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc005020 con 0x55b921f051b0 2023-07-21T22:17:54.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/1090313370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 msgr2=0x55b921f05580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00bcf0 tx=0x7ff6dc0103d0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/1090313370 shutdown_connections 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f00da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f05580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/1090313370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921f00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/1090313370 >> 172.21.15.93:0/1090313370 conn(0x55b921ef5ef0 msgr2=0x55b921efb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/1090313370 shutdown_connections 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/1090313370 wait complete. 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 -- start start 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.891+0000 7ff6f36dcb80 1 Processor -- start 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f00930 0x55b921f0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921df1ac0 con 0x55b921f051b0 2023-07-21T22:17:54.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921df17c0 con 0x55b921f00930 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921cd5920 con 0x55b921f05ac0 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 10 client.0 _ll_register_callbacks cb 0x55b921def760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45488/0 (socket says 172.21.15.93:45488) 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 -- 172.21.15.93:0/3599932623 learned_addr learned my addr 172.21.15.93:0/3599932623 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f00930 0x55b921f0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 -- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 msgr2=0x55b921f0a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f0a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 -- 172.21.15.93:0/3599932623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f00930 msgr2=0x55b921f0cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f00930 0x55b921f0cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 -- 172.21.15.93:0/3599932623 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01a2a00 con 0x55b921f051b0 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005910 tx=0x7ff6d009b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b615d265cd0506d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00a3340 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff6d009f020 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.892+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b921dab4f0 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d009ab70 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 10 client.15996 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff6dc00b920 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d00c9080 con 0x55b921f051b0 2023-07-21T22:17:54.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff6d00cbbb0 con 0x55b921f051b0 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 10 client.15996 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 10 client.15996 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 20 client.15996 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f1a1b0 0x55b921f1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 4 client.15996 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b921e4aeb0 con 0x55b921f1a1b0 2023-07-21T22:17:54.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.893+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f1a1b0 0x55b921f1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6c8ff9700 10 client.15996 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f1a1b0 0x55b921f1c590 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d9065872fb378b6 server_cookie=ec3aea576ef6483e in_seq=0 out_seq=0 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3599932623 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b921e4aeb0 con 0x55b921f1a1b0 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6c8ff9700 10 client.15996 handle_command_reply: tid=0 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 client.15996 shutdown 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 msgr2=0x55b921f09e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005910 tx=0x7ff6d009b750 comp rx=0 tx=0).stop 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 shutdown_connections 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f0a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f00930 0x55b921f0cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f1a1b0 0x55b921f1c590 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.895+0000 7ff6c8ff9700 0 client.15996 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.894+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3599932623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f051b0 0x55b921f09e90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.895+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 >> 172.21.15.93:0/3599932623 conn(0x55b921ef5ef0 msgr2=0x55b921efb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.895+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 shutdown_connections 2023-07-21T22:17:54.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.895+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3599932623 wait complete. 2023-07-21T22:17:54.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.895+0000 7ff6f36dcb80 20 client.15996 trim_cache size 0 max 16384 2023-07-21T22:17:54.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.907+0000 7ff6f36dcb80 1 Processor -- start 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 -- start start 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f09e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f0c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921ef59b0 con 0x55b921f05ac0 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921f016d0 con 0x55b921f051b0 2023-07-21T22:17:54.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921ef8010 con 0x55b921f00930 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45498/0 (socket says 172.21.15.93:45498) 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 -- 172.21.15.93:0/2465840059 learned_addr learned my addr 172.21.15.93:0/2465840059 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f0c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f09e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 -- 172.21.15.93:0/2465840059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 msgr2=0x55b921f09e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f09e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.959 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 -- 172.21.15.93:0/2465840059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 msgr2=0x55b921f0c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f0c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 -- 172.21.15.93:0/2465840059 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b921c68c60 con 0x55b921f05ac0 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80199a0 tx=0x7ff6d801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d6a7d05beeab79 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6cbfff700 1 -- 172.21.15.93:0/2465840059 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800e040 con 0x55b921f05ac0 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6cbfff700 1 -- 172.21.15.93:0/2465840059 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d800f040 con 0x55b921f05ac0 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.908+0000 7ff6cbfff700 1 -- 172.21.15.93:0/2465840059 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8004020 con 0x55b921f05ac0 2023-07-21T22:17:54.960 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/2465840059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 msgr2=0x55b921eff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80199a0 tx=0x7ff6d801c000 comp rx=0 tx=0).stop 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/2465840059 shutdown_connections 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f05ac0 0x55b921eff9f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f00930 0x55b921f09e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/2465840059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f0c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/2465840059 >> 172.21.15.93:0/2465840059 conn(0x55b921ef5ef0 msgr2=0x55b921f09730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.961 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/2465840059 shutdown_connections 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/2465840059 wait complete. 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.909+0000 7ff6f36dcb80 1 -- start start 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 Processor -- start 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f10620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921f6e050 con 0x55b921f00930 2023-07-21T22:17:54.962 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921f6e190 con 0x55b921f051b0 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b921f6e2d0 con 0x55b921f05ac0 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6f36dcb80 10 client.0 _ll_register_callbacks cb 0x7ff6d0097550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45500/0 (socket says 172.21.15.93:45500) 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 -- 172.21.15.93:0/3615589869 learned_addr learned my addr 172.21.15.93:0/3615589869 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.963 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 -- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 msgr2=0x55b921f10620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f10620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 -- 172.21.15.93:0/3615589869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 msgr2=0x55b921f100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 -- 172.21.15.93:0/3615589869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d8019650 con 0x55b921f00930 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.964 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e5af4700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f10620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6e52f3700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b510 tx=0x7ff6dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e864b472b57ca8b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.910+0000 7ff6c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc020030 con 0x55b921f00930 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7ff6dc017070 con 0x55b921f00930 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6dc015070 con 0x55b921f00930 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6dc028c40 con 0x55b921f00930 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 10 client.16002 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:54.965 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b921ca6250 con 0x55b921f00930 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6dc015070 con 0x55b921f00930 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff6dc004340 con 0x55b921f00930 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 20 client.16002 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f6f910 0x55b921f71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 4 client.16002 mds_command: new command op to 24448 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:54.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b921e0d050 con 0x55b921f6f910 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.911+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f6f910 0x55b921f71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6e4af2700 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f6f910 0x55b921f71d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17d108f1da00c609 server_cookie=6e614fd3b3fc071a in_seq=0 out_seq=0 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6c8ff9700 10 client.16002 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6c8ff9700 1 -- 172.21.15.93:0/3615589869 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b921e0d050 con 0x55b921f6f910 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6c8ff9700 10 client.16002 handle_command_reply: tid=0 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 client.16002 shutdown 2023-07-21T22:17:54.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 msgr2=0x55b921f0fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b510 tx=0x7ff6dc005fd0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 shutdown_connections 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b921f05ac0 0x55b921f10620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x55b921f6f910 0x55b921f71d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b921f00930 0x55b921f0fba0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6c8ff9700 0 client.16002 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:17:54.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 --2- 172.21.15.93:0/3615589869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b921f051b0 0x55b921f100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 >> 172.21.15.93:0/3615589869 conn(0x55b921ef5ef0 msgr2=0x55b921eff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 shutdown_connections 2023-07-21T22:17:54.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 1 -- 172.21.15.93:0/3615589869 wait complete. 2023-07-21T22:17:54.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.912+0000 7ff6f36dcb80 20 client.16002 trim_cache size 0 max 16384 2023-07-21T22:17:54.969 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:54.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 msgr2=0x7ff6cc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 0x7ff6cc0a3910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6e019c710 tx=0x7ff6d8009920 comp rx=0 tx=0).stop 2023-07-21T22:17:54.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 msgr2=0x7ff6e019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:54.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d00047b0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 shutdown_connections 2023-07-21T22:17:54.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0105c00 0x7ff6e0195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6e00fe200 0x7ff6e019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6cc0a1460 0x7ff6cc0a3910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 --2- 172.21.15.93:0/2074994837 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6e00feb90 0x7ff6e019bf50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:54.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 >> 172.21.15.93:0/2074994837 conn(0x7ff6e0076200 msgr2=0x7ff6e00fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:54.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 shutdown_connections 2023-07-21T22:17:54.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:54.913+0000 7ff6e7557700 1 -- 172.21.15.93:0/2074994837 wait complete. 2023-07-21T22:17:54.975 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:17:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 -- 172.21.15.93:0/2091963656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480fe9d0 msgr2=0x7f97480fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.102 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 --2- 172.21.15.93:0/2091963656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480fe9d0 0x7f97480fee40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9738009990 tx=0x7f9738009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 -- 172.21.15.93:0/2091963656 shutdown_connections 2023-07-21T22:17:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 --2- 172.21.15.93:0/2091963656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97480ff380 0x7f974810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.103 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 --2- 172.21.15.93:0/2091963656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97481049e0 0x7f9748104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 --2- 172.21.15.93:0/2091963656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480fe9d0 0x7f97480fee40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 -- 172.21.15.93:0/2091963656 >> 172.21.15.93:0/2091963656 conn(0x7f97480fa4b0 msgr2=0x7f97480fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 -- 172.21.15.93:0/2091963656 shutdown_connections 2023-07-21T22:17:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 -- 172.21.15.93:0/2091963656 wait complete. 2023-07-21T22:17:55.104 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.055+0000 7f974cced700 1 Processor -- start 2023-07-21T22:17:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 -- start start 2023-07-21T22:17:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97480fe9d0 0x7f974819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.105 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481049e0 0x7f97481948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748194f50 con 0x7f97480ff380 2023-07-21T22:17:55.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481950c0 con 0x7f97481049e0 2023-07-21T22:17:55.106 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748195280 con 0x7f97480fe9d0 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45516/0 (socket says 172.21.15.93:45516) 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97480fe9d0 0x7f974819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 -- 172.21.15.93:0/2148602727 learned_addr learned my addr 172.21.15.93:0/2148602727 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9746d9d700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481049e0 0x7f97481948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.107 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 -- 172.21.15.93:0/2148602727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97480fe9d0 msgr2=0x7f974819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97480fe9d0 0x7f974819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 -- 172.21.15.93:0/2148602727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481049e0 msgr2=0x7f97481948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481049e0 0x7f97481948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 -- 172.21.15.93:0/2148602727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380095f0 con 0x7f97480ff380 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9738000c00 tx=0x7f97380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f97377fe700 1 -- 172.21.15.93:0/2148602727 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973801d070 con 0x7f97480ff380 2023-07-21T22:17:55.108 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f97377fe700 1 -- 172.21.15.93:0/2148602727 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9738021410 con 0x7f97480ff380 2023-07-21T22:17:55.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.056+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748195500 con 0x7f97480ff380 2023-07-21T22:17:55.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.057+0000 7f97377fe700 1 -- 172.21.15.93:0/2148602727 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738017650 con 0x7f97480ff380 2023-07-21T22:17:55.109 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.057+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97481a1860 con 0x7f97480ff380 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.059+0000 7f97377fe700 1 -- 172.21.15.93:0/2148602727 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f97380217f0 con 0x7f97480ff380 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.059+0000 7f97377fe700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 0x7f97300a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.059+0000 7f974659c700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 0x7f97300a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.060+0000 7f97377fe700 1 -- 172.21.15.93:0/2148602727 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f97380ce340 con 0x7f97480ff380 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.060+0000 7f974659c700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 0x7f97300a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97480ffb10 tx=0x7f973c00b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.060+0000 7f9758e72b80 1 Processor -- start 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 -- start start 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821eac0 0x5596b8219350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b8219da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b811ecb0 con 0x5596b821e1b0 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b8171a40 con 0x5596b821eac0 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9758e72b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b816aed0 con 0x5596b8219930 2023-07-21T22:17:55.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45522/0 (socket says 172.21.15.93:45522) 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 -- 172.21.15.93:0/3529176389 learned_addr learned my addr 172.21.15.93:0/3529176389 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9746d9d700 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b8219da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f9745d9b700 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821eac0 0x5596b8219350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 -- 172.21.15.93:0/3529176389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 msgr2=0x5596b8219da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b8219da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 -- 172.21.15.93:0/3529176389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821eac0 msgr2=0x5596b8219350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821eac0 0x5596b8219350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 -- 172.21.15.93:0/3529176389 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b7f98de0 con 0x5596b821e1b0 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.061+0000 7f974659c700 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e30 tx=0x7f973c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622bb0f4bc2ff11e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9734ff9700 1 -- 172.21.15.93:0/3529176389 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c011db0 con 0x5596b821e1b0 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9734ff9700 1 -- 172.21.15.93:0/3529176389 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f973c015070 con 0x5596b821e1b0 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9734ff9700 1 -- 172.21.15.93:0/3529176389 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c01c5f0 con 0x5596b821e1b0 2023-07-21T22:17:55.117 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 -- 172.21.15.93:0/3529176389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 msgr2=0x5596b821e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e30 tx=0x7f973c013000 comp rx=0 tx=0).stop 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 -- 172.21.15.93:0/3529176389 shutdown_connections 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b8219da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b821e580 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 --2- 172.21.15.93:0/3529176389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821eac0 0x5596b8219350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 -- 172.21.15.93:0/3529176389 >> 172.21.15.93:0/3529176389 conn(0x5596b820eef0 msgr2=0x5596b8214830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 -- 172.21.15.93:0/3529176389 shutdown_connections 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.062+0000 7f9758e72b80 1 -- 172.21.15.93:0/3529176389 wait complete. 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 -- start start 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 Processor -- start 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 0x5596b8225c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8223520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b810aac0 con 0x5596b821e1b0 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b810a7c0 con 0x5596b8219930 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b7fee920 con 0x5596b821eac0 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9758e72b80 10 client.0 _ll_register_callbacks cb 0x5596b8108760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9745d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9745d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45530/0 (socket says 172.21.15.93:45530) 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9745d9b700 1 -- 172.21.15.93:0/2678653994 learned_addr learned my addr 172.21.15.93:0/2678653994 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f974659c700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 0x5596b8225c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.063+0000 7f9746d9d700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8223520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 -- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 msgr2=0x5596b8223520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8223520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 -- 172.21.15.93:0/2678653994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 msgr2=0x5596b8225c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 0x5596b8225c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 -- 172.21.15.93:0/2678653994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c010ae0 con 0x5596b821e1b0 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f974659c700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 0x5596b8225c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:55.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f973809af70 tx=0x7f973809b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc6c643beea79cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380a33a0 con 0x5596b821e1b0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f97380a1110 con 0x5596b821e1b0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596b80c44f0 con 0x5596b821e1b0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f973809b270 con 0x5596b821e1b0 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 10 client.16014 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5596b7f81c60 con 0x5596b821e1b0 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596b80c44f0 con 0x5596b821e1b0 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f97380cbc10 con 0x5596b821e1b0 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 10 client.16014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 20 client.16014 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.064+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8233070 0x5596b8235450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.065+0000 7f9758e72b80 4 client.16014 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.065+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596b8163eb0 con 0x5596b8233070 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.065+0000 7f974659c700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8233070 0x5596b8235450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.065+0000 7f974659c700 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8233070 0x5596b8235450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a2a48c97379ec4d server_cookie=6ee7f1443f8343a9 in_seq=0 out_seq=0 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.065+0000 7f972dffb700 10 client.16014 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f972dffb700 1 -- 172.21.15.93:0/2678653994 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596b8163eb0 con 0x5596b8233070 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f972dffb700 10 client.16014 handle_command_reply: tid=0 2023-07-21T22:17:55.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 client.16014 shutdown 2023-07-21T22:17:55.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 msgr2=0x5596b8222e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f973809af70 tx=0x7f973809b7b0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 shutdown_connections 2023-07-21T22:17:55.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8233070 0x5596b8235450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8223520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f972dffb700 0 client.16014 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:55.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b8219930 0x5596b8225c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2678653994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821e1b0 0x5596b8222e30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 >> 172.21.15.93:0/2678653994 conn(0x5596b820eef0 msgr2=0x5596b82143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 shutdown_connections 2023-07-21T22:17:55.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 1 -- 172.21.15.93:0/2678653994 wait complete. 2023-07-21T22:17:55.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.066+0000 7f9758e72b80 20 client.16014 trim_cache size 0 max 16384 2023-07-21T22:17:55.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 Processor -- start 2023-07-21T22:17:55.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 -- start start 2023-07-21T22:17:55.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b822a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b8225750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b821de30 con 0x5596b821eac0 2023-07-21T22:17:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b8223b00 con 0x5596b821e1b0 2023-07-21T22:17:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9758e72b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b820c000 con 0x5596b8219930 2023-07-21T22:17:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9746d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f974659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b822a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9746d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45540/0 (socket says 172.21.15.93:45540) 2023-07-21T22:17:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9746d9d700 1 -- 172.21.15.93:0/2230765073 learned_addr learned my addr 172.21.15.93:0/2230765073 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b8225750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9746d9d700 1 -- 172.21.15.93:0/2230765073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 msgr2=0x5596b822a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.079+0000 7f9746d9d700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b822a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f9746d9d700 1 -- 172.21.15.93:0/2230765073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 msgr2=0x5596b8225750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f9746d9d700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b8225750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f9746d9d700 1 -- 172.21.15.93:0/2230765073 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596b8213230 con 0x5596b821eac0 2023-07-21T22:17:55.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f974659c700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b822a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:55.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f9745d9b700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b8225750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:55.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.080+0000 7f9746d9d700 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9740011960 tx=0x7f9740009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e626b88858e400 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f97357fa700 1 -- 172.21.15.93:0/2230765073 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000e040 con 0x5596b821eac0 2023-07-21T22:17:55.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f97357fa700 1 -- 172.21.15.93:0/2230765073 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f974000f040 con 0x5596b821eac0 2023-07-21T22:17:55.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f97357fa700 1 -- 172.21.15.93:0/2230765073 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740017c30 con 0x5596b821eac0 2023-07-21T22:17:55.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 -- 172.21.15.93:0/2230765073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 msgr2=0x5596b8225c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9740011960 tx=0x7f9740009920 comp rx=0 tx=0).stop 2023-07-21T22:17:55.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 -- 172.21.15.93:0/2230765073 shutdown_connections 2023-07-21T22:17:55.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b821eac0 0x5596b8225c90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8219930 0x5596b822a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 --2- 172.21.15.93:0/2230765073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b8225750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 -- 172.21.15.93:0/2230765073 >> 172.21.15.93:0/2230765073 conn(0x5596b820eef0 msgr2=0x5596b8219d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 -- 172.21.15.93:0/2230765073 shutdown_connections 2023-07-21T22:17:55.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.081+0000 7f9758e72b80 1 -- 172.21.15.93:0/2230765073 wait complete. 2023-07-21T22:17:55.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 -- start start 2023-07-21T22:17:55.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 Processor -- start 2023-07-21T22:17:55.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b822a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8228760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b8228e70 con 0x5596b8219930 2023-07-21T22:17:55.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b8228fe0 con 0x5596b821e1b0 2023-07-21T22:17:55.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596b82291a0 con 0x5596b821eac0 2023-07-21T22:17:55.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9758e72b80 10 client.0 _ll_register_callbacks cb 0x5596b8108760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45542/0 (socket says 172.21.15.93:45542) 2023-07-21T22:17:55.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 -- 172.21.15.93:0/937064287 learned_addr learned my addr 172.21.15.93:0/937064287 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9745d9b700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b822a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f9746d9d700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8228760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 -- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 msgr2=0x5596b8228760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8228760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 -- 172.21.15.93:0/937064287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 msgr2=0x5596b822a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b822a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.149 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.082+0000 7f974659c700 1 -- 172.21.15.93:0/937064287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596b7f98de0 con 0x5596b8219930 2023-07-21T22:17:55.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f974659c700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e00 tx=0x7f973c011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10f03a7dd39cff7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c02b040 con 0x5596b8219930 2023-07-21T22:17:55.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f973c017070 con 0x5596b8219930 2023-07-21T22:17:55.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f973c015070 con 0x5596b8219930 2023-07-21T22:17:55.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f973c0329d0 con 0x5596b8219930 2023-07-21T22:17:55.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 10 client.16020 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5596b7f81c60 con 0x5596b8219930 2023-07-21T22:17:55.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f973c015070 con 0x5596b8219930 2023-07-21T22:17:55.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f973c038030 con 0x5596b8219930 2023-07-21T22:17:55.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:17:55.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 20 client.16020 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8287f90 0x5596b828a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 4 client.16020 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:55.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.083+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596b8126050 con 0x5596b8287f90 2023-07-21T22:17:55.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9745d9b700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8287f90 0x5596b828a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9745d9b700 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8287f90 0x5596b828a3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4c863d1cd051ded server_cookie=95e97d11d080c285 in_seq=0 out_seq=0 2023-07-21T22:17:55.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f972dffb700 10 client.16020 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:55.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f972dffb700 1 -- 172.21.15.93:0/937064287 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596b8126050 con 0x5596b8287f90 2023-07-21T22:17:55.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f972dffb700 10 client.16020 handle_command_reply: tid=0 2023-07-21T22:17:55.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 client.16020 shutdown 2023-07-21T22:17:55.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 msgr2=0x5596b8229fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e00 tx=0x7f973c011b80 comp rx=0 tx=0).stop 2023-07-21T22:17:55.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 shutdown_connections 2023-07-21T22:17:55.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x5596b8287f90 0x5596b828a3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b821eac0 0x5596b8228760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f972dffb700 0 client.16020 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:17:55.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5596b8219930 0x5596b8229fb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 --2- 172.21.15.93:0/937064287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596b821e1b0 0x5596b822a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 >> 172.21.15.93:0/937064287 conn(0x5596b820eef0 msgr2=0x5596b8217a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 shutdown_connections 2023-07-21T22:17:55.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 1 -- 172.21.15.93:0/937064287 wait complete. 2023-07-21T22:17:55.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.084+0000 7f9758e72b80 20 client.16020 trim_cache size 0 max 16384 2023-07-21T22:17:55.159 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:55.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 msgr2=0x7f97300a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 0x7f97300a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f97480ffb10 tx=0x7f973c00b410 comp rx=0 tx=0).stop 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 msgr2=0x7f974819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9738000c00 tx=0x7f97380047b0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 shutdown_connections 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481049e0 0x7f97481948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97480fe9d0 0x7f974819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97300a13e0 0x7f97300a3890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 --2- 172.21.15.93:0/2148602727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f97480ff380 0x7f974819adb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 >> 172.21.15.93:0/2148602727 conn(0x7f97480fa4b0 msgr2=0x7f97480fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 shutdown_connections 2023-07-21T22:17:55.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.085+0000 7f974cced700 1 -- 172.21.15.93:0/2148602727 wait complete. 2023-07-21T22:17:55.162 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:17:55.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 -- 172.21.15.93:0/3248834128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57180fe9d0 msgr2=0x7f57180fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 --2- 172.21.15.93:0/3248834128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57180fe9d0 0x7f57180fee40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 -- 172.21.15.93:0/3248834128 shutdown_connections 2023-07-21T22:17:55.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 --2- 172.21.15.93:0/3248834128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180ff380 0x7f571810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 --2- 172.21.15.93:0/3248834128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57181049e0 0x7f5718104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 --2- 172.21.15.93:0/3248834128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57180fe9d0 0x7f57180fee40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 -- 172.21.15.93:0/3248834128 >> 172.21.15.93:0/3248834128 conn(0x7f57180fa4b0 msgr2=0x7f57180fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 -- 172.21.15.93:0/3248834128 shutdown_connections 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.241+0000 7f571fd3d700 1 -- 172.21.15.93:0/3248834128 wait complete. 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 Processor -- start 2023-07-21T22:17:55.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 -- start start 2023-07-21T22:17:55.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180fe9d0 0x7f57181969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57180ff380 0x7f5718196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57181977f0 con 0x7f57181049e0 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57181a1f60 con 0x7f57180ff380 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571fd3d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57181a20d0 con 0x7f57180fe9d0 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45560/0 (socket says 172.21.15.93:45560) 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571e2da700 1 -- 172.21.15.93:0/1559486409 learned_addr learned my addr 172.21.15.93:0/1559486409 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571d2d8700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57180ff380 0x7f5718196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.242+0000 7f571dad9700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180fe9d0 0x7f57181969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 -- 172.21.15.93:0/1559486409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180fe9d0 msgr2=0x7f57181969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180fe9d0 0x7f57181969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 -- 172.21.15.93:0/1559486409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57180ff380 msgr2=0x7f5718196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57180ff380 0x7f5718196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 -- 172.21.15.93:0/1559486409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140095f0 con 0x7f57181049e0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571e2da700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5710007ee0 tx=0x7f571000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f570effd700 1 -- 172.21.15.93:0/1559486409 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710017070 con 0x7f57181049e0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f570effd700 1 -- 172.21.15.93:0/1559486409 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f571000f040 con 0x7f57181049e0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57181a23b0 con 0x7f57181049e0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f570effd700 1 -- 172.21.15.93:0/1559486409 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710005740 con 0x7f57181049e0 2023-07-21T22:17:55.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.243+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57181a28c0 con 0x7f57181049e0 2023-07-21T22:17:55.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.245+0000 7f570effd700 1 -- 172.21.15.93:0/1559486409 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f571000dc60 con 0x7f57181049e0 2023-07-21T22:17:55.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.245+0000 7f570effd700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 0x7f57040a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.245+0000 7f571dad9700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 0x7f57040a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.293 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.245+0000 7f570effd700 1 -- 172.21.15.93:0/1559486409 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f57100cc7c0 con 0x7f57181049e0 2023-07-21T22:17:55.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.246+0000 7f571dad9700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 0x7f57040a3910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57180ffb10 tx=0x7f5708009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.246+0000 7f572bec2b80 1 Processor -- start 2023-07-21T22:17:55.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 -- start start 2023-07-21T22:17:55.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd3f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abc3fcb0 con 0x55c3abd3a930 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abc92a40 con 0x55c3abd3fac0 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f572bec2b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abc8bed0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45568/0 (socket says 172.21.15.93:45568) 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 -- 172.21.15.93:0/537179335 learned_addr learned my addr 172.21.15.93:0/537179335 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571dad9700 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd3f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571d2d8700 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 -- 172.21.15.93:0/537179335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 msgr2=0x55c3abd3f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd3f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 -- 172.21.15.93:0/537179335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 msgr2=0x55c3abd3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 -- 172.21.15.93:0/537179335 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3abab9de0 con 0x55c3abd3a930 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f571e2da700 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f571000d7b0 tx=0x7f571000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca71fb2ed8a8108 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f5703fff700 1 -- 172.21.15.93:0/537179335 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100b90d0 con 0x55c3abd3a930 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.247+0000 7f5703fff700 1 -- 172.21.15.93:0/537179335 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f571009d630 con 0x55c3abd3a930 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f5703fff700 1 -- 172.21.15.93:0/537179335 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710005b00 con 0x55c3abd3a930 2023-07-21T22:17:55.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 -- 172.21.15.93:0/537179335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 msgr2=0x55c3abd3ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f571000d7b0 tx=0x7f571000df60 comp rx=0 tx=0).stop 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 -- 172.21.15.93:0/537179335 shutdown_connections 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd3ada0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd3f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 --2- 172.21.15.93:0/537179335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 -- 172.21.15.93:0/537179335 >> 172.21.15.93:0/537179335 conn(0x55c3abd2fef0 msgr2=0x55c3abd35830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 -- 172.21.15.93:0/537179335 shutdown_connections 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.248+0000 7f572bec2b80 1 -- 172.21.15.93:0/537179335 wait complete. 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 -- start start 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 Processor -- start 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3f1b0 0x55c3abd43e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abc2bac0 con 0x55c3abd3fac0 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abc2b7c0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abb0f920 con 0x55c3abd3a930 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f572bec2b80 10 client.0 _ll_register_callbacks cb 0x55c3abc29760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f571e2da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45574/0 (socket says 172.21.15.93:45574) 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f571e2da700 1 -- 172.21.15.93:0/4269150353 learned_addr learned my addr 172.21.15.93:0/4269150353 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f571d2d8700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3f1b0 0x55c3abd43e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.249+0000 7f571dad9700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 -- 172.21.15.93:0/4269150353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 msgr2=0x55c3abd46bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd46bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 -- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3f1b0 msgr2=0x55c3abd43e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3f1b0 0x55c3abd43e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 -- 172.21.15.93:0/4269150353 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57181a28c0 con 0x55c3abd3fac0 2023-07-21T22:17:55.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571dad9700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd46bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f571e2da700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5710004c80 tx=0x7f57100a1b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaa6ad133ef86df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100b90d0 con 0x55c3abd3fac0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5710022460 con 0x55c3abd3fac0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3abbe54f0 con 0x55c3abd3fac0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5710084050 con 0x55c3abd3fac0 2023-07-21T22:17:55.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 10 client.16032 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c3abaa2c60 con 0x55c3abd3fac0 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3abbe54f0 con 0x55c3abd3fac0 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f57100a15c0 con 0x55c3abd3fac0 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 10 client.16032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 20 client.16032 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abd54050 0x55c3abd56430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 4 client.16032 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.250+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3abc84eb0 con 0x55c3abd54050 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.251+0000 7f571dad9700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abd54050 0x55c3abd56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.251+0000 7f57017fa700 10 client.16032 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.251+0000 7f571dad9700 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abd54050 0x55c3abd56430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=881dbd0710559030 server_cookie=c1367f0e3cc83ab9 in_seq=0 out_seq=0 2023-07-21T22:17:55.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f57017fa700 1 -- 172.21.15.93:0/4269150353 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3abc84eb0 con 0x55c3abd54050 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f57017fa700 10 client.16032 handle_command_reply: tid=0 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 client.16032 shutdown 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 msgr2=0x55c3abd444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5710004c80 tx=0x7f57100a1b00 comp rx=0 tx=0).stop 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 shutdown_connections 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abd54050 0x55c3abd56430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3fac0 0x55c3abd444d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd46bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 --2- 172.21.15.93:0/4269150353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3f1b0 0x55c3abd43e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 >> 172.21.15.93:0/4269150353 conn(0x55c3abd2fef0 msgr2=0x55c3abd35420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f57017fa700 0 client.16032 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 shutdown_connections 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.252+0000 7f572bec2b80 1 -- 172.21.15.93:0/4269150353 wait complete. 2023-07-21T22:17:55.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.253+0000 7f572bec2b80 20 client.16032 trim_cache size 0 max 16384 2023-07-21T22:17:55.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 22047 4.25 MB/sec warmup 32 sec latency 1307.655 ms 2023-07-21T22:17:55.312 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.267+0000 7f572bec2b80 1 Processor -- start 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.267+0000 7f572bec2b80 1 -- start start 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.267+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.267+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd46750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f572bec2b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abd44860 con 0x55c3abd3a930 2023-07-21T22:17:55.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f572bec2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abd47300 con 0x55c3abd3fac0 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f572bec2b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abd3b1a0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45578/0 (socket says 172.21.15.93:45578) 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 -- 172.21.15.93:0/3577592426 learned_addr learned my addr 172.21.15.93:0/3577592426 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571d2d8700 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571e2da700 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 -- 172.21.15.93:0/3577592426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 msgr2=0x55c3abd46750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd46750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 -- 172.21.15.93:0/3577592426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 msgr2=0x55c3abd38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 -- 172.21.15.93:0/3577592426 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3abae0250 con 0x55c3abd3a930 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f571dad9700 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f57080097d0 tx=0x7f5708014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c436c95ac109131 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f570cff9700 1 -- 172.21.15.93:0/3577592426 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708027070 con 0x55c3abd3a930 2023-07-21T22:17:55.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f570cff9700 1 -- 172.21.15.93:0/3577592426 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5708016070 con 0x55c3abd3a930 2023-07-21T22:17:55.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.268+0000 7f570cff9700 1 -- 172.21.15.93:0/3577592426 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708028bd0 con 0x55c3abd3a930 2023-07-21T22:17:55.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- 172.21.15.93:0/3577592426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 msgr2=0x55c3abd4b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f57080097d0 tx=0x7f5708014000 comp rx=0 tx=0).stop 2023-07-21T22:17:55.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- 172.21.15.93:0/3577592426 shutdown_connections 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abd38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3a930 0x55c3abd4b580 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 --2- 172.21.15.93:0/3577592426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3f1b0 0x55c3abd46750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- 172.21.15.93:0/3577592426 >> 172.21.15.93:0/3577592426 conn(0x55c3abd2fef0 msgr2=0x55c3abd56060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- 172.21.15.93:0/3577592426 shutdown_connections 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- 172.21.15.93:0/3577592426 wait complete. 2023-07-21T22:17:55.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 -- start start 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.269+0000 7f572bec2b80 1 Processor -- start 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abda6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abd9ccd0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abda73f0 con 0x55c3abd3fac0 2023-07-21T22:17:55.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3abda75b0 con 0x55c3abd3a930 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f572bec2b80 10 client.0 _ll_register_callbacks cb 0x7f5708005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45586/0 (socket says 172.21.15.93:45586) 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 -- 172.21.15.93:0/2569697479 learned_addr learned my addr 172.21.15.93:0/2569697479 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571dad9700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd9bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571e2da700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abda6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 -- 172.21.15.93:0/2569697479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 msgr2=0x55c3abd9bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd9bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 -- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 msgr2=0x55c3abda6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abda6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571d2d8700 1 -- 172.21.15.93:0/2569697479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5708011cf0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.270+0000 7f571e2da700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abda6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f571d2d8700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f5714005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd46bcdb6919b94a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571401d040 con 0x55c3abd3f1b0 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5714004700 con 0x55c3abd3f1b0 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f571009d630 con 0x55c3abd3f1b0 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57140048c0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 10 client.16038 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c3abab82b0 con 0x55c3abd3f1b0 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f571009d630 con 0x55c3abd3f1b0 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f571402c020 con 0x55c3abd3f1b0 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 20 client.16038 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abda8cb0 0x55c3abdab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 4 client.16038 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.271+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3abc47050 con 0x55c3abda8cb0 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f571dad9700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abda8cb0 0x55c3abdab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f57017fa700 10 client.16038 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f571dad9700 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abda8cb0 0x55c3abdab0c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69eb7cf71f06394a server_cookie=2d259b3bcb4f034e in_seq=0 out_seq=0 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f57017fa700 1 -- 172.21.15.93:0/2569697479 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3abc47050 con 0x55c3abda8cb0 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f57017fa700 10 client.16038 handle_command_reply: tid=0 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 client.16038 shutdown 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 msgr2=0x55c3abd9c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f5714005320 comp rx=0 tx=0).stop 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 shutdown_connections 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x55c3abda8cb0 0x55c3abdab0c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3abd3fac0 0x55c3abda6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c3abd3a930 0x55c3abd9bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 --2- 172.21.15.93:0/2569697479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3abd3f1b0 0x55c3abd9c530 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f57017fa700 0 client.16038 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:17:55.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 >> 172.21.15.93:0/2569697479 conn(0x55c3abd2fef0 msgr2=0x55c3abd38a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 shutdown_connections 2023-07-21T22:17:55.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 1 -- 172.21.15.93:0/2569697479 wait complete. 2023-07-21T22:17:55.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.272+0000 7f572bec2b80 20 client.16038 trim_cache size 0 max 16384 2023-07-21T22:17:55.329 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:55.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 msgr2=0x7f57040a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 0x7f57040a3910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57180ffb10 tx=0x7f5708009260 comp rx=0 tx=0).stop 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 msgr2=0x7f57181a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5710007ee0 tx=0x7f571000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 shutdown_connections 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f57040a1460 0x7f57040a3910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57181049e0 0x7f57181a1a20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f57180fe9d0 0x7f57181969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 --2- 172.21.15.93:0/1559486409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57180ff380 0x7f5718196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 >> 172.21.15.93:0/1559486409 conn(0x7f57180fa4b0 msgr2=0x7f57180fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 shutdown_connections 2023-07-21T22:17:55.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.273+0000 7f571fd3d700 1 -- 172.21.15.93:0/1559486409 wait complete. 2023-07-21T22:17:55.331 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:55 smithi093 ceph-mon[96579]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:55.332 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:55 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3373111365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:55.332 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:17:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:55 smithi134 ceph-mon[97873]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:55.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:55 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3373111365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:55.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 -- 172.21.15.93:0/1805914489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980ff380 msgr2=0x7f5a9810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/1805914489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980ff380 0x7f5a9810b810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000b0a0 tx=0x7f5a9000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 -- 172.21.15.93:0/1805914489 shutdown_connections 2023-07-21T22:17:55.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/1805914489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980ff380 0x7f5a9810b810 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/1805914489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 0x7f5a98104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/1805914489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980fe9d0 0x7f5a980fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 -- 172.21.15.93:0/1805914489 >> 172.21.15.93:0/1805914489 conn(0x7f5a980fa4b0 msgr2=0x7f5a980fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 -- 172.21.15.93:0/1805914489 shutdown_connections 2023-07-21T22:17:55.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.411+0000 7f5a9f84f700 1 -- 172.21.15.93:0/1805914489 wait complete. 2023-07-21T22:17:55.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 Processor -- start 2023-07-21T22:17:55.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 -- start start 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980ff380 0x7f5a98196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 0x7f5a981a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981a2000 con 0x7f5a980fe9d0 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981a2140 con 0x7f5a981049e0 2023-07-21T22:17:55.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9f84f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981a2280 con 0x7f5a980ff380 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9d5eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9d5eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45598/0 (socket says 172.21.15.93:45598) 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/518842329 learned_addr learned my addr 172.21.15.93:0/518842329 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 0x7f5a981a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980ff380 0x7f5a98196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/518842329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980ff380 msgr2=0x7f5a98196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.412+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980ff380 0x7f5a98196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/518842329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 msgr2=0x7f5a981a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 0x7f5a981a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/518842329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009d00 con 0x7f5a980fe9d0 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c970 tx=0x7f5a8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a8e7fc700 1 -- 172.21.15.93:0/518842329 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88004220 con 0x7f5a980fe9d0 2023-07-21T22:17:55.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a8e7fc700 1 -- 172.21.15.93:0/518842329 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a88007d60 con 0x7f5a980fe9d0 2023-07-21T22:17:55.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a981a24e0 con 0x7f5a980fe9d0 2023-07-21T22:17:55.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a8e7fc700 1 -- 172.21.15.93:0/518842329 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88003c90 con 0x7f5a980fe9d0 2023-07-21T22:17:55.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.413+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a981a29c0 con 0x7f5a980fe9d0 2023-07-21T22:17:55.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.415+0000 7f5a8e7fc700 1 -- 172.21.15.93:0/518842329 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f5a88004380 con 0x7f5a980fe9d0 2023-07-21T22:17:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.416+0000 7f5a8e7fc700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 0x7f5a840a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.416+0000 7f5a8e7fc700 1 -- 172.21.15.93:0/518842329 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5a880cca90 con 0x7f5a980fe9d0 2023-07-21T22:17:55.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.416+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 0x7f5a840a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.416+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 0x7f5a840a3890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a94005fd0 tx=0x7f5a94005e30 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 Processor -- start 2023-07-21T22:17:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 -- start start 2023-07-21T22:17:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b30f7cb0 con 0x55e2b31f2930 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b314aa40 con 0x55e2b31f71b0 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b3143ed0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9ddec700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9ddec700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45610/0 (socket says 172.21.15.93:45610) 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9ddec700 1 -- 172.21.15.93:0/2137006678 learned_addr learned my addr 172.21.15.93:0/2137006678 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9ddec700 1 -- 172.21.15.93:0/2137006678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 msgr2=0x55e2b31f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.417+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a9ddec700 1 -- 172.21.15.93:0/2137006678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 msgr2=0x55e2b31f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a9ddec700 1 -- 172.21.15.93:0/2137006678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2b2f71de0 con 0x55e2b31f2930 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000cf60 tx=0x7f5a9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867efcb1dc07bf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a837fe700 1 -- 172.21.15.93:0/2137006678 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000e040 con 0x55e2b31f2930 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a837fe700 1 -- 172.21.15.93:0/2137006678 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a90009b80 con 0x55e2b31f2930 2023-07-21T22:17:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.418+0000 7f5a837fe700 1 -- 172.21.15.93:0/2137006678 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9001c770 con 0x55e2b31f2930 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/2137006678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 msgr2=0x55e2b31f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000cf60 tx=0x7f5a9000cf90 comp rx=0 tx=0).stop 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/2137006678 shutdown_connections 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b31f2da0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/2137006678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/2137006678 >> 172.21.15.93:0/2137006678 conn(0x55e2b31e7ef0 msgr2=0x55e2b31ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/2137006678 shutdown_connections 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/2137006678 wait complete. 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- start start 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 Processor -- start 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b30e3ac0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b30e37c0 con 0x55e2b31f71b0 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.419+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b2fc7920 con 0x55e2b31f2930 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5aab9d4b80 10 client.0 _ll_register_callbacks cb 0x55e2b30e1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45616/0 (socket says 172.21.15.93:45616) 2023-07-21T22:17:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 -- 172.21.15.93:0/421576085 learned_addr learned my addr 172.21.15.93:0/421576085 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 -- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 msgr2=0x55e2b31febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 -- 172.21.15.93:0/421576085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 msgr2=0x55e2b31fbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31fbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 -- 172.21.15.93:0/421576085 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a981a29c0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5a90003c30 tx=0x7f5a90003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3602d1ac8e1c9c10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9002d070 con 0x55e2b31f7ac0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5a900074a0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2b313ceb0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9000e040 con 0x55e2b31f7ac0 2023-07-21T22:17:55.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5aab9d4b80 10 client.16050 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.420+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5a9810c110 con 0x55e2b31f7ac0 2023-07-21T22:17:55.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2b313ceb0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5a90030020 con 0x55e2b31f7ac0 2023-07-21T22:17:55.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 20 client.16050 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b320c1f0 0x55e2b320e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 4 client.16050 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2b30ff050 con 0x55e2b320c1f0 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b320c1f0 0x55e2b320e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5a80ff9700 10 client.16050 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.421+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b320c1f0 0x55e2b320e600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ff05ab83a0cf94 server_cookie=268de1043f28c200 in_seq=0 out_seq=0 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5a80ff9700 1 -- 172.21.15.93:0/421576085 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e2b30ff050 con 0x55e2b320c1f0 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5a80ff9700 10 client.16050 handle_command_reply: tid=0 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 client.16050 shutdown 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 msgr2=0x55e2b31fc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5a90003c30 tx=0x7f5a90003c60 comp rx=0 tx=0).stop 2023-07-21T22:17:55.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 shutdown_connections 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b320c1f0 0x55e2b320e600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f7ac0 0x55e2b31fc540 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/421576085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b31fbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5a80ff9700 0 client.16050 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 >> 172.21.15.93:0/421576085 conn(0x55e2b31e7ef0 msgr2=0x55e2b31ed420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.481 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 shutdown_connections 2023-07-21T22:17:55.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/421576085 wait complete. 2023-07-21T22:17:55.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.422+0000 7f5aab9d4b80 20 client.16050 trim_cache size 0 max 16384 2023-07-21T22:17:55.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 Processor -- start 2023-07-21T22:17:55.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 -- start start 2023-07-21T22:17:55.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b31eeca0 con 0x55e2b31f71b0 2023-07-21T22:17:55.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b31e79b0 con 0x55e2b31f7ac0 2023-07-21T22:17:55.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.435+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b31ea010 con 0x55e2b31f2930 2023-07-21T22:17:55.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45622/0 (socket says 172.21.15.93:45622) 2023-07-21T22:17:55.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 -- 172.21.15.93:0/3727382680 learned_addr learned my addr 172.21.15.93:0/3727382680 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 -- 172.21.15.93:0/3727382680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 msgr2=0x55e2b31faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 -- 172.21.15.93:0/3727382680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f7ac0 msgr2=0x55e2b31f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 -- 172.21.15.93:0/3727382680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2b2f5ac60 con 0x55e2b31f71b0 2023-07-21T22:17:55.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55e2b31eee70 tx=0x7f5a94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ce9fe0298d949c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a83fff700 1 -- 172.21.15.93:0/3727382680 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9401fd80 con 0x55e2b31f71b0 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a83fff700 1 -- 172.21.15.93:0/3727382680 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a94004cf0 con 0x55e2b31f71b0 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.436+0000 7f5a83fff700 1 -- 172.21.15.93:0/3727382680 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94010430 con 0x55e2b31f71b0 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3727382680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 msgr2=0x55e2b31fe720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55e2b31eee70 tx=0x7f5a94011000 comp rx=0 tx=0).stop 2023-07-21T22:17:55.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3727382680 shutdown_connections 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f7ac0 0x55e2b31f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f2930 0x55e2b31faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3727382680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f71b0 0x55e2b31fe720 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3727382680 >> 172.21.15.93:0/3727382680 conn(0x55e2b31e7ef0 msgr2=0x55e2b32087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3727382680 shutdown_connections 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3727382680 wait complete. 2023-07-21T22:17:55.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 -- start start 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.437+0000 7f5aab9d4b80 1 Processor -- start 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b32034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b3201500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b3201b00 con 0x55e2b31f2930 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b3201c70 con 0x55e2b31f71b0 2023-07-21T22:17:55.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b3201e30 con 0x55e2b31f7ac0 2023-07-21T22:17:55.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 10 client.0 _ll_register_callbacks cb 0x7f5a8800a550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:55.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.496 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45626/0 (socket says 172.21.15.93:45626) 2023-07-21T22:17:55.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/3385184097 learned_addr learned my addr 172.21.15.93:0/3385184097 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:55.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9ddec700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b3201500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b32034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 msgr2=0x55e2b3201500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b3201500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/3385184097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 msgr2=0x55e2b32034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:55.498 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b32034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 -- 172.21.15.93:0/3385184097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a981a24e0 con 0x55e2b31f2930 2023-07-21T22:17:55.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a9d5eb700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5a88095e30 tx=0x7f5a880036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3782889ed7a59733 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:55.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:55.499 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88084040 con 0x55e2b31f2930 2023-07-21T22:17:55.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2b309d4f0 con 0x55e2b31f2930 2023-07-21T22:17:55.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f5a880a1d80 con 0x55e2b31f2930 2023-07-21T22:17:55.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a880b7ac0 con 0x55e2b31f2930 2023-07-21T22:17:55.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 10 client.16056 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:55.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.438+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5a9400ea80 con 0x55e2b31f2930 2023-07-21T22:17:55.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e2b309d4f0 con 0x55e2b31f2930 2023-07-21T22:17:55.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5a880c8070 con 0x55e2b31f2930 2023-07-21T22:17:55.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:55.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:55.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 20 client.16056 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:55.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b3261280 0x55e2b3263690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:55.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 4 client.16056 mds_command: new command op to 24484 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:17:55.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e2b2f92f70 con 0x55e2b3261280 2023-07-21T22:17:55.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b3261280 0x55e2b3263690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:55.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a80ff9700 10 client.16056 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:55.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a9cdea700 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b3261280 0x55e2b3263690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c948dbba7783525e server_cookie=9927d4e3a7769e8d in_seq=0 out_seq=0 2023-07-21T22:17:55.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a80ff9700 1 -- 172.21.15.93:0/3385184097 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e2b2f92f70 con 0x55e2b3261280 2023-07-21T22:17:55.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5a80ff9700 10 client.16056 handle_command_reply: tid=0 2023-07-21T22:17:55.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.439+0000 7f5aab9d4b80 1 client.16056 shutdown 2023-07-21T22:17:55.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 msgr2=0x55e2b3202f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5a88095e30 tx=0x7f5a880036a0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 shutdown_connections 2023-07-21T22:17:55.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e2b3261280 0x55e2b3263690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e2b31f7ac0 0x55e2b3201500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a80ff9700 0 client.16056 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:55.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2b31f2930 0x55e2b3202f60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 --2- 172.21.15.93:0/3385184097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2b31f71b0 0x55e2b32034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 >> 172.21.15.93:0/3385184097 conn(0x55e2b31e7ef0 msgr2=0x55e2b31f0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 shutdown_connections 2023-07-21T22:17:55.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 1 -- 172.21.15.93:0/3385184097 wait complete. 2023-07-21T22:17:55.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5aab9d4b80 20 client.16056 trim_cache size 0 max 16384 2023-07-21T22:17:55.509 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:17:55.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 msgr2=0x7f5a840a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 0x7f5a840a3890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a94005fd0 tx=0x7f5a94005e30 comp rx=0 tx=0).stop 2023-07-21T22:17:55.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 msgr2=0x7f5a98196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:55.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c970 tx=0x7f5a8800cc80 comp rx=0 tx=0).stop 2023-07-21T22:17:55.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 shutdown_connections 2023-07-21T22:17:55.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a981049e0 0x7f5a981a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a840a13e0 0x7f5a840a3890 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a980fe9d0 0x7f5a98196a40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 --2- 172.21.15.93:0/518842329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a980ff380 0x7f5a98196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:55.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 >> 172.21.15.93:0/518842329 conn(0x7f5a980fa4b0 msgr2=0x7f5a980fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:55.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 shutdown_connections 2023-07-21T22:17:55.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:55.440+0000 7f5a9f84f700 1 -- 172.21.15.93:0/518842329 wait complete. 2023-07-21T22:17:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:55 smithi194 ceph-mon[98288]: pgmap v47: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:55 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3373111365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:17:56.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 22115 4.13 MB/sec warmup 33 sec latency 1200.350 ms 2023-07-21T22:17:56.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:17:56.516 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:17:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 -- 172.21.15.93:0/2638508034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 msgr2=0x7f2d8010b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 --2- 172.21.15.93:0/2638508034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d8010b820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800b0a0 tx=0x7f2d7800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 -- 172.21.15.93:0/2638508034 shutdown_connections 2023-07-21T22:17:56.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 --2- 172.21.15.93:0/2638508034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d8010b820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 --2- 172.21.15.93:0/2638508034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d801049f0 0x7f2d80104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 --2- 172.21.15.93:0/2638508034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d800fe9e0 0x7f2d800fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 -- 172.21.15.93:0/2638508034 >> 172.21.15.93:0/2638508034 conn(0x7f2d800fa4e0 msgr2=0x7f2d800fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 -- 172.21.15.93:0/2638508034 shutdown_connections 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 -- 172.21.15.93:0/2638508034 wait complete. 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.608+0000 7f2d87dea700 1 Processor -- start 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 -- start start 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800fe9e0 0x7f2d80196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d801049f0 0x7f2d801a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80197920 con 0x7f2d800ff390 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801a2090 con 0x7f2d800fe9e0 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801a2200 con 0x7f2d801049f0 2023-07-21T22:17:56.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45648/0 (socket says 172.21.15.93:45648) 2023-07-21T22:17:56.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 -- 172.21.15.93:0/166408119 learned_addr learned my addr 172.21.15.93:0/166408119 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d86387700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d801049f0 0x7f2d801a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85b86700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800fe9e0 0x7f2d80196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 -- 172.21.15.93:0/166408119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d801049f0 msgr2=0x7f2d801a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d801049f0 0x7f2d801a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 -- 172.21.15.93:0/166408119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800fe9e0 msgr2=0x7f2d80196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800fe9e0 0x7f2d80196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 -- 172.21.15.93:0/166408119 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009d00 con 0x7f2d800ff390 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d85385700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c920 tx=0x7f2d7c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d76ffd700 1 -- 172.21.15.93:0/166408119 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007b00 con 0x7f2d800ff390 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d76ffd700 1 -- 172.21.15.93:0/166408119 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7c004cf0 con 0x7f2d800ff390 2023-07-21T22:17:56.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d801a24e0 con 0x7f2d800ff390 2023-07-21T22:17:56.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.609+0000 7f2d76ffd700 1 -- 172.21.15.93:0/166408119 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005610 con 0x7f2d800ff390 2023-07-21T22:17:56.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.610+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d801a2a40 con 0x7f2d800ff390 2023-07-21T22:17:56.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.611+0000 7f2d76ffd700 1 -- 172.21.15.93:0/166408119 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f2d7c004750 con 0x7f2d800ff390 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.612+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 0x7f2d6c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.612+0000 7f2d85b86700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 0x7f2d6c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.612+0000 7f2d76ffd700 1 -- 172.21.15.93:0/166408119 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2d7c0ccee0 con 0x7f2d800ff390 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.613+0000 7f2d85b86700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 0x7f2d6c0a3910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005fd0 tx=0x7f2d70005dd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.613+0000 7f2d93f6fb80 1 Processor -- start 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 -- start start 2023-07-21T22:17:56.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e871600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 0x557c8e86c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e771d30 con 0x557c8e86c9b0 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e7c4ac0 con 0x557c8e871230 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e7bdf50 con 0x557c8e871b40 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45658/0 (socket says 172.21.15.93:45658) 2023-07-21T22:17:56.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 -- 172.21.15.93:0/2825597733 learned_addr learned my addr 172.21.15.93:0/2825597733 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d85b86700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e871600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d85385700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 0x557c8e86c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 -- 172.21.15.93:0/2825597733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 msgr2=0x557c8e86c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 0x557c8e86c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 -- 172.21.15.93:0/2825597733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 msgr2=0x557c8e871600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e871600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 -- 172.21.15.93:0/2825597733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8e5ebe60 con 0x557c8e86c9b0 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d85b86700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e871600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d85385700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 0x557c8e86c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d86387700 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d780091d0 tx=0x7f2d78009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b973d5d161e8ce7a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.614+0000 7f2d6bfff700 1 -- 172.21.15.93:0/2825597733 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800e050 con 0x557c8e86c9b0 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d6bfff700 1 -- 172.21.15.93:0/2825597733 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d780094d0 con 0x557c8e86c9b0 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d6bfff700 1 -- 172.21.15.93:0/2825597733 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780057c0 con 0x557c8e86c9b0 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2825597733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 msgr2=0x557c8e86ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d780091d0 tx=0x7f2d78009200 comp rx=0 tx=0).stop 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2825597733 shutdown_connections 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e86ce20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e871600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2825597733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871b40 0x557c8e86c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2825597733 >> 172.21.15.93:0/2825597733 conn(0x557c8e861f70 msgr2=0x557c8e8678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2825597733 shutdown_connections 2023-07-21T22:17:56.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.615+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2825597733 wait complete. 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 -- start start 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 Processor -- start 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e86c9b0 0x557c8e878ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e875db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e75db40 con 0x557c8e871b40 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e75d840 con 0x557c8e871230 2023-07-21T22:17:56.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e6419a0 con 0x557c8e86c9b0 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d93f6fb80 10 client.0 _ll_register_callbacks cb 0x557c8e75b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d86387700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d86387700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45664/0 (socket says 172.21.15.93:45664) 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d86387700 1 -- 172.21.15.93:0/2512995042 learned_addr learned my addr 172.21.15.93:0/2512995042 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.616+0000 7f2d85b86700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e86c9b0 0x557c8e878ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d85385700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e875db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 -- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e86c9b0 msgr2=0x557c8e878ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e86c9b0 0x557c8e878ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 -- 172.21.15.93:0/2512995042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 msgr2=0x557c8e875db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e875db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 -- 172.21.15.93:0/2512995042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8010c120 con 0x557c8e871b40 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d86387700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d78004f30 tx=0x7f2d780076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3322fbb1af1d5d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78005730 con 0x557c8e871b40 2023-07-21T22:17:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2d78005890 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c8e717570 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7800e020 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d93f6fb80 10 client.16068 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c8e5d4ce0 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c8e717570 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.617+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2d7800b0d0 con 0x557c8e871b40 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 10 client.16068 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 20 client.16068 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e885e90 0x557c8e888270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 4 client.16068 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c8e7b6f30 con 0x557c8e885e90 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d85b86700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e885e90 0x557c8e888270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d85b86700 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e885e90 0x557c8e888270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362e7df373e4bdc3 server_cookie=2e6e10c6be671fcd in_seq=0 out_seq=0 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.618+0000 7f2d697fa700 10 client.16068 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d697fa700 1 -- 172.21.15.93:0/2512995042 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c8e7b6f30 con 0x557c8e885e90 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d697fa700 10 client.16068 handle_command_reply: tid=0 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 client.16068 shutdown 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 msgr2=0x557c8e876350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d78004f30 tx=0x7f2d780076d0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 shutdown_connections 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e885e90 0x557c8e888270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e871b40 0x557c8e876350 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d697fa700 0 client.16068 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e86c9b0 0x557c8e878ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/2512995042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871230 0x557c8e875db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 >> 172.21.15.93:0/2512995042 conn(0x557c8e861f70 msgr2=0x557c8e8674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 shutdown_connections 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/2512995042 wait complete. 2023-07-21T22:17:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.619+0000 7f2d93f6fb80 20 client.16068 trim_cache size 0 max 16384 2023-07-21T22:17:56.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 Processor -- start 2023-07-21T22:17:56.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 -- start start 2023-07-21T22:17:56.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e8786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e86fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e882df0 con 0x557c8e86c9b0 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e861a30 con 0x557c8e871b40 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e864090 con 0x557c8e871230 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45668/0 (socket says 172.21.15.93:45668) 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 -- 172.21.15.93:0/1828150958 learned_addr learned my addr 172.21.15.93:0/1828150958 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.682 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85385700 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e8786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d86387700 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e86fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 -- 172.21.15.93:0/1828150958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 msgr2=0x557c8e8786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e8786d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 -- 172.21.15.93:0/1828150958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 msgr2=0x557c8e86fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e86fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.636+0000 7f2d85b86700 1 -- 172.21.15.93:0/1828150958 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8e875920 con 0x557c8e86c9b0 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d85b86700 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d70013040 tx=0x7f2d70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92cae5444232198 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d74ff9700 1 -- 172.21.15.93:0/1828150958 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70023040 con 0x557c8e86c9b0 2023-07-21T22:17:56.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d74ff9700 1 -- 172.21.15.93:0/1828150958 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d700254c0 con 0x557c8e86c9b0 2023-07-21T22:17:56.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d74ff9700 1 -- 172.21.15.93:0/1828150958 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70003b20 con 0x557c8e86c9b0 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/1828150958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 msgr2=0x557c8e87d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d70013040 tx=0x7f2d70014000 comp rx=0 tx=0).stop 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/1828150958 shutdown_connections 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e86fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87d590 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/1828150958 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e8786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/1828150958 >> 172.21.15.93:0/1828150958 conn(0x557c8e861f70 msgr2=0x557c8e86a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/1828150958 shutdown_connections 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.637+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/1828150958 wait complete. 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 -- start start 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 Processor -- start 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e87c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e8d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e87cf00 con 0x557c8e86c9b0 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e8da270 con 0x557c8e871b40 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e8da3e0 con 0x557c8e871230 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d93f6fb80 10 client.0 _ll_register_callbacks cb 0x7f2d78009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45674/0 (socket says 172.21.15.93:45674) 2023-07-21T22:17:56.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 -- 172.21.15.93:0/706715637 learned_addr learned my addr 172.21.15.93:0/706715637 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d86387700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e8d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 -- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 msgr2=0x557c8e87c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85385700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e87c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e87c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 -- 172.21.15.93:0/706715637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 msgr2=0x557c8e8d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e8d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d85b86700 1 -- 172.21.15.93:0/706715637 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7000ed10 con 0x557c8e86c9b0 2023-07-21T22:17:56.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.638+0000 7f2d86387700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e8d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d85b86700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000a7f0 tx=0x7f2d700053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d0e3938ad79da8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70023040 con 0x557c8e86c9b0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2d70004960 con 0x557c8e86c9b0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d780094d0 con 0x557c8e86c9b0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d700056f0 con 0x557c8e86c9b0 2023-07-21T22:17:56.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 10 client.16074 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557c8e6122d0 con 0x557c8e86c9b0 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d780094d0 con 0x557c8e86c9b0 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2d70034020 con 0x557c8e86c9b0 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 10 client.16074 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 20 client.16074 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e8dbbd0 0x557c8e8ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 4 client.16074 mds_command: new command op to 24484 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.639+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557c8e7790d0 con 0x557c8e8dbbd0 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.640+0000 7f2d85385700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e8dbbd0 0x557c8e8ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.640+0000 7f2d697fa700 10 client.16074 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.640+0000 7f2d85385700 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e8dbbd0 0x557c8e8ddfe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93339f0df755dc1 server_cookie=c043efb9d7528dd3 in_seq=0 out_seq=0 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d697fa700 1 -- 172.21.15.93:0/706715637 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557c8e7790d0 con 0x557c8e8dbbd0 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d697fa700 10 client.16074 handle_command_reply: tid=0 2023-07-21T22:17:56.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 client.16074 shutdown 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 msgr2=0x557c8e87c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000a7f0 tx=0x7f2d700053a0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 shutdown_connections 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c8e8dbbd0 0x557c8e8ddfe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c8e871b40 0x557c8e8d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c8e86c9b0 0x557c8e87c2b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 --2- 172.21.15.93:0/706715637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c8e871230 0x557c8e87c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.694 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d697fa700 0 client.16074 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 >> 172.21.15.93:0/706715637 conn(0x557c8e861f70 msgr2=0x557c8e86b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 shutdown_connections 2023-07-21T22:17:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 1 -- 172.21.15.93:0/706715637 wait complete. 2023-07-21T22:17:56.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.641+0000 7f2d93f6fb80 20 client.16074 trim_cache size 0 max 16384 2023-07-21T22:17:56.695 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:56.696 INFO:teuthology.orchestra.run.smithi093.stdout: "return_code": 0, 2023-07-21T22:17:56.696 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_tag": "13e63bbb-e5e5-4f6f-a1e3-cd84017a2a1f", 2023-07-21T22:17:56.696 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": "asynchronous" 2023-07-21T22:17:56.696 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 msgr2=0x7f2d6c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 0x7f2d6c0a3910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005fd0 tx=0x7f2d70005dd0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 msgr2=0x7f2d80197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c920 tx=0x7f2d7c00cce0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 shutdown_connections 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d801049f0 0x7f2d801a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d800fe9e0 0x7f2d80196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d6c0a1460 0x7f2d6c0a3910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 --2- 172.21.15.93:0/166408119 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff390 0x7f2d80197010 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 >> 172.21.15.93:0/166408119 conn(0x7f2d800fa4e0 msgr2=0x7f2d800fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 shutdown_connections 2023-07-21T22:17:56.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.642+0000 7f2d87dea700 1 -- 172.21.15.93:0/166408119 wait complete. 2023-07-21T22:17:56.699 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:17:56.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 -- 172.21.15.93:0/1473837857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe258103430 msgr2=0x7fe25810b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1473837857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe258103430 0x7fe25810b000 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c240 tx=0x7fe25000c550 comp rx=0 tx=0).stop 2023-07-21T22:17:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 -- 172.21.15.93:0/1473837857 shutdown_connections 2023-07-21T22:17:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1473837857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe258103430 0x7fe25810b000 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1473837857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 0x7fe2580fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1473837857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2580fee10 0x7fe258102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 -- 172.21.15.93:0/1473837857 >> 172.21.15.93:0/1473837857 conn(0x7fe258076200 msgr2=0x7fe258076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 -- 172.21.15.93:0/1473837857 shutdown_connections 2023-07-21T22:17:56.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.782+0000 7fe25d09b700 1 -- 172.21.15.93:0/1473837857 wait complete. 2023-07-21T22:17:56.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.783+0000 7fe25d09b700 1 Processor -- start 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.783+0000 7fe25d09b700 1 -- start start 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 0x7fe2581998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe258103430 0x7fe258193950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258193fc0 con 0x7fe2580fee10 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258194130 con 0x7fe2580fe470 2023-07-21T22:17:56.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2581942f0 con 0x7fe258103430 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45690/0 (socket says 172.21.15.93:45690) 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 -- 172.21.15.93:0/1776945886 learned_addr learned my addr 172.21.15.93:0/1776945886 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 0x7fe2581998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25759e700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe258103430 0x7fe258193950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 -- 172.21.15.93:0/1776945886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe258103430 msgr2=0x7fe258193950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe258103430 0x7fe258193950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 -- 172.21.15.93:0/1776945886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 msgr2=0x7fe2581998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 0x7fe2581998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 -- 172.21.15.93:0/1776945886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25000ae10 con 0x7fe2580fee10 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25659c700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e940 tx=0x7fe24c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe23ffff700 1 -- 172.21.15.93:0/1776945886 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c009730 con 0x7fe2580fee10 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe23ffff700 1 -- 172.21.15.93:0/1776945886 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24c010430 con 0x7fe2580fee10 2023-07-21T22:17:56.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2581945d0 con 0x7fe2580fee10 2023-07-21T22:17:56.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe23ffff700 1 -- 172.21.15.93:0/1776945886 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00cea0 con 0x7fe2580fee10 2023-07-21T22:17:56.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.784+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2581a08d0 con 0x7fe2580fee10 2023-07-21T22:17:56.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.787+0000 7fe23ffff700 1 -- 172.21.15.93:0/1776945886 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fe24c010810 con 0x7fe2580fee10 2023-07-21T22:17:56.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.787+0000 7fe23ffff700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 0x7fe2400a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.787+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 0x7fe2400a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.787+0000 7fe23ffff700 1 -- 172.21.15.93:0/1776945886 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe24c014070 con 0x7fe2580fee10 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 0x7fe2400a3910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe248009c60 tx=0x7fe248009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 Processor -- start 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 -- start start 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d2053d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 0x555d20538350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20538da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2043dcb0 con 0x555d2053dac0 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20490a40 con 0x555d2053d1b0 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.788+0000 7fe269220b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20489ed0 con 0x555d20538930 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 0x555d20538350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 0x555d20538350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45698/0 (socket says 172.21.15.93:45698) 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25659c700 1 -- 172.21.15.93:0/4019384985 learned_addr learned my addr 172.21.15.93:0/4019384985 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20538da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe256d9d700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d2053d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 -- 172.21.15.93:0/4019384985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 msgr2=0x555d2053d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d2053d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 -- 172.21.15.93:0/4019384985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 msgr2=0x555d20538350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 0x555d20538350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 -- 172.21.15.93:0/4019384985 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d202b7de0 con 0x555d20538930 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe256d9d700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d2053d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:56.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe25759e700 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20538da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c8e0 tx=0x7fe25000ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12992d684283ee70 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe23d7fa700 1 -- 172.21.15.93:0/4019384985 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000f040 con 0x555d20538930 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe23d7fa700 1 -- 172.21.15.93:0/4019384985 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe250004d90 con 0x555d20538930 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.789+0000 7fe23d7fa700 1 -- 172.21.15.93:0/4019384985 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25001d6a0 con 0x555d20538930 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- 172.21.15.93:0/4019384985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 msgr2=0x555d20538da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20538da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe25000c8e0 tx=0x7fe25000ad50 comp rx=0 tx=0).stop 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- 172.21.15.93:0/4019384985 shutdown_connections 2023-07-21T22:17:56.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20538da0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d2053d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 --2- 172.21.15.93:0/4019384985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053dac0 0x555d20538350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- 172.21.15.93:0/4019384985 >> 172.21.15.93:0/4019384985 conn(0x555d2052def0 msgr2=0x555d20533830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- 172.21.15.93:0/4019384985 shutdown_connections 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- 172.21.15.93:0/4019384985 wait complete. 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.790+0000 7fe269220b80 1 -- start start 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 Processor -- start 2023-07-21T22:17:56.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d20541de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053dac0 0x555d20542520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20429ac0 con 0x555d20538930 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d204297c0 con 0x555d2053d1b0 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2030d920 con 0x555d2053dac0 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe269220b80 10 client.0 _ll_register_callbacks cb 0x555d20427760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:56.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45702/0 (socket says 172.21.15.93:45702) 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 -- 172.21.15.93:0/607145087 learned_addr learned my addr 172.21.15.93:0/607145087 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe25759e700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053dac0 0x555d20542520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe25659c700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d20541de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 -- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053dac0 msgr2=0x555d20542520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053dac0 0x555d20542520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 -- 172.21.15.93:0/607145087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 msgr2=0x555d20541de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d20541de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 -- 172.21.15.93:0/607145087 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe248017070 con 0x555d20538930 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe256d9d700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe248003900 tx=0x7fe2480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4103a482283d5cdd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe23aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.791+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24801f6b0 con 0x555d20538930 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe24801f810 con 0x555d20538930 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d203e34f0 con 0x555d20538930 2023-07-21T22:17:56.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24801f9d0 con 0x555d20538930 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 10 client.16086 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe25810b810 con 0x555d20538930 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d203e34f0 con 0x555d20538930 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe248031ce0 con 0x555d20538930 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 20 client.16086 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:56.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d20552450 0x555d20554860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 4 client.16086 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d20482eb0 con 0x555d20552450 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.792+0000 7fe25659c700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d20552450 0x555d20554860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe25659c700 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d20552450 0x555d20554860 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d66046833bf1f84 server_cookie=411e8f0a7608c048 in_seq=0 out_seq=0 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe23aff5700 10 client.16086 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe23aff5700 1 -- 172.21.15.93:0/607145087 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d20482eb0 con 0x555d20552450 2023-07-21T22:17:56.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe23aff5700 10 client.16086 handle_command_reply: tid=0 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 client.16086 shutdown 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 msgr2=0x555d20544bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe248003900 tx=0x7fe2480047c0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 shutdown_connections 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d20552450 0x555d20554860 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053dac0 0x555d20542520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20544bd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 --2- 172.21.15.93:0/607145087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053d1b0 0x555d20541de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe23aff5700 0 client.16086 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 >> 172.21.15.93:0/607145087 conn(0x555d2052def0 msgr2=0x555d20533420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 shutdown_connections 2023-07-21T22:17:56.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.793+0000 7fe269220b80 1 -- 172.21.15.93:0/607145087 wait complete. 2023-07-21T22:17:56.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.794+0000 7fe269220b80 20 client.16086 trim_cache size 0 max 16384 2023-07-21T22:17:56.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 Processor -- start 2023-07-21T22:17:56.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 -- start start 2023-07-21T22:17:56.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053d1b0 0x555d20544700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20541350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20534ea0 con 0x555d20538930 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20542b30 con 0x555d2053dac0 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.812+0000 7fe269220b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20540120 con 0x555d2053d1b0 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45710/0 (socket says 172.21.15.93:45710) 2023-07-21T22:17:56.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 -- 172.21.15.93:0/1225259331 learned_addr learned my addr 172.21.15.93:0/1225259331 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe25759e700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20541350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe25659c700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053d1b0 0x555d20544700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 -- 172.21.15.93:0/1225259331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053d1b0 msgr2=0x555d20544700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053d1b0 0x555d20544700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 -- 172.21.15.93:0/1225259331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 msgr2=0x555d20541350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20541350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 -- 172.21.15.93:0/1225259331 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d202a0c60 con 0x555d20538930 2023-07-21T22:17:56.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe25759e700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20541350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:56.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe256d9d700 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe248009b20 tx=0x7fe248009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e453664ead931f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe23dffb700 1 -- 172.21.15.93:0/1225259331 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480287b0 con 0x555d20538930 2023-07-21T22:17:56.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe23dffb700 1 -- 172.21.15.93:0/1225259331 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe248004cf0 con 0x555d20538930 2023-07-21T22:17:56.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.813+0000 7fe23dffb700 1 -- 172.21.15.93:0/1225259331 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248005550 con 0x555d20538930 2023-07-21T22:17:56.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- 172.21.15.93:0/1225259331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 msgr2=0x555d20537fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe248009b20 tx=0x7fe248009eb0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- 172.21.15.93:0/1225259331 shutdown_connections 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20541350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d20538930 0x555d20537fb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 --2- 172.21.15.93:0/1225259331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d2053d1b0 0x555d20544700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- 172.21.15.93:0/1225259331 >> 172.21.15.93:0/1225259331 conn(0x555d2052def0 msgr2=0x555d205378a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- 172.21.15.93:0/1225259331 shutdown_connections 2023-07-21T22:17:56.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- 172.21.15.93:0/1225259331 wait complete. 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.814+0000 7fe269220b80 1 -- start start 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 Processor -- start 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20549530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20547e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d205484f0 con 0x555d2053d1b0 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20548660 con 0x555d2053dac0 2023-07-21T22:17:56.865 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d205a62e0 con 0x555d20538930 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe269220b80 10 client.0 _ll_register_callbacks cb 0x7fe248004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe25659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45718/0 (socket says 172.21.15.93:45718) 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe25659c700 1 -- 172.21.15.93:0/486440368 learned_addr learned my addr 172.21.15.93:0/486440368 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20549530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe25759e700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20547e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 -- 172.21.15.93:0/486440368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 msgr2=0x555d20547e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20547e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 -- 172.21.15.93:0/486440368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 msgr2=0x555d20547910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe256d9d700 1 -- 172.21.15.93:0/486440368 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d202b7de0 con 0x555d20538930 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.815+0000 7fe25659c700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe256d9d700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20549530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe248028540 tx=0x7fe248004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a798ad4392ce9ad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248005150 con 0x555d20538930 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe250004d90 con 0x555d20538930 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe248036030 con 0x555d20538930 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2480052b0 con 0x555d20538930 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 10 client.25537 fetch_fsmap learned FSMap version 30 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe25810b810 con 0x555d20538930 2023-07-21T22:17:56.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe24804f080 con 0x555d20538930 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe248041030 con 0x555d20538930 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 10 client.25537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 20 client.25537 populate_metadata read hostname 'smithi093' 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d205a75a0 0x555d205a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 4 client.25537 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.816+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d20445050 con 0x555d205a75a0 2023-07-21T22:17:56.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe25659c700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d205a75a0 0x555d205a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe25659c700 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d205a75a0 0x555d205a99b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7625d3f58267a6e6 server_cookie=6dcbae11f6f54012 in_seq=0 out_seq=0 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe23aff5700 10 client.25537 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe23aff5700 1 -- 172.21.15.93:0/486440368 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d20445050 con 0x555d205a75a0 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe23aff5700 10 client.25537 handle_command_reply: tid=0 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 client.25537 shutdown 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 msgr2=0x555d20549530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20549530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe248028540 tx=0x7fe248004e60 comp rx=0 tx=0).stop 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 shutdown_connections 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555d205a75a0 0x555d205a99b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d2053dac0 0x555d20547e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe23aff5700 0 client.25537 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d20538930 0x555d20549530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 --2- 172.21.15.93:0/486440368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d2053d1b0 0x555d20547910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 >> 172.21.15.93:0/486440368 conn(0x555d2052def0 msgr2=0x555d2053c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 shutdown_connections 2023-07-21T22:17:56.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 1 -- 172.21.15.93:0/486440368 wait complete. 2023-07-21T22:17:56.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.817+0000 7fe269220b80 20 client.25537 trim_cache size 0 max 16384 2023-07-21T22:17:56.872 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:17:56.872 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": { 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: "13e63bbb-e5e5-4f6f-a1e3-cd84017a2a1f": { 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/", 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: "tag": "13e63bbb-e5e5-4f6f-a1e3-cd84017a2a1f", 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: "options": "recursive,force" 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:17:56.873 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:17:56.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 msgr2=0x7fe2400a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 0x7fe2400a3910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe248009c60 tx=0x7fe248009260 comp rx=0 tx=0).stop 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 msgr2=0x7fe258199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e940 tx=0x7fe24c00ec50 comp rx=0 tx=0).stop 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 shutdown_connections 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe258103430 0x7fe258193950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2580fe470 0x7fe2581998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2400a1460 0x7fe2400a3910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 --2- 172.21.15.93:0/1776945886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe2580fee10 0x7fe258199e20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:56.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 >> 172.21.15.93:0/1776945886 conn(0x7fe258076200 msgr2=0x7fe258101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:56.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 shutdown_connections 2023-07-21T22:17:56.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:17:56.818+0000 7fe25d09b700 1 -- 172.21.15.93:0/1776945886 wait complete. 2023-07-21T22:17:56.876 INFO:tasks.cephfs.filesystem:scrub status for tag:13e63bbb-e5e5-4f6f-a1e3-cd84017a2a1f - {'path': '/', 'tag': '13e63bbb-e5e5-4f6f-a1e3-cd84017a2a1f', 'options': 'recursive,force'} 2023-07-21T22:17:57.087 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.648+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021f [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT auth v417 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.455314+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff500680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v20>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021f:[<0x10000000215/ROSE.CPT v417>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.087 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.648+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021b [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT auth v383 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.434315+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff500c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v12>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021b:[<0x10000000215/TEAPOT2.CPT v383>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.088 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.649+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS auth{0=1} v494 ap=1 s=260096 n(v0 rc2023-07-21T22:17:50.323747+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@31},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55694da89080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v2>,<0x10000000009/EXCEL v3189>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000020d:[<0x1000000020c/PCMAGCD.XLS v494>,<0x10000000009/EXCEL v3848>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.088 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.649+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020e [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS auth{0=1} v573 ap=1 s=260096 n(v0 rc2023-07-21T22:17:50.485744+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@31},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55694da89b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v4>,<0x10000000009/EXCEL v3189>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000020e:[<0x1000000020c/SALES.XLS v573>,<0x10000000009/EXCEL v3848>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.088 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.649+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021a [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT auth v381 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.429315+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff501180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v10>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021a:[<0x10000000215/MAILBOX2.CPT v381>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.089 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.649+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000220 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT auth v364 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.460314+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff501700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v22>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000220:[<0x10000000215/CORELDRW.CDT v364>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.089 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.651+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000241 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL auth v15 ap=1 s=327680 n(v0 rc2023-07-21T22:17:29.994176+0000 b327680 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55694db0b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v2>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000241:[<0x10000000240/SMALL.FIL v15>,<0x10000000009/SEED v3494>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.089 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.651+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000242 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL auth v17 ap=1 s=1048576 n(v0 rc2023-07-21T22:17:29.995176+0000 b1048576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55694db0bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v4>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000242:[<0x10000000240/MEDIUM.FIL v17>,<0x10000000009/SEED v3494>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.089 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.649+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021e [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR auth v402 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.450314+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff568580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v18>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021e:[<0x10000000215/GRAPHIC2.CDR v402>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.090 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.649+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000218 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL auth v357 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.419315+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff569080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v6>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000218:[<0x10000000215/ARTISTIC.ACL v357>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.090 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.651+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000243 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL auth v19 ap=1 s=1867776 n(v0 rc2023-07-21T22:17:29.995176+0000 b1867776 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@6},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55694db0c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v6>,<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000243:[<0x10000000240/LARGE.FIL v19>,<0x10000000009/SEED v3494>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.090 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:17:56.670+0000 7f6babc80700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023f [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB auth v198 ap=1 s=260096 n(v0 rc2023-07-21T22:17:52.886693+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@12},l=24968 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55694db0a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v156>,<0x10000000009/ACCESS v3259>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000023f:[<0x1000000023e/FASTENER.MDB v198>,<0x10000000009/ACCESS v3862>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.091 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.650+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021c [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT auth v385 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.440314+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff569b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v14>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021c:[<0x10000000215/SIGN2.CPT v385>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.091 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.650+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000217 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG auth v355 ap=1 s=260096 n(v0 rc2023-07-21T22:17:29.988176+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff56a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v4>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000217:[<0x10000000215/CDRBARS.CFG v355>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.091 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.650+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000216 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG auth v353 ap=1 s=260096 n(v0 rc2023-07-21T22:17:29.994176+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff56b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v2>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000216:[<0x10000000215/CDRROLS.CFG v353>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.091 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.651+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000219 [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR auth v366 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.424315+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff56b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v8>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x10000000219:[<0x10000000215/GRAPHIC1.CDR v366>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.091 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:17:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:17:56.651+0000 7fb1fccb7700 -1 mds.3.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000021d [3,head] /volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR auth v387 ap=1 s=260096 n(v0 rc2023-07-21T22:17:23.445314+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24968=0-4194304@2} caps={24968=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@21},l=24968 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55c4ff56ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v16>,<0x10000000009/COREL v3311>,<0x10000000008/~dmtmp v93>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v34>,<0x10000000001/sv_1 v55>,<0x10000000000/_nogroup v40>,<0x1/volumes v38>]//[]","memoryvalue":"(3)0x1000000021d:[<0x10000000215/TRAIN.CDR v387>,<0x10000000009/COREL v3852>,<0x10000000008/~dmtmp v232>,<0x10000000007/client0 v56>,<0x10000000006/clients v47>,<0x10000000005/tmp v47>,<0x100000001fb/client.0 v48>,<0x100000001fa/a9794218-501c-42de-ad2b-a96b2b0f2880 v50>,<0x10000000001/sv_1 v81>,<0x10000000000/_nogroup v50>,<0x1/volumes v20>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:17:57.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 23775 4.20 MB/sec warmup 34 sec latency 461.576 ms 2023-07-21T22:17:57.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:57.379 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.380 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.383 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:57 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:57 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: pgmap v48: 97 pgs: 97 active+clean; 25 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 3.0 MiB/s wr, 86 op/s 2023-07-21T22:17:57.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:57.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021b (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021a (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000020d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000020e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000220 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021e (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000218 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021c (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000217 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000216 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000219 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000241 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000021d (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR) see mds.f log and `damage ls` output for details 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000242 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x10000000243 (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: Scrub error on inode 0x1000000023f (/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB) see mds.c log and `damage ls` output for details 2023-07-21T22:17:57.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:57 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 25445 4.42 MB/sec warmup 35 sec latency 10.210 ms 2023-07-21T22:17:58.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:58 smithi093 ceph-mon[96579]: scrub queued for path: / 2023-07-21T22:17:58.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:58 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:58 smithi093 ceph-mon[96579]: scrub summary: active paths [/] 2023-07-21T22:17:58.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:58 smithi093 ceph-mon[96579]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:58 smithi093 ceph-mon[96579]: mgrmap e24: x(active, since 91s), standbys: z, y 2023-07-21T22:17:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:58 smithi194 ceph-mon[98288]: scrub queued for path: / 2023-07-21T22:17:58.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:58 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:58 smithi194 ceph-mon[98288]: scrub summary: active paths [/] 2023-07-21T22:17:58.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:58 smithi194 ceph-mon[98288]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:58 smithi194 ceph-mon[98288]: mgrmap e24: x(active, since 91s), standbys: z, y 2023-07-21T22:17:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:58 smithi134 ceph-mon[97873]: scrub queued for path: / 2023-07-21T22:17:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:58 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:58 smithi134 ceph-mon[97873]: scrub summary: active paths [/] 2023-07-21T22:17:58.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:58 smithi134 ceph-mon[97873]: scrub summary: idle+waiting paths [/] 2023-07-21T22:17:58.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:58 smithi134 ceph-mon[97873]: mgrmap e24: x(active, since 91s), standbys: z, y 2023-07-21T22:17:59.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 25665 4.31 MB/sec warmup 36 sec latency 861.110 ms 2023-07-21T22:17:59.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:17:59 smithi093 ceph-mon[96579]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.1 MiB/s wr, 128 op/s 2023-07-21T22:17:59.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:17:59 smithi194 ceph-mon[98288]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.1 MiB/s wr, 128 op/s 2023-07-21T22:17:59.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:17:59 smithi134 ceph-mon[97873]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.1 MiB/s wr, 128 op/s 2023-07-21T22:18:00.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 26363 4.32 MB/sec warmup 37 sec latency 1531.643 ms 2023-07-21T22:18:00.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:18:00.094+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:00.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:00 smithi093 ceph-mon[96579]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:00.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:00 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:18:00.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:00 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:00.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:00 smithi194 ceph-mon[98288]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:00.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:00 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:18:00.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:00 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:00.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:00 smithi134 ceph-mon[97873]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:00.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:00 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] up:active 2023-07-21T22:18:00.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:00 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 -- 172.21.15.93:0/4029948357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd4103430 msgr2=0x7f4fd410b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/4029948357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd4103430 0x7f4fd410b000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00099c0 tx=0x7f4fd0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 -- 172.21.15.93:0/4029948357 shutdown_connections 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/4029948357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd4103430 0x7f4fd410b000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/4029948357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd40fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/4029948357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd4102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 -- 172.21.15.93:0/4029948357 >> 172.21.15.93:0/4029948357 conn(0x7f4fd4075f90 msgr2=0x7f4fd4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 -- 172.21.15.93:0/4029948357 shutdown_connections 2023-07-21T22:18:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.684+0000 7f4fdc02f700 1 -- 172.21.15.93:0/4029948357 wait complete. 2023-07-21T22:18:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 Processor -- start 2023-07-21T22:18:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 -- start start 2023-07-21T22:18:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 0x7f4fd41959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4196050 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41961c0 con 0x7f4fd4103430 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fdc02f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4196380 con 0x7f4fd40fed90 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45730/0 (socket says 172.21.15.93:45730) 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3549932037 learned_addr learned my addr 172.21.15.93:0/3549932037 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd95ca700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 0x7f4fd41959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3549932037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 msgr2=0x7f4fd419c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd419c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3549932037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 msgr2=0x7f4fd41959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 0x7f4fd41959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3549932037 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd00095f0 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.685+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 0x7f4fd41959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009fb0 tx=0x7f4fcc0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fcaffd700 1 -- 172.21.15.93:0/3549932037 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc018070 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fcaffd700 1 -- 172.21.15.93:0/3549932037 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc010040 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd4196660 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fcaffd700 1 -- 172.21.15.93:0/3549932037 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc005020 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.686+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd40fff00 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.688+0000 7f4fcaffd700 1 -- 172.21.15.93:0/3549932037 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4fcc004ad0 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.689+0000 7f4fcaffd700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 0x7f4fc00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.689+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 0x7f4fc00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.689+0000 7f4fcaffd700 1 -- 172.21.15.93:0/3549932037 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4fcc0cca70 con 0x7f4fd40fe3f0 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.689+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 0x7f4fc00a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40fdaa0 tx=0x7f4fc400f820 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 Processor -- start 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 -- start start 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e75f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 0x55f497e764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497d7bc50 con 0x55f497cd26b0 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497dce9e0 con 0x55f497e7c660 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497dc7e70 con 0x55f497e7c210 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45738/0 (socket says 172.21.15.93:45738) 2023-07-21T22:18:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3815000880 learned_addr learned my addr 172.21.15.93:0/3815000880 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e75f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 0x55f497e764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3815000880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 msgr2=0x55f497e75f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.691+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e75f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3815000880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 msgr2=0x55f497e764a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 0x55f497e764a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3815000880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f497bf5d80 con 0x55f497cd26b0 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 0x55f497e764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0a1a80 tx=0x7f4fcc0232d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6cd55dfca52808 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fbffff700 1 -- 172.21.15.93:0/3815000880 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc09db90 con 0x55f497cd26b0 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fbffff700 1 -- 172.21.15.93:0/3815000880 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc0a1fb0 con 0x55f497cd26b0 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.692+0000 7f4fbffff700 1 -- 172.21.15.93:0/3815000880 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc099290 con 0x55f497cd26b0 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3815000880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 msgr2=0x55f497e7bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0a1a80 tx=0x7f4fcc0232d0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3815000880 shutdown_connections 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e7c660 0x55f497e764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e7bcd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3815000880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e75f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3815000880 >> 172.21.15.93:0/3815000880 conn(0x55f497e6bae0 msgr2=0x55f497e710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3815000880 shutdown_connections 2023-07-21T22:18:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3815000880 wait complete. 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 -- start start 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.693+0000 7f4fe81b4b80 1 Processor -- start 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e7f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 0x55f497e7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497d67a60 con 0x55f497e7c660 2023-07-21T22:18:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497d67760 con 0x55f497cd26b0 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497c4b8c0 con 0x55f497e7c210 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fe81b4b80 10 client.0 _ll_register_callbacks cb 0x55f497d65700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:56906/0 (socket says 172.21.15.93:56906) 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3936818224 learned_addr learned my addr 172.21.15.93:0/3936818224 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e7f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 0x55f497e7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 msgr2=0x55f497e7f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e7f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3936818224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 msgr2=0x55f497e7ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 0x55f497e7ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/3936818224 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd410b840 con 0x55f497cd26b0 2023-07-21T22:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.694+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 0x55f497e7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0965d0 tx=0x7f4fcc099260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f999a9b1c845375 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.751 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0b2380 con 0x55f497cd26b0 2023-07-21T22:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f497d21490 con 0x55f497cd26b0 2023-07-21T22:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4fcc0b9050 con 0x55f497cd26b0 2023-07-21T22:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fcc09db90 con 0x55f497cd26b0 2023-07-21T22:18:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 10 client.25661 fetch_fsmap learned FSMap version 31 2023-07-21T22:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4fd40fff00 con 0x55f497cd26b0 2023-07-21T22:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f497d21490 con 0x55f497cd26b0 2023-07-21T22:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4fcc0143f0 con 0x55f497cd26b0 2023-07-21T22:18:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 10 client.25661 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 20 client.25661 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e8f760 0x55f497e91b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 4 client.25661 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f497dc0e50 con 0x55f497e8f760 2023-07-21T22:18:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.695+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e8f760 0x55f497e91b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.696+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e8f760 0x55f497e91b70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520eca2549b8fa1c server_cookie=9e3a25627df2a3d1 in_seq=0 out_seq=0 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.696+0000 7f4fbd7fa700 10 client.25661 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.696+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/3936818224 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f497dc0e50 con 0x55f497e8f760 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.696+0000 7f4fbd7fa700 10 client.25661 handle_command_reply: tid=0 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.696+0000 7f4fe81b4b80 1 client.25661 shutdown 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 msgr2=0x55f497e825c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.755 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0965d0 tx=0x7f4fcc099260 comp rx=0 tx=0).stop 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 shutdown_connections 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e8f760 0x55f497e91b70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e7c660 0x55f497e7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fbd7fa700 0 client.25661 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e825c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/3936818224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e7f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.756 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 >> 172.21.15.93:0/3936818224 conn(0x55f497e6bae0 msgr2=0x55f497e703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 shutdown_connections 2023-07-21T22:18:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/3936818224 wait complete. 2023-07-21T22:18:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.697+0000 7f4fe81b4b80 20 client.25661 trim_cache size 0 max 16384 2023-07-21T22:18:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 Processor -- start 2023-07-21T22:18:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 -- start start 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e7b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e82160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497e71f40 con 0x55f497e6bae0 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497e80e20 con 0x55f497cd26b0 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497e76f20 con 0x55f497e7c210 2023-07-21T22:18:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45752/0 (socket says 172.21.15.93:45752) 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.712+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2149965161 learned_addr learned my addr 172.21.15.93:0/2149965161 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e82160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e7b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2149965161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 msgr2=0x55f497e82160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e82160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2149965161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 msgr2=0x55f497e7b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e7b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2149965161 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f497bdec00 con 0x55f497e6bae0 2023-07-21T22:18:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004ff0 tx=0x7f4fcc099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35877bae4d74c626 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fc8ff9700 1 -- 172.21.15.93:0/2149965161 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0b2380 con 0x55f497e6bae0 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fc8ff9700 1 -- 172.21.15.93:0/2149965161 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc09db90 con 0x55f497e6bae0 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.713+0000 7f4fc8ff9700 1 -- 172.21.15.93:0/2149965161 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004d80 con 0x55f497e6bae0 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2149965161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 msgr2=0x55f497e86b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc004ff0 tx=0x7f4fcc099ce0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2149965161 shutdown_connections 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497e82160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497e6bae0 0x55f497e86b80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2149965161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497cd26b0 0x55f497e7b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2149965161 >> 172.21.15.93:0/2149965161 conn(0x55f497e76520 msgr2=0x55f497e7a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2149965161 shutdown_connections 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2149965161 wait complete. 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 -- start start 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.714+0000 7f4fe81b4b80 1 Processor -- start 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 0x55f497e838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497edc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497edcae0 con 0x55f497cd26b0 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497edcc50 con 0x55f497e6bae0 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497edce10 con 0x55f497e7c210 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fe81b4b80 10 client.0 _ll_register_callbacks cb 0x55f497d65700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45758/0 (socket says 172.21.15.93:45758) 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2902940808 learned_addr learned my addr 172.21.15.93:0/2902940808 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 0x55f497e838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fda5cc700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497edc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 msgr2=0x55f497edc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497edc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2902940808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 msgr2=0x55f497e838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 0x55f497e838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.715+0000 7f4fd9dcb700 1 -- 172.21.15.93:0/2902940808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd40fff00 con 0x55f497cd26b0 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 0x55f497e838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fd9dcb700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0147b0 tx=0x7f4fcc099260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1a334751d2d5dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004d80 con 0x55f497cd26b0 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4fcc0b9050 con 0x55f497cd26b0 2023-07-21T22:18:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fcc0a1fb0 con 0x55f497cd26b0 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fcc0232d0 con 0x55f497cd26b0 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 10 client.16110 fetch_fsmap learned FSMap version 31 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f497c1c1f0 con 0x55f497cd26b0 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fcc0a1fb0 con 0x55f497cd26b0 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 31) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4fcc078a30 con 0x55f497cd26b0 2023-07-21T22:18:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:18:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 20 client.16110 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:00.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e7c660 0x55f497ee6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 4 client.16110 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:18:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.716+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f497d82ff0 con 0x55f497e7c660 2023-07-21T22:18:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.717+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e7c660 0x55f497ee6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.717+0000 7f4fbd7fa700 10 client.16110 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.717+0000 7f4fd95ca700 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e7c660 0x55f497ee6130 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11387f08e356a6eb server_cookie=63e2e44bfa6a90ac in_seq=0 out_seq=0 2023-07-21T22:18:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.745+0000 7f4fbd7fa700 1 -- 172.21.15.93:0/2902940808 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f497d82ff0 con 0x55f497e7c660 2023-07-21T22:18:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.745+0000 7f4fbd7fa700 10 client.16110 handle_command_reply: tid=0 2023-07-21T22:18:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.745+0000 7f4fe81b4b80 1 client.16110 shutdown 2023-07-21T22:18:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.745+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 msgr2=0x55f497e83370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.745+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0147b0 tx=0x7f4fcc099260 comp rx=0 tx=0).stop 2023-07-21T22:18:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 shutdown_connections 2023-07-21T22:18:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f497e7c660 0x55f497ee6130 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f497e7c210 0x55f497edc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f497cd26b0 0x55f497e83370 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fbd7fa700 0 client.16110 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 --2- 172.21.15.93:0/2902940808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f497e6bae0 0x55f497e838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 >> 172.21.15.93:0/2902940808 conn(0x55f497e76520 msgr2=0x55f497e7b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 shutdown_connections 2023-07-21T22:18:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 1 -- 172.21.15.93:0/2902940808 wait complete. 2023-07-21T22:18:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fe81b4b80 20 client.16110 trim_cache size 0 max 16384 2023-07-21T22:18:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:18:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:18:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:18:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:18:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 msgr2=0x7f4fc00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 0x7f4fc00a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40fdaa0 tx=0x7f4fc400f820 comp rx=0 tx=0).stop 2023-07-21T22:18:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 msgr2=0x7f4fd419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009fb0 tx=0x7f4fcc0076c0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 shutdown_connections 2023-07-21T22:18:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4103430 0x7f4fd41959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4fc00a1340 0x7f4fc00a37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fd40fe3f0 0x7f4fd419bb80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 --2- 172.21.15.93:0/3549932037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fd40fed90 0x7f4fd419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.746+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 >> 172.21.15.93:0/3549932037 conn(0x7f4fd4075f90 msgr2=0x7f4fd40fbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.747+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 shutdown_connections 2023-07-21T22:18:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:00.747+0000 7f4fdc02f700 1 -- 172.21.15.93:0/3549932037 wait complete. 2023-07-21T22:18:01.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 28381 4.53 MB/sec warmup 38 sec latency 23.320 ms 2023-07-21T22:18:01.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:01 smithi194 ceph-mon[98288]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 233 KiB/s wr, 70 op/s 2023-07-21T22:18:01.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:01 smithi134 ceph-mon[97873]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 233 KiB/s wr, 70 op/s 2023-07-21T22:18:01.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:01 smithi093 ceph-mon[96579]: pgmap v50: 97 pgs: 97 active+clean; 26 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 233 KiB/s wr, 70 op/s 2023-07-21T22:18:02.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 29139 4.47 MB/sec warmup 39 sec latency 272.436 ms 2023-07-21T22:18:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:02 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:18:02.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:02 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:02.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:02 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:18:02.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:02 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:02.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:02 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] up:active 2023-07-21T22:18:02.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:02 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:18:03.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 29139 4.36 MB/sec warmup 40 sec latency 1272.521 ms 2023-07-21T22:18:03.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:03 smithi194 ceph-mon[98288]: pgmap v51: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 666 KiB/s wr, 178 op/s 2023-07-21T22:18:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:03 smithi134 ceph-mon[97873]: pgmap v51: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 666 KiB/s wr, 178 op/s 2023-07-21T22:18:03.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:03 smithi093 ceph-mon[96579]: pgmap v51: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 666 KiB/s wr, 178 op/s 2023-07-21T22:18:04.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 29139 4.25 MB/sec warmup 41 sec latency 2272.622 ms 2023-07-21T22:18:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:04 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:18:04.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:04 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:18:04.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:04 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:18:04.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:04 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:18:04.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:04 smithi194 ceph-mon[98288]: scrub summary: idle 2023-07-21T22:18:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:04 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:18:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:04 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:18:04.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:04 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:18:04.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:04 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:18:04.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:04 smithi134 ceph-mon[97873]: scrub summary: idle 2023-07-21T22:18:04.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:04 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:18:04.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:04 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:18:04.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:04 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:18:04.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:04 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:18:04.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:04 smithi093 ceph-mon[96579]: scrub summary: idle 2023-07-21T22:18:05.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 29674 4.19 MB/sec warmup 42 sec latency 2942.760 ms 2023-07-21T22:18:05.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:05 smithi194 ceph-mon[98288]: pgmap v52: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:05 smithi134 ceph-mon[97873]: pgmap v52: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:05.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:05 smithi093 ceph-mon[96579]: pgmap v52: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:06.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32218 4.48 MB/sec warmup 43 sec latency 9.932 ms 2023-07-21T22:18:07.308 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32687 4.39 MB/sec warmup 44 sec latency 522.572 ms 2023-07-21T22:18:07.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:18:07.037+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:07.441 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:07 smithi093 ceph-mon[96579]: pgmap v53: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:07.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:07 smithi093 ceph-mon[96579]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:07.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:07 smithi194 ceph-mon[98288]: pgmap v53: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:07.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:07 smithi194 ceph-mon[98288]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:07.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:07 smithi134 ceph-mon[97873]: pgmap v53: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 573 KiB/s wr, 150 op/s 2023-07-21T22:18:07.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:07 smithi134 ceph-mon[97873]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:18:08.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32687 4.30 MB/sec warmup 45 sec latency 1522.671 ms 2023-07-21T22:18:09.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32687 4.20 MB/sec warmup 46 sec latency 2522.761 ms 2023-07-21T22:18:09.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:09 smithi194 ceph-mon[98288]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 815 KiB/s wr, 194 op/s 2023-07-21T22:18:09.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:09 smithi134 ceph-mon[97873]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 815 KiB/s wr, 194 op/s 2023-07-21T22:18:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:09 smithi093 ceph-mon[96579]: pgmap v54: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 815 KiB/s wr, 194 op/s 2023-07-21T22:18:10.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32755 4.12 MB/sec warmup 47 sec latency 3191.124 ms 2023-07-21T22:18:11.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32755 4.03 MB/sec warmup 48 sec latency 1285.375 ms 2023-07-21T22:18:11.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:11 smithi194 ceph-mon[98288]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 675 KiB/s wr, 152 op/s 2023-07-21T22:18:11.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:11 smithi134 ceph-mon[97873]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 675 KiB/s wr, 152 op/s 2023-07-21T22:18:11.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:11 smithi093 ceph-mon[96579]: pgmap v55: 97 pgs: 97 active+clean; 32 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 675 KiB/s wr, 152 op/s 2023-07-21T22:18:12.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32755 3.95 MB/sec warmup 49 sec latency 2285.502 ms 2023-07-21T22:18:13.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32755 3.87 MB/sec warmup 50 sec latency 3285.556 ms 2023-07-21T22:18:13.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:13 smithi194 ceph-mon[98288]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 704 KiB/s wr, 164 op/s 2023-07-21T22:18:13.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:13 smithi134 ceph-mon[97873]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 704 KiB/s wr, 164 op/s 2023-07-21T22:18:13.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:13 smithi093 ceph-mon[96579]: pgmap v56: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 704 KiB/s wr, 164 op/s 2023-07-21T22:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 -- 172.21.15.93:0/30723142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 msgr2=0x7f3a7c10b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 --2- 172.21.15.93:0/30723142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c10b7c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400b0a0 tx=0x7f3a7400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 -- 172.21.15.93:0/30723142 shutdown_connections 2023-07-21T22:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 --2- 172.21.15.93:0/30723142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c10b7c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 --2- 172.21.15.93:0/30723142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 0x7f3a7c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 --2- 172.21.15.93:0/30723142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.862+0000 7f3a84207700 1 -- 172.21.15.93:0/30723142 >> 172.21.15.93:0/30723142 conn(0x7f3a7c075270 msgr2=0x7f3a7c075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- 172.21.15.93:0/30723142 shutdown_connections 2023-07-21T22:18:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- 172.21.15.93:0/30723142 wait complete. 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 Processor -- start 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- start start 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 0x7f3a7c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a2180 con 0x7f3a7c1093e0 2023-07-21T22:18:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a22f0 con 0x7f3a7c0ffe30 2023-07-21T22:18:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a84207700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1a24b0 con 0x7f3a7c100740 2023-07-21T22:18:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.863+0000 7f3a827a4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45776/0 (socket says 172.21.15.93:45776) 2023-07-21T22:18:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 -- 172.21.15.93:0/2262804680 learned_addr learned my addr 172.21.15.93:0/2262804680 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 0x7f3a7c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a817a2700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 -- 172.21.15.93:0/2262804680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 msgr2=0x7f3a7c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 -- 172.21.15.93:0/2262804680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 msgr2=0x7f3a7c196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 0x7f3a7c196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 -- 172.21.15.93:0/2262804680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009d00 con 0x7f3a7c1093e0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a817a2700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a827a4700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3a740079b0 tx=0x7f3a740079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a72ffd700 1 -- 172.21.15.93:0/2262804680 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e040 con 0x7f3a7c1093e0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1a2730 con 0x7f3a7c1093e0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a72ffd700 1 -- 172.21.15.93:0/2262804680 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a74006080 con 0x7f3a7c1093e0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a72ffd700 1 -- 172.21.15.93:0/2262804680 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7401b950 con 0x7f3a7c1093e0 2023-07-21T22:18:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.864+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1a2bb0 con 0x7f3a7c1093e0 2023-07-21T22:18:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.866+0000 7f3a72ffd700 1 -- 172.21.15.93:0/2262804680 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3a7401bab0 con 0x7f3a7c1093e0 2023-07-21T22:18:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.867+0000 7f3a72ffd700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 0x7f3a680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.867+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 0x7f3a680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.867+0000 7f3a72ffd700 1 -- 172.21.15.93:0/2262804680 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3a740ce980 con 0x7f3a7c1093e0 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 0x7f3a680a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005950 tx=0x7f3a7800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 Processor -- start 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 -- start start 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1498bfc50 con 0x55a1499c0210 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499129e0 con 0x55a1499c0660 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.868+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14990be70 con 0x55a1498166b0 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45782/0 (socket says 172.21.15.93:45782) 2023-07-21T22:18:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 -- 172.21.15.93:0/2988189070 learned_addr learned my addr 172.21.15.93:0/2988189070 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a827a4700 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 -- 172.21.15.93:0/2988189070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 msgr2=0x55a1499bfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 -- 172.21.15.93:0/2988189070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 msgr2=0x55a1499ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 -- 172.21.15.93:0/2988189070 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a149739d80 con 0x55a1499c0210 2023-07-21T22:18:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a817a2700 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c008ba0 tx=0x7f3a6c008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17133e8384b7c43 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a67fff700 1 -- 172.21.15.93:0/2988189070 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00ed10 con 0x55a1499c0210 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a67fff700 1 -- 172.21.15.93:0/2988189070 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c00ee70 con 0x55a1499c0210 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.869+0000 7f3a67fff700 1 -- 172.21.15.93:0/2988189070 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c010430 con 0x55a1499c0210 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- 172.21.15.93:0/2988189070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 msgr2=0x55a1499b9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c008ba0 tx=0x7f3a6c008eb0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- 172.21.15.93:0/2988189070 shutdown_connections 2023-07-21T22:18:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/2988189070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499b9f40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- 172.21.15.93:0/2988189070 >> 172.21.15.93:0/2988189070 conn(0x55a1499afae0 msgr2=0x55a1499b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- 172.21.15.93:0/2988189070 shutdown_connections 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- 172.21.15.93:0/2988189070 wait complete. 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 -- start start 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.870+0000 7f3a9038cb80 1 Processor -- start 2023-07-21T22:18:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1498166b0 0x55a1499c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 0x55a1499c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1498aba60 con 0x55a1499c0210 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1498ab760 con 0x55a1498166b0 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a14978f8c0 con 0x55a1499c0660 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a9038cb80 10 client.0 _ll_register_callbacks cb 0x55a1498a9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:13.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45788/0 (socket says 172.21.15.93:45788) 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 -- 172.21.15.93:0/4138934448 learned_addr learned my addr 172.21.15.93:0/4138934448 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1498166b0 0x55a1499c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a827a4700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 0x55a1499c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 -- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 msgr2=0x55a1499c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 0x55a1499c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 -- 172.21.15.93:0/4138934448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1498166b0 msgr2=0x55a1499c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1498166b0 0x55a1499c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 -- 172.21.15.93:0/4138934448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c008800 con 0x55a1499c0210 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a827a4700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 0x55a1499c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a817a2700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3509cc52d86442b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.871+0000 7f3a657fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c01f070 con 0x55a1499c0210 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3a6c004a60 con 0x55a1499c0210 2023-07-21T22:18:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a149865490 con 0x55a1499c0210 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a6c004c20 con 0x55a1499c0210 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 10 client.16122 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a149722c00 con 0x55a1499c0210 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c030020 con 0x55a1499c0210 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f3a6c023410 con 0x55a1499c0210 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 10 client.16122 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 20 client.16122 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a1499d39d0 0x55a1499d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 4 client.16122 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a149904e50 con 0x55a1499d39d0 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.872+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a1499d39d0 0x55a1499d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.873+0000 7f3a657fa700 10 client.16122 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.873+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a1499d39d0 0x55a1499d5de0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d272ab6f82e5d74 server_cookie=eca9e2aed81ed87b in_seq=0 out_seq=0 2023-07-21T22:18:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a657fa700 1 -- 172.21.15.93:0/4138934448 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a149904e50 con 0x55a1499d39d0 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a657fa700 10 client.16122 handle_command_reply: tid=0 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 client.16122 shutdown 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 msgr2=0x55a1499c37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c000c00 tx=0x7f3a6c0056a0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 shutdown_connections 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a1499d39d0 0x55a1499d5de0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1499c0660 0x55a1499c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a657fa700 0 client.16122 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1498166b0 0x55a1499c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/4138934448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499c37a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 >> 172.21.15.93:0/4138934448 conn(0x55a1499afae0 msgr2=0x55a1499b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 shutdown_connections 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 1 -- 172.21.15.93:0/4138934448 wait complete. 2023-07-21T22:18:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.874+0000 7f3a9038cb80 20 client.16122 trim_cache size 0 max 16384 2023-07-21T22:18:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 Processor -- start 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 -- start start 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499bb160 con 0x55a1499c0210 2023-07-21T22:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499acb20 con 0x55a1499c0660 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499b1c00 con 0x55a1498166b0 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45796/0 (socket says 172.21.15.93:45796) 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 -- 172.21.15.93:0/3990767717 learned_addr learned my addr 172.21.15.93:0/3990767717 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a827a4700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 -- 172.21.15.93:0/3990767717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 msgr2=0x55a1499c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 -- 172.21.15.93:0/3990767717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 msgr2=0x55a1499c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 -- 172.21.15.93:0/3990767717 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1497601f0 con 0x55a1499c0210 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a827a4700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.889+0000 7f3a817a2700 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c008af0 tx=0x7f3a6c00e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d2c458f79cdc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a70ff9700 1 -- 172.21.15.93:0/3990767717 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c01f040 con 0x55a1499c0210 2023-07-21T22:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a70ff9700 1 -- 172.21.15.93:0/3990767717 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c030030 con 0x55a1499c0210 2023-07-21T22:18:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a70ff9700 1 -- 172.21.15.93:0/3990767717 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c016710 con 0x55a1499c0210 2023-07-21T22:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- 172.21.15.93:0/3990767717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 msgr2=0x55a1499caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c008af0 tx=0x7f3a6c00e760 comp rx=0 tx=0).stop 2023-07-21T22:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- 172.21.15.93:0/3990767717 shutdown_connections 2023-07-21T22:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0660 0x55a1499c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/3990767717 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0210 0x55a1499caba0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- 172.21.15.93:0/3990767717 >> 172.21.15.93:0/3990767717 conn(0x55a1499afae0 msgr2=0x55a1499c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- 172.21.15.93:0/3990767717 shutdown_connections 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- 172.21.15.93:0/3990767717 wait complete. 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.890+0000 7f3a9038cb80 1 -- start start 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 Processor -- start 2023-07-21T22:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0210 0x55a1499bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499bd2d0 con 0x55a1499c0660 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499bd440 con 0x55a1499c0210 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1499bc1b0 con 0x55a1498166b0 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a9038cb80 10 client.0 _ll_register_callbacks cb 0x7f3a74009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45804/0 (socket says 172.21.15.93:45804) 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 -- 172.21.15.93:0/735526507 learned_addr learned my addr 172.21.15.93:0/735526507 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a817a2700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0210 0x55a1499bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 -- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 msgr2=0x55a1499bccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 -- 172.21.15.93:0/735526507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0210 msgr2=0x55a1499bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0210 0x55a1499bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 -- 172.21.15.93:0/735526507 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a149722c00 con 0x55a1499c0660 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a827a4700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a74098000 tx=0x7f3a740099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3e053dc69d5114 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.891+0000 7f3a657fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740b40d0 con 0x55a1499c0660 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3a740b4230 con 0x55a1499c0660 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a6c00ee70 con 0x55a1499c0660 2023-07-21T22:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a74011c60 con 0x55a1499c0660 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 10 client.16128 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a149738250 con 0x55a1499c0660 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6c00ee70 con 0x55a1499c0660 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f3a7409aff0 con 0x55a1499c0660 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 10 client.16128 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 20 client.16128 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a149a2cb90 0x55a149a2efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 4 client.16128 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1498c6ff0 con 0x55a149a2cb90 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.892+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a149a2cb90 0x55a149a2efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.893+0000 7f3a657fa700 10 client.16128 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.893+0000 7f3a81fa3700 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a149a2cb90 0x55a149a2efa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361a34f10cab5b75 server_cookie=b3f6d3e853cc007f in_seq=0 out_seq=0 2023-07-21T22:18:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a657fa700 1 -- 172.21.15.93:0/735526507 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1498c6ff0 con 0x55a149a2cb90 2023-07-21T22:18:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a657fa700 10 client.16128 handle_command_reply: tid=0 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 client.16128 shutdown 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 msgr2=0x55a1499bbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a74098000 tx=0x7f3a740099e0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 shutdown_connections 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a149a2cb90 0x55a149a2efa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a1499c0660 0x55a1499bbc70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a657fa700 0 client.16128 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a1498166b0 0x55a1499bccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 --2- 172.21.15.93:0/735526507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a1499c0210 0x55a1499bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 >> 172.21.15.93:0/735526507 conn(0x55a1499afae0 msgr2=0x55a1499c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 shutdown_connections 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 1 -- 172.21.15.93:0/735526507 wait complete. 2023-07-21T22:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.923+0000 7f3a9038cb80 20 client.16128 trim_cache size 0 max 16384 2023-07-21T22:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 msgr2=0x7f3a680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 0x7f3a680a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a78005950 tx=0x7f3a7800b410 comp rx=0 tx=0).stop 2023-07-21T22:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 msgr2=0x7f3a7c1a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3a740079b0 tx=0x7f3a740079e0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 shutdown_connections 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3a680a12c0 0x7f3a680a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a7c1093e0 0x7f3a7c1a1c10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a7c0ffe30 0x7f3a7c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 --2- 172.21.15.93:0/2262804680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a7c100740 0x7f3a7c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 >> 172.21.15.93:0/2262804680 conn(0x7f3a7c075270 msgr2=0x7f3a7c0fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 shutdown_connections 2023-07-21T22:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:13.924+0000 7f3a84207700 1 -- 172.21.15.93:0/2262804680 wait complete. 2023-07-21T22:18:14.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 32755 3.79 MB/sec warmup 51 sec latency 4285.620 ms 2023-07-21T22:18:15.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 33483 3.81 MB/sec warmup 52 sec latency 4956.460 ms 2023-07-21T22:18:15.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:15 smithi194 ceph-mon[98288]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:15.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:15 smithi134 ceph-mon[97873]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:15.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:15 smithi093 ceph-mon[96579]: pgmap v57: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:16.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 35871 4.00 MB/sec warmup 53 sec latency 11.172 ms 2023-07-21T22:18:17.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 36174 3.93 MB/sec warmup 54 sec latency 665.838 ms 2023-07-21T22:18:17.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:17 smithi194 ceph-mon[98288]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:17.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:17 smithi134 ceph-mon[97873]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:17 smithi093 ceph-mon[96579]: pgmap v58: 97 pgs: 97 active+clean; 32 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 272 KiB/s wr, 56 op/s 2023-07-21T22:18:18.309 INFO:tasks.workunit.client.0.smithi093.stdout: 1 36174 3.86 MB/sec warmup 55 sec latency 1665.943 ms 2023-07-21T22:18:19.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 36174 3.79 MB/sec warmup 56 sec latency 2666.034 ms 2023-07-21T22:18:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:19 smithi194 ceph-mon[98288]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 528 KiB/s wr, 108 op/s 2023-07-21T22:18:19.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:19 smithi134 ceph-mon[97873]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 528 KiB/s wr, 108 op/s 2023-07-21T22:18:19.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:19 smithi093 ceph-mon[96579]: pgmap v59: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 528 KiB/s wr, 108 op/s 2023-07-21T22:18:20.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 36715 3.76 MB/sec warmup 57 sec latency 3337.173 ms 2023-07-21T22:18:21.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 39300 3.98 MB/sec warmup 58 sec latency 18.373 ms 2023-07-21T22:18:21.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:21 smithi194 ceph-mon[98288]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 285 KiB/s wr, 64 op/s 2023-07-21T22:18:21.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:21 smithi134 ceph-mon[97873]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 285 KiB/s wr, 64 op/s 2023-07-21T22:18:21.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:21 smithi093 ceph-mon[96579]: pgmap v60: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 285 KiB/s wr, 64 op/s 2023-07-21T22:18:22.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 39722 3.92 MB/sec warmup 59 sec latency 527.823 ms 2023-07-21T22:18:23.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 39722 3.86 MB/sec warmup 60 sec latency 1527.894 ms 2023-07-21T22:18:23.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:23 smithi194 ceph-mon[98288]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 780 KiB/s wr, 116 op/s 2023-07-21T22:18:23.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:23 smithi134 ceph-mon[97873]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 780 KiB/s wr, 116 op/s 2023-07-21T22:18:23.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:23 smithi093 ceph-mon[96579]: pgmap v61: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 780 KiB/s wr, 116 op/s 2023-07-21T22:18:24.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 39722 3.79 MB/sec warmup 61 sec latency 2527.937 ms 2023-07-21T22:18:25.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 40211 3.76 MB/sec warmup 62 sec latency 3198.752 ms 2023-07-21T22:18:25.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:25 smithi194 ceph-mon[98288]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:25.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:25 smithi134 ceph-mon[97873]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:25 smithi093 ceph-mon[96579]: pgmap v62: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:26.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 42834 3.97 MB/sec warmup 63 sec latency 13.048 ms 2023-07-21T22:18:26.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:18:27.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 -- 172.21.15.93:0/3470805968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900feb90 msgr2=0x7f029010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 --2- 172.21.15.93:0/3470805968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900feb90 0x7f029010c9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f028800b0a0 tx=0x7f028800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 -- 172.21.15.93:0/3470805968 shutdown_connections 2023-07-21T22:18:27.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 --2- 172.21.15.93:0/3470805968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900feb90 0x7f029010c9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 --2- 172.21.15.93:0/3470805968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0290105c00 0x7f0290105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 --2- 172.21.15.93:0/3470805968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02900fe200 0x7f02900fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 -- 172.21.15.93:0/3470805968 >> 172.21.15.93:0/3470805968 conn(0x7f0290076200 msgr2=0x7f0290076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 -- 172.21.15.93:0/3470805968 shutdown_connections 2023-07-21T22:18:27.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.958+0000 7f0297d57700 1 -- 172.21.15.93:0/3470805968 wait complete. 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 Processor -- start 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 -- start start 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 0x7f0290196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0290105c00 0x7f029019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019adb0 con 0x7f02900fe200 2023-07-21T22:18:27.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019af20 con 0x7f0290105c00 2023-07-21T22:18:27.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0297d57700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019b0e0 con 0x7f02900feb90 2023-07-21T22:18:27.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45814/0 (socket says 172.21.15.93:45814) 2023-07-21T22:18:27.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 -- 172.21.15.93:0/3364522974 learned_addr learned my addr 172.21.15.93:0/3364522974 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f02962f4700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0290105c00 0x7f029019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f02952f2700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 0x7f0290196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 -- 172.21.15.93:0/3364522974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 msgr2=0x7f0290196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 0x7f0290196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 -- 172.21.15.93:0/3364522974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0290105c00 msgr2=0x7f029019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0290105c00 0x7f029019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 -- 172.21.15.93:0/3364522974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288009d00 con 0x7f02900fe200 2023-07-21T22:18:27.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f02952f2700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 0x7f0290196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.959+0000 7f0295af3700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.960+0000 7f0286ffd700 1 -- 172.21.15.93:0/3364522974 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c009940 con 0x7f02900fe200 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.960+0000 7f0286ffd700 1 -- 172.21.15.93:0/3364522974 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c010460 con 0x7f02900fe200 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.960+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029019b3c0 con 0x7f02900fe200 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.960+0000 7f0286ffd700 1 -- 172.21.15.93:0/3364522974 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00f620 con 0x7f02900fe200 2023-07-21T22:18:27.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.960+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901a2c00 con 0x7f02900fe200 2023-07-21T22:18:27.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.962+0000 7f0286ffd700 1 -- 172.21.15.93:0/3364522974 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f028c0108d0 con 0x7f02900fe200 2023-07-21T22:18:27.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 Processor -- start 2023-07-21T22:18:27.016 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 -- start start 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f0286ffd700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 0x7f027c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82775ac0 0x560e82770350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82770930 0x560e82770da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82675cb0 con 0x560e827751b0 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e826c8a40 con 0x560e82775ac0 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e826c1ed0 con 0x560e82770930 2023-07-21T22:18:27.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f02962f4700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 0x7f027c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.963+0000 7f0286ffd700 1 -- 172.21.15.93:0/3364522974 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f028c0ccab0 con 0x7f02900fe200 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f02962f4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82770930 0x560e82770da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45822/0 (socket says 172.21.15.93:45822) 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 -- 172.21.15.93:0/1188372825 learned_addr learned my addr 172.21.15.93:0/1188372825 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f02952f2700 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82775ac0 0x560e82770350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 -- 172.21.15.93:0/1188372825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82770930 msgr2=0x560e82770da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82770930 0x560e82770da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 -- 172.21.15.93:0/1188372825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82775ac0 msgr2=0x560e82770350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82775ac0 0x560e82770350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 -- 172.21.15.93:0/1188372825 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e824efde0 con 0x560e827751b0 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f02962f4700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 0x7f027c0a37f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0288009cd0 tx=0x7f028800bd00 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f0295af3700 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f028c096040 tx=0x7f028c0997b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bd97a9f76d317d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f027bfff700 1 -- 172.21.15.93:0/1188372825 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c019dd0 con 0x560e827751b0 2023-07-21T22:18:27.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f027bfff700 1 -- 172.21.15.93:0/1188372825 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c099b40 con 0x560e827751b0 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.964+0000 7f027bfff700 1 -- 172.21.15.93:0/1188372825 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c097fb0 con 0x560e827751b0 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1188372825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 msgr2=0x560e82775580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f028c096040 tx=0x7f028c0997b0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1188372825 shutdown_connections 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82770930 0x560e82770da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e82775580 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1188372825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82775ac0 0x560e82770350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1188372825 >> 172.21.15.93:0/1188372825 conn(0x560e82765ef0 msgr2=0x560e8276b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1188372825 shutdown_connections 2023-07-21T22:18:27.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1188372825 wait complete. 2023-07-21T22:18:27.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 -- start start 2023-07-21T22:18:27.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.965+0000 7f02a3edcb80 1 Processor -- start 2023-07-21T22:18:27.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e82779e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82661ac0 con 0x560e827751b0 2023-07-21T22:18:27.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e826617c0 con 0x560e82770930 2023-07-21T22:18:27.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82545920 con 0x560e82775ac0 2023-07-21T22:18:27.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02a3edcb80 10 client.0 _ll_register_callbacks cb 0x560e8265f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45830/0 (socket says 172.21.15.93:45830) 2023-07-21T22:18:27.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 -- 172.21.15.93:0/22093942 learned_addr learned my addr 172.21.15.93:0/22093942 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f0295af3700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e82779e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 -- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 msgr2=0x560e8277a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 -- 172.21.15.93:0/22093942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 msgr2=0x560e82779e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e82779e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.966+0000 7f02952f2700 1 -- 172.21.15.93:0/22093942 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901a2c00 con 0x560e827751b0 2023-07-21T22:18:27.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02952f2700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f02800136e0 tx=0x7f02800139f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3066a12e822617e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280004030 con 0x560e827751b0 2023-07-21T22:18:27.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0280004190 con 0x560e827751b0 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e8261b4f0 con 0x560e827751b0 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0280004350 con 0x560e827751b0 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 10 client.16140 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560e824d8c60 con 0x560e827751b0 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e8261b4f0 con 0x560e827751b0 2023-07-21T22:18:27.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f028000e7a0 con 0x560e827751b0 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 20 client.16140 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e8278a590 0x560e8278c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 4 client.16140 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:27.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.967+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e826baeb0 con 0x560e8278a590 2023-07-21T22:18:27.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f0295af3700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e8278a590 0x560e8278c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f0295af3700 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e8278a590 0x560e8278c9a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e7a47694209209 server_cookie=98a7664dc9bfe4ad in_seq=0 out_seq=0 2023-07-21T22:18:27.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f02797fa700 10 client.16140 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f02797fa700 1 -- 172.21.15.93:0/22093942 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560e826baeb0 con 0x560e8278a590 2023-07-21T22:18:27.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f02797fa700 10 client.16140 handle_command_reply: tid=0 2023-07-21T22:18:27.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.968+0000 7f02a3edcb80 1 client.16140 shutdown 2023-07-21T22:18:27.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 msgr2=0x560e8277a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f02800136e0 tx=0x7f02800139f0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 shutdown_connections 2023-07-21T22:18:27.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e8278a590 0x560e8278c9a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e82779e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02797fa700 0 client.16140 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/22093942 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e827751b0 0x560e8277a3c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 >> 172.21.15.93:0/22093942 conn(0x560e82765ef0 msgr2=0x560e8276b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 shutdown_connections 2023-07-21T22:18:27.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 1 -- 172.21.15.93:0/22093942 wait complete. 2023-07-21T22:18:27.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.969+0000 7f02a3edcb80 20 client.16140 trim_cache size 0 max 16384 2023-07-21T22:18:27.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 Processor -- start 2023-07-21T22:18:27.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 -- start start 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e827814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 0x560e8277a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82787b10 con 0x560e82775ac0 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e827819f0 con 0x560e82770930 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82779430 con 0x560e827751b0 2023-07-21T22:18:27.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45840/0 (socket says 172.21.15.93:45840) 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 -- 172.21.15.93:0/1954462368 learned_addr learned my addr 172.21.15.93:0/1954462368 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f0295af3700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e827814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02952f2700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 0x560e8277a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 -- 172.21.15.93:0/1954462368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 msgr2=0x560e8277a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 0x560e8277a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 -- 172.21.15.93:0/1954462368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 msgr2=0x560e827814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e827814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 -- 172.21.15.93:0/1954462368 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e82516250 con 0x560e82775ac0 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02952f2700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 0x560e8277a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f0295af3700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e827814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.987+0000 7f02962f4700 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0288009500 tx=0x7f0288022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2e23ab6d98d7a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f0284ff9700 1 -- 172.21.15.93:0/1954462368 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800e040 con 0x560e82775ac0 2023-07-21T22:18:27.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f0284ff9700 1 -- 172.21.15.93:0/1954462368 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028802a940 con 0x560e82775ac0 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f0284ff9700 1 -- 172.21.15.93:0/1954462368 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028801f860 con 0x560e82775ac0 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1954462368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 msgr2=0x560e8276f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0288009500 tx=0x7f0288022040 comp rx=0 tx=0).stop 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1954462368 shutdown_connections 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82775ac0 0x560e8276f9b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e82770930 0x560e827814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/1954462368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e827751b0 0x560e8277a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1954462368 >> 172.21.15.93:0/1954462368 conn(0x560e82765ef0 msgr2=0x560e8276ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1954462368 shutdown_connections 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.988+0000 7f02a3edcb80 1 -- 172.21.15.93:0/1954462368 wait complete. 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 -- start start 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 Processor -- start 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 0x560e8277f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e8277ff20 con 0x560e82770930 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82780090 con 0x560e827751b0 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.989+0000 7f02a3edcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e82780250 con 0x560e82775ac0 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02a3edcb80 10 client.0 _ll_register_callbacks cb 0x560e8265f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45842/0 (socket says 172.21.15.93:45842) 2023-07-21T22:18:27.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 -- 172.21.15.93:0/2150641193 learned_addr learned my addr 172.21.15.93:0/2150641193 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02952f2700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 0x560e8277f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02962f4700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 -- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 msgr2=0x560e8277f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 -- 172.21.15.93:0/2150641193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 msgr2=0x560e8277f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 0x560e8277f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 -- 172.21.15.93:0/2150641193 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0288026070 con 0x560e82770930 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02952f2700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 0x560e8277f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f0295af3700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f028c010040 tx=0x7f028c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f08a2ca276924 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02797fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0105d0 con 0x560e82770930 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f028c09db70 con 0x560e82770930 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f028c099b40 con 0x560e82770930 2023-07-21T22:18:27.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f028c019e10 con 0x560e82770930 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02a3edcb80 10 client.16146 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560e824ee2b0 con 0x560e82770930 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.990+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c099b40 con 0x560e82770930 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f028c099150 con 0x560e82770930 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 10 client.16146 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 20 client.16146 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e827df440 0x560e827e1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 4 client.16146 mds_command: new command op to 24484 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e8267d050 con 0x560e827df440 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02952f2700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e827df440 0x560e827e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02797fa700 10 client.16146 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02952f2700 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e827df440 0x560e827e1850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d1783a7d14642c server_cookie=463f974958cd64c1 in_seq=0 out_seq=0 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02797fa700 1 -- 172.21.15.93:0/2150641193 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560e8267d050 con 0x560e827df440 2023-07-21T22:18:27.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02797fa700 10 client.16146 handle_command_reply: tid=0 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.991+0000 7f02a3edcb80 1 client.16146 shutdown 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 msgr2=0x560e82781040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f028c010040 tx=0x7f028c005910 comp rx=0 tx=0).stop 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 shutdown_connections 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e827df440 0x560e827e1850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e82775ac0 0x560e8277f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02797fa700 0 client.16146 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e82770930 0x560e82781040 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 --2- 172.21.15.93:0/2150641193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e827751b0 0x560e8277f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 >> 172.21.15.93:0/2150641193 conn(0x560e82765ef0 msgr2=0x560e8276f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 shutdown_connections 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 1 -- 172.21.15.93:0/2150641193 wait complete. 2023-07-21T22:18:27.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f02a3edcb80 20 client.16146 trim_cache size 0 max 16384 2023-07-21T22:18:27.055 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:18:27.055 INFO:teuthology.orchestra.run.smithi093.stdout: "status": "no active scrubs running", 2023-07-21T22:18:27.055 INFO:teuthology.orchestra.run.smithi093.stdout: "scrubs": {} 2023-07-21T22:18:27.055 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:18:27.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 msgr2=0x7f027c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 0x7f027c0a37f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0288009cd0 tx=0x7f028800bd00 comp rx=0 tx=0).stop 2023-07-21T22:18:27.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 msgr2=0x7f0290195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:18:27.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 shutdown_connections 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0290105c00 0x7f029019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f027c0a1340 0x7f027c0a37f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f02900fe200 0x7f0290195dd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 --2- 172.21.15.93:0/3364522974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02900feb90 0x7f0290196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.992+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 >> 172.21.15.93:0/3364522974 conn(0x7f0290076200 msgr2=0x7f02900fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.993+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 shutdown_connections 2023-07-21T22:18:27.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:26.993+0000 7f0297d57700 1 -- 172.21.15.93:0/3364522974 wait complete. 2023-07-21T22:18:27.059 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:18:27.059 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 -- 172.21.15.93:0/1250268101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd4410a080 msgr2=0x7fcd4410c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 --2- 172.21.15.93:0/1250268101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd4410a080 0x7fcd4410c4f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b0a0 tx=0x7fcd3c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 -- 172.21.15.93:0/1250268101 shutdown_connections 2023-07-21T22:18:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 --2- 172.21.15.93:0/1250268101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd4410a080 0x7fcd4410c4f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 --2- 172.21.15.93:0/1250268101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 0x7fcd44101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 --2- 172.21.15.93:0/1250268101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd44101650 0x7fcd44109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 -- 172.21.15.93:0/1250268101 >> 172.21.15.93:0/1250268101 conn(0x7fcd440fa4d0 msgr2=0x7fcd440fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 -- 172.21.15.93:0/1250268101 shutdown_connections 2023-07-21T22:18:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.039+0000 7fcd499a5700 1 -- 172.21.15.93:0/1250268101 wait complete. 2023-07-21T22:18:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 Processor -- start 2023-07-21T22:18:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 -- start start 2023-07-21T22:18:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 0x7fcd44195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd4410a080 0x7fcd441997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44199db0 con 0x7fcd44101650 2023-07-21T22:18:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44199f20 con 0x7fcd44100d40 2023-07-21T22:18:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd499a5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4419a0e0 con 0x7fcd4410a080 2023-07-21T22:18:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45858/0 (socket says 172.21.15.93:45858) 2023-07-21T22:18:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd427fc700 1 -- 172.21.15.93:0/276584078 learned_addr learned my addr 172.21.15.93:0/276584078 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd437fe700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd4410a080 0x7fcd441997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.040+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 0x7fcd44195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 -- 172.21.15.93:0/276584078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd4410a080 msgr2=0x7fcd441997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd4410a080 0x7fcd441997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 -- 172.21.15.93:0/276584078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 msgr2=0x7fcd44195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 0x7fcd44195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 -- 172.21.15.93:0/276584078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c009d00 con 0x7fcd44101650 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd427fc700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800e9a0 tx=0x7fcd3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd489a3700 1 -- 172.21.15.93:0/276584078 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38009790 con 0x7fcd44101650 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4419a3c0 con 0x7fcd44101650 2023-07-21T22:18:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd489a3700 1 -- 172.21.15.93:0/276584078 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd38010430 con 0x7fcd44101650 2023-07-21T22:18:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd489a3700 1 -- 172.21.15.93:0/276584078 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38018680 con 0x7fcd44101650 2023-07-21T22:18:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.041+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd441a2d20 con 0x7fcd44101650 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.044+0000 7fcd489a3700 1 -- 172.21.15.93:0/276584078 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcd380105a0 con 0x7fcd44101650 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.045+0000 7fcd489a3700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 0x7fcd2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.045+0000 7fcd489a3700 1 -- 172.21.15.93:0/276584078 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcd38014070 con 0x7fcd44101650 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.045+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 0x7fcd2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.045+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 0x7fcd2c0a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009c60 tx=0x7fcd34009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.045+0000 7fcd55b2ab80 1 Processor -- start 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 -- start start 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 0x55f0902af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0901aac50 con 0x55f0902af620 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0901fd9e0 con 0x55f0902afa70 2023-07-21T22:18:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0901f6e70 con 0x55f0901016b0 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 0x55f0902af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45866/0 (socket says 172.21.15.93:45866) 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 -- 172.21.15.93:0/451641286 learned_addr learned my addr 172.21.15.93:0/451641286 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 -- 172.21.15.93:0/451641286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 msgr2=0x55f0902af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 0x55f0902af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 -- 172.21.15.93:0/451641286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 msgr2=0x55f0902a98b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902a98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 -- 172.21.15.93:0/451641286 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f090024d80 con 0x55f0902af620 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 0x55f0902af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.046+0000 7fcd427fc700 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcd380a1e40 tx=0x7fcd38099900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e653d16979779b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd29ffb700 1 -- 172.21.15.93:0/451641286 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380a2370 con 0x55f0902af620 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd29ffb700 1 -- 172.21.15.93:0/451641286 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd38010880 con 0x55f0902af620 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd29ffb700 1 -- 172.21.15.93:0/451641286 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380a1ee0 con 0x55f0902af620 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/451641286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 msgr2=0x55f0902a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcd380a1e40 tx=0x7fcd38099900 comp rx=0 tx=0).stop 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/451641286 shutdown_connections 2023-07-21T22:18:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0901016b0 0x55f0902af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/451641286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902a9350 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/451641286 >> 172.21.15.93:0/451641286 conn(0x55f09029aae0 msgr2=0x55f0902a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/451641286 shutdown_connections 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.047+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/451641286 wait complete. 2023-07-21T22:18:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 -- start start 2023-07-21T22:18:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 Processor -- start 2023-07-21T22:18:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f090196a60 con 0x55f0902afa70 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f090196760 con 0x55f0901016b0 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09007a8c0 con 0x55f0902af620 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd55b2ab80 10 client.0 _ll_register_callbacks cb 0x55f090194700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd437fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd437fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45874/0 (socket says 172.21.15.93:45874) 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 -- 172.21.15.93:0/2529660469 learned_addr learned my addr 172.21.15.93:0/2529660469 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.048+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 -- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 msgr2=0x55f0902b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 -- 172.21.15.93:0/2529660469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 msgr2=0x55f0902b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 -- 172.21.15.93:0/2529660469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd34017070 con 0x55f0902afa70 2023-07-21T22:18:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd427fc700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd437fe700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009610 tx=0x7fcd3c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e822cff9ea80cc84 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00e040 con 0x55f0902afa70 2023-07-21T22:18:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcd3c003e10 con 0x55f0902afa70 2023-07-21T22:18:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f090150490 con 0x55f0902afa70 2023-07-21T22:18:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd3c006000 con 0x55f0902afa70 2023-07-21T22:18:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd55b2ab80 10 client.16158 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fcd441a2d20 con 0x55f0902afa70 2023-07-21T22:18:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.049+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f090150490 con 0x55f0902afa70 2023-07-21T22:18:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fcd3c019030 con 0x55f0902afa70 2023-07-21T22:18:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 10 client.16158 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 20 client.16158 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f0902c3090 0x55f0902c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 4 client.16158 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0901efe50 con 0x55f0902c3090 2023-07-21T22:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f0902c3090 0x55f0902c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd22ffd700 10 client.16158 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.050+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f0902c3090 0x55f0902c54a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c216b6daba26023c server_cookie=683982810e696f71 in_seq=0 out_seq=0 2023-07-21T22:18:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd22ffd700 1 -- 172.21.15.93:0/2529660469 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0901efe50 con 0x55f0902c3090 2023-07-21T22:18:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd22ffd700 10 client.16158 handle_command_reply: tid=0 2023-07-21T22:18:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 client.16158 shutdown 2023-07-21T22:18:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 msgr2=0x55f0902b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009610 tx=0x7fcd3c00bd50 comp rx=0 tx=0).stop 2023-07-21T22:18:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 shutdown_connections 2023-07-21T22:18:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f0902c3090 0x55f0902c54a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902afa70 0x55f0902b3710 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd22ffd700 0 client.16158 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2529660469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 >> 172.21.15.93:0/2529660469 conn(0x55f09029aae0 msgr2=0x55f09029f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 shutdown_connections 2023-07-21T22:18:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2529660469 wait complete. 2023-07-21T22:18:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.051+0000 7fcd55b2ab80 20 client.16158 trim_cache size 0 max 16384 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.066+0000 7fcd55b2ab80 1 Processor -- start 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.066+0000 7fcd55b2ab80 1 -- start start 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0902ad770 con 0x55f0901016b0 2023-07-21T22:18:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0902a19d0 con 0x55f0902afa70 2023-07-21T22:18:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0902a0cb0 con 0x55f0902af620 2023-07-21T22:18:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45878/0 (socket says 172.21.15.93:45878) 2023-07-21T22:18:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 -- 172.21.15.93:0/2982219199 learned_addr learned my addr 172.21.15.93:0/2982219199 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd427fc700 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd437fe700 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 -- 172.21.15.93:0/2982219199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 msgr2=0x55f0902b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 -- 172.21.15.93:0/2982219199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 msgr2=0x55f0902b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 -- 172.21.15.93:0/2982219199 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09000dc00 con 0x55f0901016b0 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55f0902c5db0 tx=0x7fcd34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f358e68ed5c4d9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd2a7fc700 1 -- 172.21.15.93:0/2982219199 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34022030 con 0x55f0901016b0 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd2a7fc700 1 -- 172.21.15.93:0/2982219199 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd34023040 con 0x55f0901016b0 2023-07-21T22:18:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.067+0000 7fcd2a7fc700 1 -- 172.21.15.93:0/2982219199 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401d650 con 0x55f0901016b0 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2982219199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 msgr2=0x55f0902ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55f0902c5db0 tx=0x7fcd34014000 comp rx=0 tx=0).stop 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2982219199 shutdown_connections 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0902afa70 0x55f0902b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0901016b0 0x55f0902ae590 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/2982219199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902af620 0x55f0902b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2982219199 >> 172.21.15.93:0/2982219199 conn(0x55f09029aae0 msgr2=0x55f0902afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2982219199 shutdown_connections 2023-07-21T22:18:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/2982219199 wait complete. 2023-07-21T22:18:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.068+0000 7fcd55b2ab80 1 -- start start 2023-07-21T22:18:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 Processor -- start 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902afa70 0x55f090315f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0902b9da0 con 0x55f0902af620 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f090316490 con 0x55f0901016b0 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0903165d0 con 0x55f0902afa70 2023-07-21T22:18:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd55b2ab80 10 client.0 _ll_register_callbacks cb 0x7fcd3c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45886/0 (socket says 172.21.15.93:45886) 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd427fc700 1 -- 172.21.15.93:0/3638240229 learned_addr learned my addr 172.21.15.93:0/3638240229 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd437fe700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902afa70 0x55f090315f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 -- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902afa70 msgr2=0x55f090315f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902afa70 0x55f090315f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 -- 172.21.15.93:0/3638240229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 msgr2=0x55f0902b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd42ffd700 1 -- 172.21.15.93:0/3638240229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3402bab0 con 0x55f0901016b0 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.069+0000 7fcd427fc700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd42ffd700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b8f70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009fd0 tx=0x7fcd340057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7931fd3f9d8aa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34022070 con 0x55f0901016b0 2023-07-21T22:18:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fcd3402b2b0 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd38010880 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd3402b470 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 10 client.25712 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f09004b1f0 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd38010880 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fcd34032070 con 0x55f0901016b0 2023-07-21T22:18:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 10 client.25712 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 20 client.25712 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f090317c60 0x55f09031a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 4 client.25712 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.070+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0901b1ff0 con 0x55f090317c60 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.071+0000 7fcd427fc700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f090317c60 0x55f09031a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.071+0000 7fcd22ffd700 10 client.25712 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.071+0000 7fcd427fc700 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f090317c60 0x55f09031a070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8e02750a50a18d server_cookie=27312c4ffde46aa0 in_seq=0 out_seq=0 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd22ffd700 1 -- 172.21.15.93:0/3638240229 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0901b1ff0 con 0x55f090317c60 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd22ffd700 10 client.25712 handle_command_reply: tid=0 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 client.25712 shutdown 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 msgr2=0x55f0902b8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b8f70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009fd0 tx=0x7fcd340057e0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 shutdown_connections 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f090317c60 0x55f09031a070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f0902afa70 0x55f090315f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd22ffd700 0 client.25712 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0901016b0 0x55f0902b8f70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 --2- 172.21.15.93:0/3638240229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f0902af620 0x55f0902b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 >> 172.21.15.93:0/3638240229 conn(0x55f09029aae0 msgr2=0x55f0902ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 shutdown_connections 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 1 -- 172.21.15.93:0/3638240229 wait complete. 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.099+0000 7fcd55b2ab80 20 client.25712 trim_cache size 0 max 16384 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:18:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:18:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 msgr2=0x7fcd2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 0x7fcd2c0a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009c60 tx=0x7fcd34009260 comp rx=0 tx=0).stop 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 msgr2=0x7fcd441961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800e9a0 tx=0x7fcd3800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 shutdown_connections 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd4410a080 0x7fcd441997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd44100d40 0x7fcd44195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd2c0a12c0 0x7fcd2c0a3770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 --2- 172.21.15.93:0/276584078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd44101650 0x7fcd441961f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 >> 172.21.15.93:0/276584078 conn(0x7fcd440fa4d0 msgr2=0x7fcd44101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 shutdown_connections 2023-07-21T22:18:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:27.100+0000 7fcd499a5700 1 -- 172.21.15.93:0/276584078 wait complete. 2023-07-21T22:18:27.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 -- 172.21.15.93:0/3721865902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 msgr2=0x7f0f38104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/3721865902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38104f40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009990 tx=0x7f0f28009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 -- 172.21.15.93:0/3721865902 shutdown_connections 2023-07-21T22:18:27.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/3721865902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 0x7f0f3810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/3721865902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38104f40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/3721865902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 0x7f0f380fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 -- 172.21.15.93:0/3721865902 >> 172.21.15.93:0/3721865902 conn(0x7f0f380769e0 msgr2=0x7f0f38076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 -- 172.21.15.93:0/3721865902 shutdown_connections 2023-07-21T22:18:27.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.147+0000 7f0f3dd56700 1 -- 172.21.15.93:0/3721865902 wait complete. 2023-07-21T22:18:27.195 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 Processor -- start 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 -- start start 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 0x7f0f3819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 0x7f0f3819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38196000 con 0x7f0f38104b70 2023-07-21T22:18:27.196 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38196170 con 0x7f0f380ff4f0 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f3dd56700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38196330 con 0x7f0f380feb40 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.148+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45902/0 (socket says 172.21.15.93:45902) 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 -- 172.21.15.93:0/1580031320 learned_addr learned my addr 172.21.15.93:0/1580031320 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f36ffd700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 0x7f0f3819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f377fe700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 0x7f0f3819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 -- 172.21.15.93:0/1580031320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 msgr2=0x7f0f3819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 0x7f0f3819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 -- 172.21.15.93:0/1580031320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 msgr2=0x7f0f3819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 0x7f0f3819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 -- 172.21.15.93:0/1580031320 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280095f0 con 0x7f0f38104b70 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f37fff700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b970 tx=0x7f0f3000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000d860 con 0x7f0f38104b70 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f3000de80 con 0x7f0f38104b70 2023-07-21T22:18:27.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f38196610 con 0x7f0f38104b70 2023-07-21T22:18:27.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30017400 con 0x7f0f38104b70 2023-07-21T22:18:27.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.149+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f381a2910 con 0x7f0f38104b70 2023-07-21T22:18:27.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.150+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f38066a70 con 0x7f0f38104b70 2023-07-21T22:18:27.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.152+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0f3000e630 con 0x7f0f38104b70 2023-07-21T22:18:27.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.152+0000 7f0f34ff9700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 0x7f0f200a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.153+0000 7f0f377fe700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 0x7f0f200a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.153+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0f30006e40 con 0x7f0f38104b70 2023-07-21T22:18:27.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.153+0000 7f0f377fe700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 0x7f0f200a37b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800b5c0 tx=0x7f0f28005960 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.155+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0f30095f20 con 0x7f0f38104b70 2023-07-21T22:18:27.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 43264 3.91 MB/sec warmup 64 sec latency 525.265 ms 2023-07-21T22:18:27.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.391+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f38196ea0 con 0x7f0f38104b70 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.392+0000 7f0f34ff9700 1 -- 172.21.15.93:0/1580031320 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f0f30099540 con 0x7f0f38104b70 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:18:01.146844+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/117715569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":117715569},{"type":"v1","addr":"172.21.15.134:6837","nonce":117715569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":54,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:18:27.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 msgr2=0x7f0f200a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 0x7f0f200a37b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800b5c0 tx=0x7f0f28005960 comp rx=0 tx=0).stop 2023-07-21T22:18:27.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 msgr2=0x7f0f38195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b970 tx=0x7f0f3000bc80 comp rx=0 tx=0).stop 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 shutdown_connections 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0f200a1300 0x7f0f200a37b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0f38104b70 0x7f0f38195990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f380feb40 0x7f0f3819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 --2- 172.21.15.93:0/1580031320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f380ff4f0 0x7f0f3819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.440 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 >> 172.21.15.93:0/1580031320 conn(0x7f0f380769e0 msgr2=0x7f0f38105900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 shutdown_connections 2023-07-21T22:18:27.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.394+0000 7f0f3dd56700 1 -- 172.21.15.93:0/1580031320 wait complete. 2023-07-21T22:18:27.441 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 32 2023-07-21T22:18:27.451 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:18:27.515 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:27 smithi093 ceph-mon[96579]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:27.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:27 smithi194 ceph-mon[98288]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:27.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- 172.21.15.93:0/696208963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 msgr2=0x7fef5c10b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 --2- 172.21.15.93:0/696208963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c10b790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fef5400b100 tx=0x7fef5400b410 comp rx=0 tx=0).stop 2023-07-21T22:18:27.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- 172.21.15.93:0/696208963 shutdown_connections 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 --2- 172.21.15.93:0/696208963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c10b790 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 --2- 172.21.15.93:0/696208963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c0ffe00 0x7fef5c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 --2- 172.21.15.93:0/696208963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c100710 0x7fef5c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- 172.21.15.93:0/696208963 >> 172.21.15.93:0/696208963 conn(0x7fef5c0742e0 msgr2=0x7fef5c0746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- 172.21.15.93:0/696208963 shutdown_connections 2023-07-21T22:18:27.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- 172.21.15.93:0/696208963 wait complete. 2023-07-21T22:18:27.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 Processor -- start 2023-07-21T22:18:27.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.569+0000 7fef64016700 1 -- start start 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 0x7fef5c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c100710 0x7fef5c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c1950d0 con 0x7fef5c1093b0 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c195240 con 0x7fef5c0ffe00 2023-07-21T22:18:27.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5c195400 con 0x7fef5c100710 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45916/0 (socket says 172.21.15.93:45916) 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 -- 172.21.15.93:0/2248586428 learned_addr learned my addr 172.21.15.93:0/2248586428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef615b1700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c100710 0x7fef5c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef61db2700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 0x7fef5c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 -- 172.21.15.93:0/2248586428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c100710 msgr2=0x7fef5c19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c100710 0x7fef5c19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 -- 172.21.15.93:0/2248586428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 msgr2=0x7fef5c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 0x7fef5c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 -- 172.21.15.93:0/2248586428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef54009d00 con 0x7fef5c1093b0 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef61db2700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 0x7fef5c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef625b3700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fef54007850 tx=0x7fef54007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef52ffd700 1 -- 172.21.15.93:0/2248586428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400e040 con 0x7fef5c1093b0 2023-07-21T22:18:27.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef52ffd700 1 -- 172.21.15.93:0/2248586428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef54003e10 con 0x7fef5c1093b0 2023-07-21T22:18:27.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c195680 con 0x7fef5c1093b0 2023-07-21T22:18:27.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef52ffd700 1 -- 172.21.15.93:0/2248586428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5401aba0 con 0x7fef5c1093b0 2023-07-21T22:18:27.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.570+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0feed0 con 0x7fef5c1093b0 2023-07-21T22:18:27.621 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.572+0000 7fef52ffd700 1 -- 172.21.15.93:0/2248586428 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fef5402c3f0 con 0x7fef5c1093b0 2023-07-21T22:18:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.573+0000 7fef52ffd700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 0x7fef480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.573+0000 7fef52ffd700 1 -- 172.21.15.93:0/2248586428 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fef5401e070 con 0x7fef5c1093b0 2023-07-21T22:18:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.573+0000 7fef61db2700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 0x7fef480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.573+0000 7fef61db2700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 0x7fef480a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef58005950 tx=0x7fef580058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 Processor -- start 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 -- start start 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb029801b0 0x55bb02980580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0297bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb02880cb0 con 0x55bb02980ac0 2023-07-21T22:18:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb028d3a40 con 0x55bb029801b0 2023-07-21T22:18:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef7019bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb028cced0 con 0x55bb0297b930 2023-07-21T22:18:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.574+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45922/0 (socket says 172.21.15.93:45922) 2023-07-21T22:18:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 -- 172.21.15.93:0/2141656428 learned_addr learned my addr 172.21.15.93:0/2141656428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef625b3700 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0297bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef61db2700 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb029801b0 0x55bb02980580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 -- 172.21.15.93:0/2141656428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 msgr2=0x55bb0297bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0297bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 -- 172.21.15.93:0/2141656428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb029801b0 msgr2=0x55bb02980580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb029801b0 0x55bb02980580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 -- 172.21.15.93:0/2141656428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb026fade0 con 0x55bb02980ac0 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef615b1700 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00ac00 tx=0x7fef4c00af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9a7d5946d06cc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef47fff700 1 -- 172.21.15.93:0/2141656428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00fd60 con 0x55bb02980ac0 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef47fff700 1 -- 172.21.15.93:0/2141656428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4c00fec0 con 0x55bb02980ac0 2023-07-21T22:18:27.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.575+0000 7fef47fff700 1 -- 172.21.15.93:0/2141656428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0107e0 con 0x55bb02980ac0 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- 172.21.15.93:0/2141656428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 msgr2=0x55bb0297b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00ac00 tx=0x7fef4c00af10 comp rx=0 tx=0).stop 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- 172.21.15.93:0/2141656428 shutdown_connections 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0297bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb029801b0 0x55bb02980580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2141656428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297b350 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- 172.21.15.93:0/2141656428 >> 172.21.15.93:0/2141656428 conn(0x55bb02970ef0 msgr2=0x55bb02976830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- 172.21.15.93:0/2141656428 shutdown_connections 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- 172.21.15.93:0/2141656428 wait complete. 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 -- start start 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.576+0000 7fef7019bb80 1 Processor -- start 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb02987c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb02985590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0286cac0 con 0x55bb029801b0 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0286c7c0 con 0x55bb02980ac0 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb02750920 con 0x55bb0297b930 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef7019bb80 10 client.0 _ll_register_callbacks cb 0x55bb0286a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45928/0 (socket says 172.21.15.93:45928) 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 -- 172.21.15.93:0/2143930510 learned_addr learned my addr 172.21.15.93:0/2143930510 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef61db2700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb02987c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef625b3700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb02985590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 -- 172.21.15.93:0/2143930510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 msgr2=0x55bb02987c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb02987c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 -- 172.21.15.93:0/2143930510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 msgr2=0x55bb02985590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb02985590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef615b1700 1 -- 172.21.15.93:0/2143930510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c00a830 con 0x55bb029801b0 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef61db2700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb02987c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.577+0000 7fef625b3700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb02985590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef615b1700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00f6a0 tx=0x7fef4c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae8469bc348b7f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c004950 con 0x55bb029801b0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fef4c005d70 con 0x55bb029801b0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb028c5eb0 con 0x55bb029801b0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c004ab0 con 0x55bb029801b0 2023-07-21T22:18:27.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 10 client.16182 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bb026e3c60 con 0x55bb029801b0 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb028c5eb0 con 0x55bb029801b0 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fef4c023440 con 0x55bb029801b0 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 10 client.16182 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 20 client.16182 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029953b0 0x55bb029977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 4 client.16182 mds_command: new command op to 14589 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb02888050 con 0x55bb029953b0 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.578+0000 7fef61db2700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029953b0 0x55bb029977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef457fa700 10 client.16182 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef61db2700 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029953b0 0x55bb029977c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=829e4ded43a51f08 server_cookie=dfaa8453e904b232 in_seq=0 out_seq=0 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef457fa700 1 -- 172.21.15.93:0/2143930510 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb02888050 con 0x55bb029953b0 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef457fa700 10 client.16182 handle_command_reply: tid=0 2023-07-21T22:18:27.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 client.16182 shutdown 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 msgr2=0x55bb02984ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00f6a0 tx=0x7fef4c0052f0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 shutdown_connections 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb02985590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029953b0 0x55bb029977c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb02987c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 --2- 172.21.15.93:0/2143930510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb02984ea0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef457fa700 0 client.16182 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:18:27.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 >> 172.21.15.93:0/2143930510 conn(0x55bb02970ef0 msgr2=0x55bb02976420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.579+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 shutdown_connections 2023-07-21T22:18:27.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.580+0000 7fef7019bb80 1 -- 172.21.15.93:0/2143930510 wait complete. 2023-07-21T22:18:27.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.580+0000 7fef7019bb80 20 client.16182 trim_cache size 0 max 16384 2023-07-21T22:18:27.641 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:27 smithi134 ceph-mon[97873]: pgmap v63: 97 pgs: 97 active+clean; 33 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 751 KiB/s wr, 104 op/s 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.592+0000 7fef7019bb80 1 Processor -- start 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.592+0000 7fef7019bb80 1 -- start start 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb0297b930 0x55bb029877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 0x55bb02987d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb02983430 con 0x55bb02980ac0 2023-07-21T22:18:27.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb029755e0 con 0x55bb0297b930 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef7019bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb0299c120 con 0x55bb029801b0 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45938/0 (socket says 172.21.15.93:45938) 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 -- 172.21.15.93:0/3136729223 learned_addr learned my addr 172.21.15.93:0/3136729223 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef615b1700 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 0x55bb02987d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 -- 172.21.15.93:0/3136729223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 msgr2=0x55bb02987d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 0x55bb02987d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 -- 172.21.15.93:0/3136729223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb0297b930 msgr2=0x55bb029877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb0297b930 0x55bb029877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 -- 172.21.15.93:0/3136729223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb02721250 con 0x55bb02980ac0 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef615b1700 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 0x55bb02987d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef625b3700 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fef540079a0 tx=0x7fef5409cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dc3ef799e5415b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef50ff9700 1 -- 172.21.15.93:0/3136729223 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540a1080 con 0x55bb02980ac0 2023-07-21T22:18:27.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef50ff9700 1 -- 172.21.15.93:0/3136729223 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef540a5410 con 0x55bb02980ac0 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.593+0000 7fef50ff9700 1 -- 172.21.15.93:0/3136729223 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5409c7a0 con 0x55bb02980ac0 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- 172.21.15.93:0/3136729223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 msgr2=0x55bb0297bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fef540079a0 tx=0x7fef5409cc00 comp rx=0 tx=0).stop 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- 172.21.15.93:0/3136729223 shutdown_connections 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb02980ac0 0x55bb0297bd20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb0297b930 0x55bb029877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3136729223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb029801b0 0x55bb02987d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- 172.21.15.93:0/3136729223 >> 172.21.15.93:0/3136729223 conn(0x55bb02970ef0 msgr2=0x55bb02979a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- 172.21.15.93:0/3136729223 shutdown_connections 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- 172.21.15.93:0/3136729223 wait complete. 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 -- start start 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.594+0000 7fef7019bb80 1 Processor -- start 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0298a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb029e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb029e87b0 con 0x55bb029801b0 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb029e88f0 con 0x55bb02980ac0 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb029e8ab0 con 0x55bb0297b930 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 10 client.0 _ll_register_callbacks cb 0x7fef4c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45942/0 (socket says 172.21.15.93:45942) 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 -- 172.21.15.93:0/3446747159 learned_addr learned my addr 172.21.15.93:0/3446747159 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef61db2700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0298a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef625b3700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb029e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 -- 172.21.15.93:0/3446747159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 msgr2=0x55bb0298a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0298a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 -- 172.21.15.93:0/3446747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 msgr2=0x55bb029e8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb029e8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 -- 172.21.15.93:0/3446747159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0feed0 con 0x55bb029801b0 2023-07-21T22:18:27.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef625b3700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb029e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef615b1700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0237d0 tx=0x7fef4c00a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c3f60e9f8fe522 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef457fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00f6d0 con 0x55bb029801b0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fef4c005360 con 0x55bb029801b0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.595+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb028264f0 con 0x55bb029801b0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef4c023ac0 con 0x55bb029801b0 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 10 client.16188 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.652 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bb026f92b0 con 0x55bb029801b0 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4c042080 con 0x55bb029801b0 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fef4c004220 con 0x55bb029801b0 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 20 client.16188 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029e9ea0 0x55bb029ec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.653 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 4 client.16188 mds_command: new command op to 14589 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 --> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb0271bf70 con 0x55bb029e9ea0 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef61db2700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029e9ea0 0x55bb029ec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 10 client.16188 ms_handle_connect on v2:172.21.15.93:6840/4095455709 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef61db2700 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029e9ea0 0x55bb029ec2b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5848509b8628409a server_cookie=d15d285f309bbcf in_seq=0 out_seq=0 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 1 -- 172.21.15.93:0/3446747159 <== mds.4 v2:172.21.15.93:6840/4095455709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb0271bf70 con 0x55bb029e9ea0 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef457fa700 10 client.16188 handle_command_reply: tid=0 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.596+0000 7fef7019bb80 1 client.16188 shutdown 2023-07-21T22:18:27.654 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 msgr2=0x55bb0298ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0237d0 tx=0x7fef4c00a300 comp rx=0 tx=0).stop 2023-07-21T22:18:27.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 shutdown_connections 2023-07-21T22:18:27.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb02980ac0 0x55bb029e8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:6840/4095455709,v1:172.21.15.93:6841/4095455709] conn(0x55bb029e9ea0 0x55bb029ec2b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.655 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb0297b930 0x55bb0298a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef457fa700 0 client.16188 ms_handle_reset on v2:172.21.15.93:6840/4095455709 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 --2- 172.21.15.93:0/3446747159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb029801b0 0x55bb0298ac90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 >> 172.21.15.93:0/3446747159 conn(0x55bb02970ef0 msgr2=0x55bb0297bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 shutdown_connections 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 1 -- 172.21.15.93:0/3446747159 wait complete. 2023-07-21T22:18:27.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef7019bb80 20 client.16188 trim_cache size 0 max 16384 2023-07-21T22:18:27.657 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:18:27.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 msgr2=0x7fef480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 0x7fef480a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef58005950 tx=0x7fef580058e0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.658 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 msgr2=0x7fef5c194a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fef54007850 tx=0x7fef54007880 comp rx=0 tx=0).stop 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 shutdown_connections 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fef480a12c0 0x7fef480a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef5c1093b0 0x7fef5c194a60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef5c0ffe00 0x7fef5c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 --2- 172.21.15.93:0/2248586428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef5c100710 0x7fef5c19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 >> 172.21.15.93:0/2248586428 conn(0x7fef5c0742e0 msgr2=0x7fef5c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 shutdown_connections 2023-07-21T22:18:27.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.597+0000 7fef64016700 1 -- 172.21.15.93:0/2248586428 wait complete. 2023-07-21T22:18:27.660 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:18:27.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 -- 172.21.15.93:0/401361650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0ff380 msgr2=0x7fb5ac10b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/401361650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac10b810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b0a0 tx=0x7fb5a400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 -- 172.21.15.93:0/401361650 shutdown_connections 2023-07-21T22:18:27.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/401361650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac10b810 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/401361650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/401361650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac0fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 -- 172.21.15.93:0/401361650 >> 172.21.15.93:0/401361650 conn(0x7fb5ac0fa4b0 msgr2=0x7fb5ac0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 -- 172.21.15.93:0/401361650 shutdown_connections 2023-07-21T22:18:27.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.776+0000 7fb5b0dde700 1 -- 172.21.15.93:0/401361650 wait complete. 2023-07-21T22:18:27.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.777+0000 7fb5b0dde700 1 Processor -- start 2023-07-21T22:18:27.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.777+0000 7fb5b0dde700 1 -- start start 2023-07-21T22:18:27.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1a20c0 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1a2200 con 0x7fb5ac0ff380 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5b0dde700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac1a2340 con 0x7fb5ac1049e0 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45954/0 (socket says 172.21.15.93:45954) 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 -- 172.21.15.93:0/663415558 learned_addr learned my addr 172.21.15.93:0/663415558 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 -- 172.21.15.93:0/663415558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 msgr2=0x7fb5ac1a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac1a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 -- 172.21.15.93:0/663415558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac0ff380 msgr2=0x7fb5ac197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 -- 172.21.15.93:0/663415558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0009520 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00e9a0 tx=0x7fb59c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.778+0000 7fb59b7fe700 1 -- 172.21.15.93:0/663415558 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c009790 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.779+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a4009d00 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.779+0000 7fb59b7fe700 1 -- 172.21.15.93:0/663415558 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb59c0044e0 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.779+0000 7fb59b7fe700 1 -- 172.21.15.93:0/663415558 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c003d20 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.779+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac1a2970 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.781+0000 7fb59b7fe700 1 -- 172.21.15.93:0/663415558 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb59c003e80 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.781+0000 7fb59b7fe700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 0x7fb5940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.781+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 0x7fb5940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.781+0000 7fb59b7fe700 1 -- 172.21.15.93:0/663415558 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb59c014070 con 0x7fb5ac0fe9d0 2023-07-21T22:18:27.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 0x7fb5940a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac068750 tx=0x7fb5a0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 Processor -- start 2023-07-21T22:18:27.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 -- start start 2023-07-21T22:18:27.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 0x5629e64ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e63bacb0 con 0x5629e64baac0 2023-07-21T22:18:27.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e640da40 con 0x5629e64b5930 2023-07-21T22:18:27.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.782+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6406ed0 con 0x5629e64ba1b0 2023-07-21T22:18:27.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45964/0 (socket says 172.21.15.93:45964) 2023-07-21T22:18:27.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2688221773 learned_addr learned my addr 172.21.15.93:0/2688221773 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 0x5629e64ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2688221773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 msgr2=0x5629e64ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 0x5629e64ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2688221773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 msgr2=0x5629e64b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2688221773 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e6234de0 con 0x5629e64baac0 2023-07-21T22:18:27.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 0x5629e64ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:27.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e610 tx=0x7fb5a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffbe71f6617b921 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb598ff9700 1 -- 172.21.15.93:0/2688221773 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0004220 con 0x5629e64baac0 2023-07-21T22:18:27.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb598ff9700 1 -- 172.21.15.93:0/2688221773 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a0021070 con 0x5629e64baac0 2023-07-21T22:18:27.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.783+0000 7fb598ff9700 1 -- 172.21.15.93:0/2688221773 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a001c650 con 0x5629e64baac0 2023-07-21T22:18:27.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2688221773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 msgr2=0x5629e64b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e610 tx=0x7fb5a0014000 comp rx=0 tx=0).stop 2023-07-21T22:18:27.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2688221773 shutdown_connections 2023-07-21T22:18:27.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64ba1b0 0x5629e64ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2688221773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64baac0 0x5629e64b5350 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2688221773 >> 172.21.15.93:0/2688221773 conn(0x5629e64aaef0 msgr2=0x5629e64b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2688221773 shutdown_connections 2023-07-21T22:18:27.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2688221773 wait complete. 2023-07-21T22:18:27.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 -- start start 2023-07-21T22:18:27.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.784+0000 7fb5bcf63b80 1 Processor -- start 2023-07-21T22:18:27.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e64bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e63a6ac0 con 0x5629e64ba1b0 2023-07-21T22:18:27.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e63a67c0 con 0x5629e64b5930 2023-07-21T22:18:27.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e628a920 con 0x5629e64baac0 2023-07-21T22:18:27.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 10 client.0 _ll_register_callbacks cb 0x5629e63a4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45970/0 (socket says 172.21.15.93:45970) 2023-07-21T22:18:27.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2911665824 learned_addr learned my addr 172.21.15.93:0/2911665824 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e64bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 msgr2=0x5629e64bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e64bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2911665824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 msgr2=0x5629e64c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/2911665824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a0017070 con 0x5629e64ba1b0 2023-07-21T22:18:27.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0003800 tx=0x7fb5a0003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5bc7afdfb72697 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0005480 con 0x5629e64ba1b0 2023-07-21T22:18:27.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.785+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629e63604f0 con 0x5629e64ba1b0 2023-07-21T22:18:27.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb5a0003bc0 con 0x5629e64ba1b0 2023-07-21T22:18:27.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a00055e0 con 0x5629e64ba1b0 2023-07-21T22:18:27.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 10 client.16200 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5629e621dc60 con 0x5629e64ba1b0 2023-07-21T22:18:27.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629e63604f0 con 0x5629e64ba1b0 2023-07-21T22:18:27.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fb5a0031530 con 0x5629e64ba1b0 2023-07-21T22:18:27.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 10 client.16200 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:18:27.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 20 client.16200 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e64cf340 0x5629e64d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 4 client.16200 mds_command: new command op to 24448 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:27.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629e63ffeb0 con 0x5629e64cf340 2023-07-21T22:18:27.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.786+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e64cf340 0x5629e64d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb591ffb700 10 client.16200 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:18:27.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e64cf340 0x5629e64d1750 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=944beb73b770cf31 server_cookie=f53295aac3399214 in_seq=0 out_seq=0 2023-07-21T22:18:27.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb591ffb700 1 -- 172.21.15.93:0/2911665824 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629e63ffeb0 con 0x5629e64cf340 2023-07-21T22:18:27.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb591ffb700 10 client.16200 handle_command_reply: tid=0 2023-07-21T22:18:27.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 client.16200 shutdown 2023-07-21T22:18:27.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 msgr2=0x5629e64bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0003800 tx=0x7fb5a0003830 comp rx=0 tx=0).stop 2023-07-21T22:18:27.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 shutdown_connections 2023-07-21T22:18:27.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e64bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64b5930 0x5629e64c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e64cf340 0x5629e64d1750 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/2911665824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64bee40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb591ffb700 0 client.16200 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:18:27.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 >> 172.21.15.93:0/2911665824 conn(0x5629e64aaef0 msgr2=0x5629e64b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 shutdown_connections 2023-07-21T22:18:27.860 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.787+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/2911665824 wait complete. 2023-07-21T22:18:27.861 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.788+0000 7fb5bcf63b80 20 client.16200 trim_cache size 0 max 16384 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.800+0000 7fb5bcf63b80 1 Processor -- start 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.800+0000 7fb5bcf63b80 1 -- start start 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64b5930 0x5629e64c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64baac0 0x5629e64b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e64b1a50 con 0x5629e64ba1b0 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e64af5e0 con 0x5629e64baac0 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e64bcea0 con 0x5629e64b5930 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45978/0 (socket says 172.21.15.93:45978) 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/3723107600 learned_addr learned my addr 172.21.15.93:0/3723107600 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.870 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64b5930 0x5629e64c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64baac0 0x5629e64b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/3723107600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64b5930 msgr2=0x5629e64c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64b5930 0x5629e64c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/3723107600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64baac0 msgr2=0x5629e64b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64baac0 0x5629e64b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 -- 172.21.15.93:0/3723107600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629e625b250 con 0x5629e64ba1b0 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0031990 tx=0x7fb5a0011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e0d99c6a5d5c4e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5997fa700 1 -- 172.21.15.93:0/3723107600 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0003950 con 0x5629e64ba1b0 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5997fa700 1 -- 172.21.15.93:0/3723107600 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a00092d0 con 0x5629e64ba1b0 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.801+0000 7fb5997fa700 1 -- 172.21.15.93:0/3723107600 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a00266e0 con 0x5629e64ba1b0 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/3723107600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 msgr2=0x5629e64c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0031990 tx=0x7fb5a0011bb0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/3723107600 shutdown_connections 2023-07-21T22:18:27.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64baac0 0x5629e64b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64b5930 0x5629e64c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/3723107600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64ba1b0 0x5629e64c1760 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/3723107600 >> 172.21.15.93:0/3723107600 conn(0x5629e64aaef0 msgr2=0x5629e64cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/3723107600 shutdown_connections 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/3723107600 wait complete. 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 -- start start 2023-07-21T22:18:27.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.802+0000 7fb5bcf63b80 1 Processor -- start 2023-07-21T22:18:27.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 0x5629e64c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e6522200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6522740 con 0x5629e64b5930 2023-07-21T22:18:27.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e6522880 con 0x5629e64ba1b0 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629e65229c0 con 0x5629e64baac0 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 10 client.0 _ll_register_callbacks cb 0x7fb5a000e4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45982/0 (socket says 172.21.15.93:45982) 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 -- 172.21.15.93:0/854614583 learned_addr learned my addr 172.21.15.93:0/854614583 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:27.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 0x5629e64c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aad9d700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e6522200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 -- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 msgr2=0x5629e6522200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e6522200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 -- 172.21.15.93:0/854614583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 msgr2=0x5629e64c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 0x5629e64c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 -- 172.21.15.93:0/854614583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629e621dc60 con 0x5629e64b5930 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 0x5629e64c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5aa59c700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00f010 tx=0x7fb59c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b75ff86b21b062 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0a2450 con 0x5629e64b5930 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fb59c00cd60 con 0x5629e64b5930 2023-07-21T22:18:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5a0021070 con 0x5629e64b5930 2023-07-21T22:18:27.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb59c0b82f0 con 0x5629e64b5930 2023-07-21T22:18:27.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.803+0000 7fb5bcf63b80 10 client.16206 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:27.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5629e62332b0 con 0x5629e64b5930 2023-07-21T22:18:27.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5a0021070 con 0x5629e64b5930 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fb59c0ca1a0 con 0x5629e64b5930 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 10 client.16206 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 10 client.16206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 20 client.16206 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e6524020 0x5629e6526400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 4 client.16206 mds_command: new command op to 24448 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 --> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629e63c2050 con 0x5629e6524020 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.804+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e6524020 0x5629e6526400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb591ffb700 10 client.16206 ms_handle_connect on v2:172.21.15.194:6834/127707947 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5a9d9b700 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e6524020 0x5629e6526400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac4269f9476fcadc server_cookie=7ee4c24c3d8f1a05 in_seq=0 out_seq=0 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb591ffb700 1 -- 172.21.15.93:0/854614583 <== mds.3 v2:172.21.15.194:6834/127707947 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2607 (crc 0 0 0) 0x5629e63c2050 con 0x5629e6524020 2023-07-21T22:18:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb591ffb700 10 client.16206 handle_command_reply: tid=0 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 client.16206 shutdown 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 msgr2=0x5629e64c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00f010 tx=0x7fb59c003680 comp rx=0 tx=0).stop 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 shutdown_connections 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5629e64baac0 0x5629e6522200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.194:6834/127707947,v1:172.21.15.194:6835/127707947] conn(0x5629e6524020 0x5629e6526400 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5629e64b5930 0x5629e64c4780 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 --2- 172.21.15.93:0/854614583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629e64ba1b0 0x5629e64c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 >> 172.21.15.93:0/854614583 conn(0x5629e64aaef0 msgr2=0x5629e64b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb591ffb700 0 client.16206 ms_handle_reset on v2:172.21.15.194:6834/127707947 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 shutdown_connections 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 1 -- 172.21.15.93:0/854614583 wait complete. 2023-07-21T22:18:27.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5bcf63b80 20 client.16206 trim_cache size 0 max 16384 2023-07-21T22:18:27.883 INFO:teuthology.orchestra.run.smithi093.stdout:[ 2023-07-21T22:18:27.883 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.883 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.883 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 951119240, 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628318, 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR" 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.884 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 1126671368, 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628311, 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG" 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 1975311948, 2023-07-21T22:18:27.885 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628310, 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG" 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 2065647689, 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628315, 2023-07-21T22:18:27.886 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT" 2023-07-21T22:18:27.887 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.887 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.887 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.887 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 2384460332, 2023-07-21T22:18:27.887 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628316, 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT" 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 2632043439, 2023-07-21T22:18:27.888 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628314, 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT" 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 2919677868, 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628312, 2023-07-21T22:18:27.889 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL" 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 3276777170, 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628320, 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT" 2023-07-21T22:18:27.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 3573899838, 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628313, 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR" 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.891 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.892 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.892 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 4234299463, 2023-07-21T22:18:27.892 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628319, 2023-07-21T22:18:27.892 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT" 2023-07-21T22:18:27.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 4292424092, 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628317, 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR" 2023-07-21T22:18:27.893 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:18:27.894 INFO:teuthology.orchestra.run.smithi093.stdout:] 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 msgr2=0x7fb5940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 0x7fb5940a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac068750 tx=0x7fb5a0009260 comp rx=0 tx=0).stop 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 msgr2=0x7fb5ac196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.805+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00e9a0 tx=0x7fb59c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 shutdown_connections 2023-07-21T22:18:27.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5ac1049e0 0x7fb5ac1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb5940a12c0 0x7fb5940a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5ac0fe9d0 0x7fb5ac196b00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 --2- 172.21.15.93:0/663415558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5ac0ff380 0x7fb5ac197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 >> 172.21.15.93:0/663415558 conn(0x7fb5ac0fa4b0 msgr2=0x7fb5ac102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 shutdown_connections 2023-07-21T22:18:27.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.806+0000 7fb5b0dde700 1 -- 172.21.15.93:0/663415558 wait complete. 2023-07-21T22:18:27.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:18:28.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.974+0000 7f7e1fff3700 1 -- 172.21.15.93:0/2707589366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180fe200 msgr2=0x7f7e180fe650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.974+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/2707589366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180fe200 0x7f7e180fe650 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009990 tx=0x7f7e08009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- 172.21.15.93:0/2707589366 shutdown_connections 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/2707589366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180feb90 0x7f7e1810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/2707589366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/2707589366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180fe200 0x7f7e180fe650 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- 172.21.15.93:0/2707589366 >> 172.21.15.93:0/2707589366 conn(0x7f7e18076200 msgr2=0x7f7e18076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- 172.21.15.93:0/2707589366 shutdown_connections 2023-07-21T22:18:28.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- 172.21.15.93:0/2707589366 wait complete. 2023-07-21T22:18:28.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 Processor -- start 2023-07-21T22:18:28.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- start start 2023-07-21T22:18:28.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180fe200 0x7f7e1819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e181960f0 con 0x7f7e180feb90 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e18196260 con 0x7f7e18105c00 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1fff3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e18196420 con 0x7f7e180fe200 2023-07-21T22:18:28.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.975+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45998/0 (socket says 172.21.15.93:45998) 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 -- 172.21.15.93:0/4153654092 learned_addr learned my addr 172.21.15.93:0/4153654092 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180fe200 0x7f7e1819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1e590700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 -- 172.21.15.93:0/4153654092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180fe200 msgr2=0x7f7e1819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180fe200 0x7f7e1819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 -- 172.21.15.93:0/4153654092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 msgr2=0x7f7e18195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 -- 172.21.15.93:0/4153654092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080095f0 con 0x7f7e180feb90 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1e590700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e0effd700 1 -- 172.21.15.93:0/4153654092 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0801d070 con 0x7f7e180feb90 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e0effd700 1 -- 172.21.15.93:0/4153654092 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e08003e20 con 0x7f7e180feb90 2023-07-21T22:18:28.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e181966a0 con 0x7f7e180feb90 2023-07-21T22:18:28.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e0effd700 1 -- 172.21.15.93:0/4153654092 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08005700 con 0x7f7e180feb90 2023-07-21T22:18:28.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.976+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e181a2a00 con 0x7f7e180feb90 2023-07-21T22:18:28.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.978+0000 7f7e0effd700 1 -- 172.21.15.93:0/4153654092 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7e0800fbb0 con 0x7f7e180feb90 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.979+0000 7f7e0effd700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 0x7f7e040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.979+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 0x7f7e040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.979+0000 7f7e0effd700 1 -- 172.21.15.93:0/4153654092 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e080cdf50 con 0x7f7e180feb90 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.979+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 0x7f7e040a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005950 tx=0x7f7e1400b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 Processor -- start 2023-07-21T22:18:28.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 -- start start 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e4e6cb0 con 0x563c5e5e1930 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e539a40 con 0x563c5e5e61b0 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e532ed0 con 0x563c5e5e6ac0 2023-07-21T22:18:28.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46008/0 (socket says 172.21.15.93:46008) 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 -- 172.21.15.93:0/1207959337 learned_addr learned my addr 172.21.15.93:0/1207959337 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 -- 172.21.15.93:0/1207959337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 msgr2=0x563c5e5e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 -- 172.21.15.93:0/1207959337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 msgr2=0x563c5e5e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5e6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1e590700 1 -- 172.21.15.93:0/1207959337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5e360de0 con 0x563c5e5e1930 2023-07-21T22:18:28.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.980+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:28.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e1e590700 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e40 tx=0x7f7e100077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d9531bb45bcaa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e03fff700 1 -- 172.21.15.93:0/1207959337 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10018070 con 0x563c5e5e1930 2023-07-21T22:18:28.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e03fff700 1 -- 172.21.15.93:0/1207959337 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e1000f050 con 0x563c5e5e1930 2023-07-21T22:18:28.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e03fff700 1 -- 172.21.15.93:0/1207959337 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10014410 con 0x563c5e5e1930 2023-07-21T22:18:28.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1207959337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 msgr2=0x563c5e5e1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009e40 tx=0x7f7e100077e0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1207959337 shutdown_connections 2023-07-21T22:18:28.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.981+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e1930 0x563c5e5e1da0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1207959337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1207959337 >> 172.21.15.93:0/1207959337 conn(0x563c5e5d6ef0 msgr2=0x563c5e5dc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1207959337 shutdown_connections 2023-07-21T22:18:28.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1207959337 wait complete. 2023-07-21T22:18:28.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 -- start start 2023-07-21T22:18:28.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.982+0000 7f7e2c178b80 1 Processor -- start 2023-07-21T22:18:28.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5edb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e4d2ac0 con 0x563c5e5e6ac0 2023-07-21T22:18:28.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e4d27c0 con 0x563c5e5e61b0 2023-07-21T22:18:28.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e3b6920 con 0x563c5e5e1930 2023-07-21T22:18:28.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 10 client.0 _ll_register_callbacks cb 0x563c5e4d0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46014/0 (socket says 172.21.15.93:46014) 2023-07-21T22:18:28.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 -- 172.21.15.93:0/498078387 learned_addr learned my addr 172.21.15.93:0/498078387 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 -- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 msgr2=0x563c5e5edb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5edb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 -- 172.21.15.93:0/498078387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 msgr2=0x563c5e5eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 -- 172.21.15.93:0/498078387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e10009a70 con 0x563c5e5e6ac0 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e1e590700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e100057b0 tx=0x7f7e100156d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8286eae44534e356 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e017fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10018070 con 0x563c5e5e6ac0 2023-07-21T22:18:28.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e10015e30 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.983+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c5e48c4f0 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1001e450 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 10 client.16218 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563c5e349c60 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c5e48c4f0 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7e1001d610 con 0x563c5e5e6ac0 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 10 client.16218 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 10 client.16218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:18:28.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 20 client.16218 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e5fb350 0x563c5e5fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 4 client.16218 mds_command: new command op to 24460 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:28.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c5e52beb0 con 0x563c5e5fb350 2023-07-21T22:18:28.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.984+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e5fb350 0x563c5e5fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e017fa700 10 client.16218 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:18:28.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e5fb350 0x563c5e5fd760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d13cc888963586f server_cookie=1bd3b23ddf679e1e in_seq=0 out_seq=0 2023-07-21T22:18:28.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e017fa700 1 -- 172.21.15.93:0/498078387 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c5e52beb0 con 0x563c5e5fb350 2023-07-21T22:18:28.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e017fa700 10 client.16218 handle_command_reply: tid=0 2023-07-21T22:18:28.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 client.16218 shutdown 2023-07-21T22:18:28.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 msgr2=0x563c5e5eb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e100057b0 tx=0x7f7e100156d0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 shutdown_connections 2023-07-21T22:18:28.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e6ac0 0x563c5e5eb3d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e5fb350 0x563c5e5fd760 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5edb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e017fa700 0 client.16218 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:18:28.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/498078387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e61b0 0x563c5e5eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 >> 172.21.15.93:0/498078387 conn(0x563c5e5d6ef0 msgr2=0x563c5e5dc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 shutdown_connections 2023-07-21T22:18:28.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.985+0000 7f7e2c178b80 1 -- 172.21.15.93:0/498078387 wait complete. 2023-07-21T22:18:28.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.986+0000 7f7e2c178b80 20 client.16218 trim_cache size 0 max 16384 2023-07-21T22:18:28.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 Processor -- start 2023-07-21T22:18:28.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 -- start start 2023-07-21T22:18:28.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e5d69b0 con 0x563c5e5e61b0 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e5d4050 con 0x563c5e5e6ac0 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.998+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e5f7e30 con 0x563c5e5e1930 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46020/0 (socket says 172.21.15.93:46020) 2023-07-21T22:18:28.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 -- 172.21.15.93:0/2558006989 learned_addr learned my addr 172.21.15.93:0/2558006989 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 -- 172.21.15.93:0/2558006989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 msgr2=0x563c5e5f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1e590700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 -- 172.21.15.93:0/2558006989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 msgr2=0x563c5e5e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 -- 172.21.15.93:0/2558006989 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5e387250 con 0x563c5e5e61b0 2023-07-21T22:18:28.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1e590700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b5c0 tx=0x7f7e08003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdd0b0c6a8d57d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e0cff9700 1 -- 172.21.15.93:0/2558006989 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0809b0d0 con 0x563c5e5e61b0 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e0cff9700 1 -- 172.21.15.93:0/2558006989 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0809f100 con 0x563c5e5e61b0 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:27.999+0000 7f7e0cff9700 1 -- 172.21.15.93:0/2558006989 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0809a5c0 con 0x563c5e5e61b0 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- 172.21.15.93:0/2558006989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 msgr2=0x563c5e5e1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b5c0 tx=0x7f7e08003f90 comp rx=0 tx=0).stop 2023-07-21T22:18:28.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- 172.21.15.93:0/2558006989 shutdown_connections 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e5e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e5f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/2558006989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e5e1d00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- 172.21.15.93:0/2558006989 >> 172.21.15.93:0/2558006989 conn(0x563c5e5d6ef0 msgr2=0x563c5e5dfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- 172.21.15.93:0/2558006989 shutdown_connections 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- 172.21.15.93:0/2558006989 wait complete. 2023-07-21T22:18:28.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- start start 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 Processor -- start 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e648470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e6471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e647790 con 0x563c5e5e61b0 2023-07-21T22:18:28.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e647900 con 0x563c5e5e6ac0 2023-07-21T22:18:28.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.000+0000 7f7e2c178b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5e647ac0 con 0x563c5e5e1930 2023-07-21T22:18:28.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e2c178b80 10 client.0 _ll_register_callbacks cb 0x7f7e10009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46026/0 (socket says 172.21.15.93:46026) 2023-07-21T22:18:28.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 -- 172.21.15.93:0/1742155918 learned_addr learned my addr 172.21.15.93:0/1742155918 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e648470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1e590700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e6471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 -- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 msgr2=0x563c5e648470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e648470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 -- 172.21.15.93:0/1742155918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 msgr2=0x563c5e6471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e6471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 -- 172.21.15.93:0/1742155918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e181a2a00 con 0x563c5e5e61b0 2023-07-21T22:18:28.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e1d58e700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005b40 tx=0x7f7e08009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b926ce109f2a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e017fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080b7070 con 0x563c5e5e61b0 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7e0809b0d0 con 0x563c5e5e61b0 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e1000f050 con 0x563c5e5e61b0 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0809a7d0 con 0x563c5e5e61b0 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e2c178b80 10 client.16224 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563c5e35f2b0 con 0x563c5e5e61b0 2023-07-21T22:18:28.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.001+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e080cb020 con 0x563c5e5e61b0 2023-07-21T22:18:28.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7e080a3000 con 0x563c5e5e61b0 2023-07-21T22:18:28.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 10 client.16224 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-07-21T22:18:28.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 20 client.16224 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e64ffd0 0x563c5e6523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 4 client.16224 mds_command: new command op to 24460 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:18:28.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 --> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c5e4ee050 con 0x563c5e64ffd0 2023-07-21T22:18:28.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e64ffd0 0x563c5e6523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e017fa700 10 client.16224 ms_handle_connect on v2:172.21.15.194:6836/714728190 2023-07-21T22:18:28.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e1dd8f700 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e64ffd0 0x563c5e6523e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21385865ef862336 server_cookie=41e848e3ea72c676 in_seq=0 out_seq=0 2023-07-21T22:18:28.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e017fa700 1 -- 172.21.15.93:0/1742155918 <== mds.2 v2:172.21.15.194:6836/714728190 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+1412 (crc 0 0 0) 0x563c5e4ee050 con 0x563c5e64ffd0 2023-07-21T22:18:28.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e017fa700 10 client.16224 handle_command_reply: tid=0 2023-07-21T22:18:28.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.002+0000 7f7e2c178b80 1 client.16224 shutdown 2023-07-21T22:18:28.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 msgr2=0x563c5e6489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005b40 tx=0x7f7e08009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 shutdown_connections 2023-07-21T22:18:28.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563c5e5e6ac0 0x563c5e6471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:6836/714728190,v1:172.21.15.194:6837/714728190] conn(0x563c5e64ffd0 0x563c5e6523e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c5e5e1930 0x563c5e648470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e017fa700 0 client.16224 ms_handle_reset on v2:172.21.15.194:6836/714728190 2023-07-21T22:18:28.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 --2- 172.21.15.93:0/1742155918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c5e5e61b0 0x563c5e6489b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 >> 172.21.15.93:0/1742155918 conn(0x563c5e5d6ef0 msgr2=0x563c5e5e0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 shutdown_connections 2023-07-21T22:18:28.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 1 -- 172.21.15.93:0/1742155918 wait complete. 2023-07-21T22:18:28.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e2c178b80 20 client.16224 trim_cache size 0 max 16384 2023-07-21T22:18:28.071 INFO:teuthology.orchestra.run.smithi093.stdout:[ 2023-07-21T22:18:28.072 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.072 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.072 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 32824789, 2023-07-21T22:18:28.072 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628301, 2023-07-21T22:18:28.073 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS" 2023-07-21T22:18:28.073 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:28.073 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.073 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.073 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 566634001, 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628302, 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS" 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 720894098, 2023-07-21T22:18:28.074 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628354, 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL" 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 879774425, 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628351, 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB" 2023-07-21T22:18:28.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 2890001478, 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628355, 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL" 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:18:28.076 INFO:teuthology.orchestra.run.smithi093.stdout: "damage_type": "backtrace", 2023-07-21T22:18:28.077 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 3800212888, 2023-07-21T22:18:28.077 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1099511628353, 2023-07-21T22:18:28.077 INFO:teuthology.orchestra.run.smithi093.stdout: "path": "/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL" 2023-07-21T22:18:28.077 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:18:28.077 INFO:teuthology.orchestra.run.smithi093.stdout:] 2023-07-21T22:18:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 msgr2=0x7f7e040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 0x7f7e040a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005950 tx=0x7f7e1400b410 comp rx=0 tx=0).stop 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 msgr2=0x7f7e1819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7e080098e0 tx=0x7f7e0800f690 comp rx=0 tx=0).stop 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 shutdown_connections 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e18105c00 0x7f7e18195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e180fe200 0x7f7e1819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7e040a12c0 0x7f7e040a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 --2- 172.21.15.93:0/4153654092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e180feb90 0x7f7e1819bf50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 >> 172.21.15.93:0/4153654092 conn(0x7f7e18076200 msgr2=0x7f7e180fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 shutdown_connections 2023-07-21T22:18:28.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.003+0000 7f7e1fff3700 1 -- 172.21.15.93:0/4153654092 wait complete. 2023-07-21T22:18:28.080 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:18:28.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 -- 172.21.15.93:0/1368128760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c40fedc0 msgr2=0x7fa2c40ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 --2- 172.21.15.93:0/1368128760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c40fedc0 0x7fa2c40ff190 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009990 tx=0x7fa2b8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 -- 172.21.15.93:0/1368128760 shutdown_connections 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 --2- 172.21.15.93:0/1368128760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c4104400 0x7fa2c410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 --2- 172.21.15.93:0/1368128760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c40fedc0 0x7fa2c40ff190 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 --2- 172.21.15.93:0/1368128760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 -- 172.21.15.93:0/1368128760 >> 172.21.15.93:0/1368128760 conn(0x7fa2c4074120 msgr2=0x7fa2c4074520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 -- 172.21.15.93:0/1368128760 shutdown_connections 2023-07-21T22:18:28.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.158+0000 7fa2c999f700 1 -- 172.21.15.93:0/1368128760 wait complete. 2023-07-21T22:18:28.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 Processor -- start 2023-07-21T22:18:28.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 -- start start 2023-07-21T22:18:28.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c40fedc0 0x7fa2c4194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4194d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41a41f0 con 0x7fa2c4104400 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41a4330 con 0x7fa2c40ff760 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c999f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41a4470 con 0x7fa2c40fedc0 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46042/0 (socket says 172.21.15.93:46042) 2023-07-21T22:18:28.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 -- 172.21.15.93:0/2560729827 learned_addr learned my addr 172.21.15.93:0/2560729827 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c899d700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c40fedc0 0x7fa2c4194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 -- 172.21.15.93:0/2560729827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c40fedc0 msgr2=0x7fa2c4194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c40fedc0 0x7fa2c4194810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 -- 172.21.15.93:0/2560729827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 msgr2=0x7fa2c4194d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4194d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c919e700 1 -- 172.21.15.93:0/2560729827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80095f0 con 0x7fa2c4104400 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.159+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4194d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007ee0 tx=0x7fa2bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c1ffb700 1 -- 172.21.15.93:0/2560729827 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc017070 con 0x7fa2c4104400 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c41a46d0 con 0x7fa2c4104400 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c1ffb700 1 -- 172.21.15.93:0/2560729827 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2bc00f040 con 0x7fa2c4104400 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c1ffb700 1 -- 172.21.15.93:0/2560729827 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005740 con 0x7fa2c4104400 2023-07-21T22:18:28.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.160+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41a4b90 con 0x7fa2c4104400 2023-07-21T22:18:28.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.162+0000 7fa2c1ffb700 1 -- 172.21.15.93:0/2560729827 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa2bc00dc60 con 0x7fa2c4104400 2023-07-21T22:18:28.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.162+0000 7fa2c1ffb700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 0x7fa2b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.162+0000 7fa2c1ffb700 1 -- 172.21.15.93:0/2560729827 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa2bc0cc4b0 con 0x7fa2c4104400 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.162+0000 7fa2c899d700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 0x7fa2b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.163+0000 7fa2c899d700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 0x7fa2b40a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b5c0 tx=0x7fa2b8005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.163+0000 7fa2d6d86b80 1 Processor -- start 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.163+0000 7fa2d6d86b80 1 -- start start 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995a490 0x559a5995a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 0x559a59965700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a59860cb0 con 0x559a59965c40 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a598b3a40 con 0x559a5995ada0 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a598aced0 con 0x559a5995a490 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46050/0 (socket says 172.21.15.93:46050) 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 -- 172.21.15.93:0/2674551203 learned_addr learned my addr 172.21.15.93:0/2674551203 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c899d700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995a490 0x559a5995a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 0x559a59965700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 -- 172.21.15.93:0/2674551203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995a490 msgr2=0x559a5995a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995a490 0x559a5995a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 -- 172.21.15.93:0/2674551203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 msgr2=0x559a59965700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 0x559a59965700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 -- 172.21.15.93:0/2674551203 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a596dade0 con 0x559a59965c40 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 0x559a59965700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2c919e700 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00d7b0 tx=0x7fa2bc00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c4762a2744b17a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2aeffd700 1 -- 172.21.15.93:0/2674551203 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0b90d0 con 0x559a59965c40 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2aeffd700 1 -- 172.21.15.93:0/2674551203 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2bc09d5d0 con 0x559a59965c40 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.164+0000 7fa2aeffd700 1 -- 172.21.15.93:0/2674551203 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005ae0 con 0x559a59965c40 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/2674551203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 msgr2=0x559a59968020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00d7b0 tx=0x7fa2bc00df60 comp rx=0 tx=0).stop 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/2674551203 shutdown_connections 2023-07-21T22:18:28.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a59965c40 0x559a59968020 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995a490 0x559a5995a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/2674551203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995ada0 0x559a59965700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/2674551203 >> 172.21.15.93:0/2674551203 conn(0x559a59950ef0 msgr2=0x559a59956860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/2674551203 shutdown_connections 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/2674551203 wait complete. 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.165+0000 7fa2d6d86b80 1 -- start start 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 Processor -- start 2023-07-21T22:18:28.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a599656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a59963030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5984cac0 con 0x559a5995ada0 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5984c7c0 con 0x559a5995a490 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a59730920 con 0x559a59965c40 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2d6d86b80 10 client.0 _ll_register_callbacks cb 0x559a5984a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46054/0 (socket says 172.21.15.93:46054) 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2c3fff700 1 -- 172.21.15.93:0/1914383344 learned_addr learned my addr 172.21.15.93:0/1914383344 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2c919e700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a59963030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.166+0000 7fa2c899d700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a599656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 -- 172.21.15.93:0/1914383344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 msgr2=0x559a59963030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a59963030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 -- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 msgr2=0x559a599656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a599656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 -- 172.21.15.93:0/1914383344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41a4b90 con 0x559a5995ada0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40ffb30 tx=0x7fa2b000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b735ea578c834ce9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000f8f0 con 0x559a5995ada0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa2b000fa50 con 0x559a5995ada0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a598064f0 con 0x559a5995ada0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b000fc10 con 0x559a5995ada0 2023-07-21T22:18:28.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 10 client.16236 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559a596c3c60 con 0x559a5995ada0 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b0024020 con 0x559a5995ada0 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa2b0015340 con 0x559a5995ada0 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 10 client.16236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.167+0000 7fa2d6d86b80 20 client.16236 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a59975600 0x559a59977a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2d6d86b80 4 client.16236 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:28.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a598a5eb0 con 0x559a59975600 2023-07-21T22:18:28.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2c899d700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a59975600 0x559a59977a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2a3fff700 10 client.16236 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:18:28.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.168+0000 7fa2c899d700 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a59975600 0x559a59977a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7624bcdaf36cac9 server_cookie=c130126e7156f472 in_seq=0 out_seq=0 2023-07-21T22:18:28.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2a3fff700 1 -- 172.21.15.93:0/1914383344 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a598a5eb0 con 0x559a59975600 2023-07-21T22:18:28.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2a3fff700 10 client.16236 handle_command_reply: tid=0 2023-07-21T22:18:28.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 client.16236 shutdown 2023-07-21T22:18:28.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 msgr2=0x559a599628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40ffb30 tx=0x7fa2b000ea00 comp rx=0 tx=0).stop 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 shutdown_connections 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a59975600 0x559a59977a10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a59963030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a599656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/1914383344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a599628f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 >> 172.21.15.93:0/1914383344 conn(0x559a59950ef0 msgr2=0x559a59956360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.169+0000 7fa2a3fff700 0 client.16236 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:18:28.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.170+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 shutdown_connections 2023-07-21T22:18:28.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.170+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/1914383344 wait complete. 2023-07-21T22:18:28.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.170+0000 7fa2d6d86b80 20 client.16236 trim_cache size 0 max 16384 2023-07-21T22:18:28.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 Processor -- start 2023-07-21T22:18:28.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 -- start start 2023-07-21T22:18:28.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995ada0 0x559a59965210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 0x559a59962190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a59957470 con 0x559a5995a490 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a59957e80 con 0x559a59965c40 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.182+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5995aab0 con 0x559a5995ada0 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46060/0 (socket says 172.21.15.93:46060) 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 -- 172.21.15.93:0/292630269 learned_addr learned my addr 172.21.15.93:0/292630269 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c919e700 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 0x559a59962190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 -- 172.21.15.93:0/292630269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995ada0 msgr2=0x559a59965210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995ada0 0x559a59965210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 -- 172.21.15.93:0/292630269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 msgr2=0x559a59962190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 0x559a59962190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 -- 172.21.15.93:0/292630269 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a59701250 con 0x559a5995a490 2023-07-21T22:18:28.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c919e700 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 0x559a59962190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2c899d700 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098e0 tx=0x7fa2b8005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb19f61e3eb410 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2af7fe700 1 -- 172.21.15.93:0/292630269 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8033c50 con 0x559a5995a490 2023-07-21T22:18:28.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2af7fe700 1 -- 172.21.15.93:0/292630269 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b802b040 con 0x559a5995a490 2023-07-21T22:18:28.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.183+0000 7fa2af7fe700 1 -- 172.21.15.93:0/292630269 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8023960 con 0x559a5995a490 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/292630269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 msgr2=0x559a5995d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098e0 tx=0x7fa2b8005000 comp rx=0 tx=0).stop 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/292630269 shutdown_connections 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a59965c40 0x559a59962190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995a490 0x559a5995d3f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/292630269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a5995ada0 0x559a59965210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/292630269 >> 172.21.15.93:0/292630269 conn(0x559a59950ef0 msgr2=0x559a5995dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/292630269 shutdown_connections 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/292630269 wait complete. 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 -- start start 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 Processor -- start 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a5996aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.184+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a5996be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5996c530 con 0x559a5995ada0 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5996c6a0 con 0x559a5995a490 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a5996c860 con 0x559a59965c40 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 10 client.0 _ll_register_callbacks cb 0x7fa2b000cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46068/0 (socket says 172.21.15.93:46068) 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 -- 172.21.15.93:0/4109720286 learned_addr learned my addr 172.21.15.93:0/4109720286 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c899d700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a5996aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 -- 172.21.15.93:0/4109720286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 msgr2=0x559a5996be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a5996be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 -- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 msgr2=0x559a5996aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a5996aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 -- 172.21.15.93:0/4109720286 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c4068140 con 0x559a5995ada0 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c899d700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a5996aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:28.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2c3fff700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c380 tx=0x7fa2b0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f39d682dfa7dd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000f6f0 con 0x559a5995ada0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa2b0025080 con 0x559a5995ada0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2bc09d5d0 con 0x559a5995ada0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b000f850 con 0x559a5995ada0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 10 client.16242 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559a596d92b0 con 0x559a5995ada0 2023-07-21T22:18:28.238 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.185+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b0035020 con 0x559a5995ada0 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa2b00178a0 con 0x559a5995ada0 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 20 client.16242 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a599ca710 0x559a599ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 4 client.16242 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 --> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a59868050 con 0x559a599ca710 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2c899d700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a599ca710 0x559a599ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2a3fff700 10 client.16242 ms_handle_connect on v2:172.21.15.134:6838/3043386858 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.186+0000 7fa2c899d700 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a599ca710 0x559a599ccb20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e535dba8d0fc7574 server_cookie=b289268c4a61ce70 in_seq=0 out_seq=0 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2a3fff700 1 -- 172.21.15.93:0/4109720286 <== mds.1 v2:172.21.15.134:6838/3043386858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a59868050 con 0x559a599ca710 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2a3fff700 10 client.16242 handle_command_reply: tid=0 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 client.16242 shutdown 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 msgr2=0x559a5996af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.240 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c380 tx=0x7fa2b0005dc0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 shutdown_connections 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:6838/3043386858,v1:172.21.15.134:6839/3043386858] conn(0x559a599ca710 0x559a599ccb20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a59965c40 0x559a5996be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2a3fff700 0 client.16242 ms_handle_reset on v2:172.21.15.134:6838/3043386858 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559a5995a490 0x559a5996aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 --2- 172.21.15.93:0/4109720286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559a5995ada0 0x559a5996af80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 >> 172.21.15.93:0/4109720286 conn(0x559a59950ef0 msgr2=0x559a59962190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 shutdown_connections 2023-07-21T22:18:28.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 1 -- 172.21.15.93:0/4109720286 wait complete. 2023-07-21T22:18:28.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2d6d86b80 20 client.16242 trim_cache size 0 max 16384 2023-07-21T22:18:28.242 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:18:28.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 msgr2=0x7fa2b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 0x7fa2b40a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b5c0 tx=0x7fa2b8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 msgr2=0x7fa2c41a3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007ee0 tx=0x7fa2bc00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 shutdown_connections 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2b40a12c0 0x7fa2b40a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2c4104400 0x7fa2c41a3cb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c40fedc0 0x7fa2c4194810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 --2- 172.21.15.93:0/2560729827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c40ff760 0x7fa2c4194d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.187+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 >> 172.21.15.93:0/2560729827 conn(0x7fa2c4074120 msgr2=0x7fa2c4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.188+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 shutdown_connections 2023-07-21T22:18:28.244 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.188+0000 7fa2c999f700 1 -- 172.21.15.93:0/2560729827 wait complete. 2023-07-21T22:18:28.245 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:18:28.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 43264 3.85 MB/sec warmup 65 sec latency 1525.318 ms 2023-07-21T22:18:28.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 -- 172.21.15.93:0/1292786766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c0ff580 msgr2=0x7f2e3c0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 --2- 172.21.15.93:0/1292786766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c0ff950 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2e300099c0 tx=0x7f2e30009cd0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 -- 172.21.15.93:0/1292786766 shutdown_connections 2023-07-21T22:18:28.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 --2- 172.21.15.93:0/1292786766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 --2- 172.21.15.93:0/1292786766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c0ff950 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 --2- 172.21.15.93:0/1292786766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 0x7f2e3c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 -- 172.21.15.93:0/1292786766 >> 172.21.15.93:0/1292786766 conn(0x7f2e3c076200 msgr2=0x7f2e3c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 -- 172.21.15.93:0/1292786766 shutdown_connections 2023-07-21T22:18:28.372 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.323+0000 7f2e42c83700 1 -- 172.21.15.93:0/1292786766 wait complete. 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 Processor -- start 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 -- start start 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 0x7f2e3c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19ada0 con 0x7f2e3c103ce0 2023-07-21T22:18:28.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19af10 con 0x7f2e3c0fff20 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e42c83700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19b0d0 con 0x7f2e3c0ff580 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46082/0 (socket says 172.21.15.93:46082) 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 -- 172.21.15.93:0/2833117399 learned_addr learned my addr 172.21.15.93:0/2833117399 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 0x7f2e3c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 -- 172.21.15.93:0/2833117399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 msgr2=0x7f2e3c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 -- 172.21.15.93:0/2833117399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 msgr2=0x7f2e3c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 0x7f2e3c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 -- 172.21.15.93:0/2833117399 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300095f0 con 0x7f2e3c103ce0 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.324+0000 7f2e41220700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400ada0 tx=0x7f2e3400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.325+0000 7f2e39ffb700 1 -- 172.21.15.93:0/2833117399 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400f070 con 0x7f2e3c103ce0 2023-07-21T22:18:28.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.325+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c19b3b0 con 0x7f2e3c103ce0 2023-07-21T22:18:28.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.325+0000 7f2e39ffb700 1 -- 172.21.15.93:0/2833117399 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e34007b30 con 0x7f2e3c103ce0 2023-07-21T22:18:28.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.325+0000 7f2e39ffb700 1 -- 172.21.15.93:0/2833117399 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34005440 con 0x7f2e3c103ce0 2023-07-21T22:18:28.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.325+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c1a2bf0 con 0x7f2e3c103ce0 2023-07-21T22:18:28.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.327+0000 7f2e39ffb700 1 -- 172.21.15.93:0/2833117399 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2e34004ad0 con 0x7f2e3c103ce0 2023-07-21T22:18:28.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.328+0000 7f2e39ffb700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 0x7f2e2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.328+0000 7f2e39ffb700 1 -- 172.21.15.93:0/2833117399 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2e340cc330 con 0x7f2e3c103ce0 2023-07-21T22:18:28.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.328+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 0x7f2e2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.328+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 0x7f2e2c0a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fd0 tx=0x7f2e30005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 Processor -- start 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 -- start start 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ffd0cb0 con 0x55b2a00cb930 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a0023a40 con 0x55b2a00d45c0 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a001ced0 con 0x55b2a00d4ed0 2023-07-21T22:18:28.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46090/0 (socket says 172.21.15.93:46090) 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 -- 172.21.15.93:0/3806998695 learned_addr learned my addr 172.21.15.93:0/3806998695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 -- 172.21.15.93:0/3806998695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 msgr2=0x55b2a00cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 -- 172.21.15.93:0/3806998695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 msgr2=0x55b2a00d4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 -- 172.21.15.93:0/3806998695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29fe4ade0 con 0x55b2a00cb930 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e41220700 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e34006130 tx=0x7f2e34098ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4679590879f81b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.329+0000 7f2e26ffd700 1 -- 172.21.15.93:0/3806998695 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3409d4e0 con 0x55b2a00cb930 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e26ffd700 1 -- 172.21.15.93:0/3806998695 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e340a1870 con 0x55b2a00cb930 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e26ffd700 1 -- 172.21.15.93:0/3806998695 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34099710 con 0x55b2a00cb930 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3806998695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 msgr2=0x55b2a00cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e34006130 tx=0x7f2e34098ff0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3806998695 shutdown_connections 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00cbda0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3806998695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3806998695 >> 172.21.15.93:0/3806998695 conn(0x55b2a00c0ef0 msgr2=0x55b2a00c6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3806998695 shutdown_connections 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.330+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3806998695 wait complete. 2023-07-21T22:18:28.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 -- start start 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 Processor -- start 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00cb930 0x55b2a00dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ffbcac0 con 0x55b2a00d45c0 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ffbc7c0 con 0x55b2a00cb930 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29fea0920 con 0x55b2a00d4ed0 2023-07-21T22:18:28.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e4ee08b80 10 client.0 _ll_register_callbacks cb 0x55b29ffba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46094/0 (socket says 172.21.15.93:46094) 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 -- 172.21.15.93:0/3192671361 learned_addr learned my addr 172.21.15.93:0/3192671361 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e41220700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00cb930 0x55b2a00dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 -- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 msgr2=0x55b2a00d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 -- 172.21.15.93:0/3192671361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00cb930 msgr2=0x55b2a00dc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00cb930 0x55b2a00dc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e3bfff700 1 -- 172.21.15.93:0/3192671361 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c1a2bf0 con 0x55b2a00d45c0 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.331+0000 7f2e41220700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c072c00 tx=0x7f2e28010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141af1dd198e5728 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800e980 con 0x55b2a00d45c0 2023-07-21T22:18:28.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b29ff764f0 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2e28017070 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e2800eae0 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 10 client.16254 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b29fe33c60 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b29ff764f0 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2e28025460 con 0x55b2a00d45c0 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 10 client.16254 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:28.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 20 client.16254 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a00e9a20 0x55b2a00ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 4 client.16254 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:28.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.332+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a0015eb0 con 0x55b2a00e9a20 2023-07-21T22:18:28.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.333+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a00e9a20 0x55b2a00ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.333+0000 7f2e1bfff700 10 client.16254 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.333+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a00e9a20 0x55b2a00ebe30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f98b9074911bd1 server_cookie=edf91b18c2e324d1 in_seq=0 out_seq=0 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e1bfff700 1 -- 172.21.15.93:0/3192671361 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b2a0015eb0 con 0x55b2a00e9a20 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e1bfff700 10 client.16254 handle_command_reply: tid=0 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 client.16254 shutdown 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 msgr2=0x55b2a00d9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c072c00 tx=0x7f2e28010b20 comp rx=0 tx=0).stop 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 shutdown_connections 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a00e9a20 0x55b2a00ebe30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00cb930 0x55b2a00dc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e1bfff700 0 client.16254 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3192671361 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d45c0 0x55b2a00d9350 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 >> 172.21.15.93:0/3192671361 conn(0x55b2a00c0ef0 msgr2=0x55b2a00c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 shutdown_connections 2023-07-21T22:18:28.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3192671361 wait complete. 2023-07-21T22:18:28.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.334+0000 7f2e4ee08b80 20 client.16254 trim_cache size 0 max 16384 2023-07-21T22:18:28.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 Processor -- start 2023-07-21T22:18:28.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 -- start start 2023-07-21T22:18:28.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ff76520 con 0x55b2a00cb930 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a00da920 con 0x55b2a00d45c0 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a00e7020 con 0x55b2a00d4ed0 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46100/0 (socket says 172.21.15.93:46100) 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 -- 172.21.15.93:0/3989226429 learned_addr learned my addr 172.21.15.93:0/3989226429 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e41220700 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 -- 172.21.15.93:0/3989226429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 msgr2=0x55b2a00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 -- 172.21.15.93:0/3989226429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 msgr2=0x55b2a00dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 -- 172.21.15.93:0/3989226429 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29fe71250 con 0x55b2a00cb930 2023-07-21T22:18:28.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e41220700 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2e300099c0 tx=0x7f2e3001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f012c2a7ff0c04f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e277fe700 1 -- 172.21.15.93:0/3989226429 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000bc70 con 0x55b2a00cb930 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e277fe700 1 -- 172.21.15.93:0/3989226429 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e3001e040 con 0x55b2a00cb930 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.347+0000 7f2e277fe700 1 -- 172.21.15.93:0/3989226429 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30004d20 con 0x55b2a00cb930 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3989226429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 msgr2=0x55b2a00e0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2e300099c0 tx=0x7f2e3001a050 comp rx=0 tx=0).stop 2023-07-21T22:18:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3989226429 shutdown_connections 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00d4ed0 0x55b2a00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00cb930 0x55b2a00e0980 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/3989226429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3989226429 >> 172.21.15.93:0/3989226429 conn(0x55b2a00c0ef0 msgr2=0x55b2a00c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3989226429 shutdown_connections 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/3989226429 wait complete. 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 -- start start 2023-07-21T22:18:28.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.348+0000 7f2e4ee08b80 1 Processor -- start 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00cb930 0x55b2a00d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a00e0f90 con 0x55b2a00d4ed0 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a013d8e0 con 0x55b2a00d45c0 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a013daa0 con 0x55b2a00cb930 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 10 client.0 _ll_register_callbacks cb 0x7f2e2800cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:28.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46110/0 (socket says 172.21.15.93:46110) 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 -- 172.21.15.93:0/2370396076 learned_addr learned my addr 172.21.15.93:0/2370396076 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00cb930 0x55b2a00d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e3bfff700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 -- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00cb930 msgr2=0x55b2a00d7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00cb930 0x55b2a00d7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 -- 172.21.15.93:0/2370396076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 msgr2=0x55b2a00d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 -- 172.21.15.93:0/2370396076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c10c130 con 0x55b2a00d4ed0 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e41220700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2e340991b0 tx=0x7f2e340951f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded9676f2ac969cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3409d120 con 0x55b2a00d4ed0 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f2e3409d280 con 0x55b2a00d4ed0 2023-07-21T22:18:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.349+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e340a1870 con 0x55b2a00d4ed0 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e340989a0 con 0x55b2a00d4ed0 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 10 client.16260 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b29fe492b0 con 0x55b2a00d4ed0 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e340c9020 con 0x55b2a00d4ed0 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2e340b1030 con 0x55b2a00d4ed0 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 20 client.16260 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a013ee10 0x55b2a0141220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 4 client.16260 mds_command: new command op to 24484 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b29ffd8050 con 0x55b2a013ee10 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.350+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a013ee10 0x55b2a0141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e1bfff700 10 client.16260 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e40a1f700 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a013ee10 0x55b2a0141220 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca2f0bbc84c46f8 server_cookie=aca2743f8874d720 in_seq=0 out_seq=0 2023-07-21T22:18:28.406 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e1bfff700 1 -- 172.21.15.93:0/2370396076 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b29ffd8050 con 0x55b2a013ee10 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e1bfff700 10 client.16260 handle_command_reply: tid=0 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 client.16260 shutdown 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 msgr2=0x55b2a00d89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2e340991b0 tx=0x7f2e340951f0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 shutdown_connections 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b2a013ee10 0x55b2a0141220 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b2a00d4ed0 0x55b2a00d89d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2a00cb930 0x55b2a00d7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 --2- 172.21.15.93:0/2370396076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2a00d45c0 0x55b2a00d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 >> 172.21.15.93:0/2370396076 conn(0x55b2a00c0ef0 msgr2=0x55b2a00caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e1bfff700 0 client.16260 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 shutdown_connections 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 1 -- 172.21.15.93:0/2370396076 wait complete. 2023-07-21T22:18:28.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.351+0000 7f2e4ee08b80 20 client.16260 trim_cache size 0 max 16384 2023-07-21T22:18:28.409 INFO:teuthology.orchestra.run.smithi093.stdout:[] 2023-07-21T22:18:28.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 msgr2=0x7f2e2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 0x7f2e2c0a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fd0 tx=0x7f2e30005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 msgr2=0x7f2e3c19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400ada0 tx=0x7f2e3400c330 comp rx=0 tx=0).stop 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 shutdown_connections 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2e2c0a12c0 0x7f2e2c0a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3c103ce0 0x7f2e3c19a760 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e3c0ff580 0x7f2e3c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 --2- 172.21.15.93:0/2833117399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e3c0fff20 0x7f2e3c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 >> 172.21.15.93:0/2833117399 conn(0x7f2e3c076200 msgr2=0x7f2e3c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:28.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 shutdown_connections 2023-07-21T22:18:28.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:18:28.352+0000 7f2e42c83700 1 -- 172.21.15.93:0/2833117399 wait complete. 2023-07-21T22:18:28.412 ERROR:tasks.fwd_scrub:rank 3 damaged: [{'damage_type': 'backtrace', 'id': 951119240, 'ino': 1099511628318, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC2.CDR'}, {'damage_type': 'backtrace', 'id': 1126671368, 'ino': 1099511628311, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRBARS.CFG'}, {'damage_type': 'backtrace', 'id': 1975311948, 'ino': 1099511628310, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CDRROLS.CFG'}, {'damage_type': 'backtrace', 'id': 2065647689, 'ino': 1099511628315, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TEAPOT2.CPT'}, {'damage_type': 'backtrace', 'id': 2384460332, 'ino': 1099511628316, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/SIGN2.CPT'}, {'damage_type': 'backtrace', 'id': 2632043439, 'ino': 1099511628314, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/MAILBOX2.CPT'}, {'damage_type': 'backtrace', 'id': 2919677868, 'ino': 1099511628312, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ARTISTIC.ACL'}, {'damage_type': 'backtrace', 'id': 3276777170, 'ino': 1099511628320, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/CORELDRW.CDT'}, {'damage_type': 'backtrace', 'id': 3573899838, 'ino': 1099511628313, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/GRAPHIC1.CDR'}, {'damage_type': 'backtrace', 'id': 4234299463, 'ino': 1099511628319, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/ROSE.CPT'}, {'damage_type': 'backtrace', 'id': 4292424092, 'ino': 1099511628317, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/COREL/TRAIN.CDR'}] 2023-07-21T22:18:28.412 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 32824789, 'ino': 1099511628301, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/PCMAGCD.XLS'}, {'damage_type': 'backtrace', 'id': 566634001, 'ino': 1099511628302, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/EXCEL/SALES.XLS'}, {'damage_type': 'backtrace', 'id': 720894098, 'ino': 1099511628354, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/MEDIUM.FIL'}, {'damage_type': 'backtrace', 'id': 879774425, 'ino': 1099511628351, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/ACCESS/FASTENER.MDB'}, {'damage_type': 'backtrace', 'id': 2890001478, 'ino': 1099511628355, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/LARGE.FIL'}, {'damage_type': 'backtrace', 'id': 3800212888, 'ino': 1099511628353, 'path': '/volumes/_nogroup/sv_1/a9794218-501c-42de-ad2b-a96b2b0f2880/client.0/tmp/clients/client0/~dmtmp/SEED/SMALL.FIL'}] 2023-07-21T22:18:28.412 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-21T22:18:28.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:28 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1580031320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:28.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:28 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1580031320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:28.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:28 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1580031320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:29.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 43264 3.80 MB/sec warmup 66 sec latency 2525.396 ms 2023-07-21T22:18:29.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:29 smithi194 ceph-mon[98288]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2023-07-21T22:18:29.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:29 smithi134 ceph-mon[97873]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2023-07-21T22:18:29.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:29 smithi093 ceph-mon[96579]: pgmap v64: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.0 MiB/s wr, 155 op/s 2023-07-21T22:18:30.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 43818 3.77 MB/sec warmup 67 sec latency 3196.393 ms 2023-07-21T22:18:31.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 46152 3.94 MB/sec warmup 68 sec latency 41.492 ms 2023-07-21T22:18:31.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:31 smithi194 ceph-mon[98288]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 781 KiB/s wr, 102 op/s 2023-07-21T22:18:31.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:31 smithi134 ceph-mon[97873]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 781 KiB/s wr, 102 op/s 2023-07-21T22:18:31.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:31 smithi093 ceph-mon[96579]: pgmap v65: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 781 KiB/s wr, 102 op/s 2023-07-21T22:18:32.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 46814 3.91 MB/sec warmup 69 sec latency 372.889 ms 2023-07-21T22:18:32.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:32 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:32 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:32 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:33.310 INFO:tasks.workunit.client.0.smithi093.stdout: 1 46814 3.85 MB/sec warmup 70 sec latency 1373.015 ms 2023-07-21T22:18:33.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:33 smithi194 ceph-mon[98288]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2023-07-21T22:18:33.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:33.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:33.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:18:33.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:33 smithi134 ceph-mon[97873]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2023-07-21T22:18:33.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:18:33.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:33 smithi093 ceph-mon[96579]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2023-07-21T22:18:33.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:18:34.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 46814 3.80 MB/sec warmup 71 sec latency 2373.113 ms 2023-07-21T22:18:35.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 47354 3.77 MB/sec warmup 72 sec latency 3046.281 ms 2023-07-21T22:18:35.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:35 smithi194 ceph-mon[98288]: pgmap v67: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:35 smithi134 ceph-mon[97873]: pgmap v67: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:35 smithi093 ceph-mon[96579]: pgmap v67: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:36.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 49928 3.95 MB/sec warmup 73 sec latency 14.659 ms 2023-07-21T22:18:37.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50355 3.90 MB/sec warmup 74 sec latency 483.165 ms 2023-07-21T22:18:37.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:37 smithi194 ceph-mon[98288]: pgmap v68: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:37.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:37 smithi134 ceph-mon[97873]: pgmap v68: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:37.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:37 smithi093 ceph-mon[96579]: pgmap v68: 97 pgs: 97 active+clean; 40 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 930 KiB/s wr, 124 op/s 2023-07-21T22:18:38.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50355 3.85 MB/sec warmup 75 sec latency 1483.220 ms 2023-07-21T22:18:39.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50355 3.80 MB/sec warmup 76 sec latency 2483.344 ms 2023-07-21T22:18:39.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:39 smithi194 ceph-mon[98288]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-07-21T22:18:39.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:39 smithi134 ceph-mon[97873]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-07-21T22:18:39.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:39 smithi093 ceph-mon[96579]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2023-07-21T22:18:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 -- 172.21.15.93:0/2116200192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd954103ed0 msgr2=0x7fd95410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/2116200192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd954103ed0 0x7fd95410baa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00b0d0 tx=0x7fd94c00b3e0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 -- 172.21.15.93:0/2116200192 shutdown_connections 2023-07-21T22:18:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/2116200192 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd954103ed0 0x7fd95410baa0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/2116200192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9540ff3b0 0x7fd9540ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/2116200192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 0x7fd9541037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 -- 172.21.15.93:0/2116200192 >> 172.21.15.93:0/2116200192 conn(0x7fd954075f90 msgr2=0x7fd954076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 -- 172.21.15.93:0/2116200192 shutdown_connections 2023-07-21T22:18:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.220+0000 7fd95a2b8700 1 -- 172.21.15.93:0/2116200192 wait complete. 2023-07-21T22:18:40.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 Processor -- start 2023-07-21T22:18:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 -- start start 2023-07-21T22:18:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 0x7fd9541974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 0x7fd9541a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a23e0 con 0x7fd9540ff3b0 2023-07-21T22:18:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a2550 con 0x7fd954103ed0 2023-07-21T22:18:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd95a2b8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9541a2710 con 0x7fd9540ffd50 2023-07-21T22:18:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46120/0 (socket says 172.21.15.93:46120) 2023-07-21T22:18:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 -- 172.21.15.93:0/1226795879 learned_addr learned my addr 172.21.15.93:0/1226795879 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd958855700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 0x7fd9541a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 -- 172.21.15.93:0/1226795879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 msgr2=0x7fd9541974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd9537fe700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 0x7fd9541974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 0x7fd9541974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 -- 172.21.15.93:0/1226795879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 msgr2=0x7fd9541a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.221+0000 7fd953fff700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 0x7fd9541a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd953fff700 1 -- 172.21.15.93:0/1226795879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009d00 con 0x7fd9540ff3b0 2023-07-21T22:18:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd958855700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 0x7fd9541a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd953fff700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9a0 tx=0x7fd93c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd9517fa700 1 -- 172.21.15.93:0/1226795879 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c009790 con 0x7fd9540ff3b0 2023-07-21T22:18:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd9517fa700 1 -- 172.21.15.93:0/1226795879 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c00cbd0 con 0x7fd9540ff3b0 2023-07-21T22:18:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9541a29f0 con 0x7fd9540ff3b0 2023-07-21T22:18:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd9517fa700 1 -- 172.21.15.93:0/1226795879 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c018680 con 0x7fd9540ff3b0 2023-07-21T22:18:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.222+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9541a2f00 con 0x7fd9540ff3b0 2023-07-21T22:18:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.224+0000 7fd9517fa700 1 -- 172.21.15.93:0/1226795879 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd93c00cd40 con 0x7fd9540ff3b0 2023-07-21T22:18:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.225+0000 7fd9517fa700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 0x7fd9400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.225+0000 7fd9537fe700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 0x7fd9400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.225+0000 7fd9517fa700 1 -- 172.21.15.93:0/1226795879 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd93c014070 con 0x7fd9540ff3b0 2023-07-21T22:18:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.225+0000 7fd9537fe700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 0x7fd9400a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9540feaf0 tx=0x7fd944009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 Processor -- start 2023-07-21T22:18:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 -- start start 2023-07-21T22:18:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd37c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37c620 0x55d1cd376350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd277c50 con 0x55d1cd37ca70 2023-07-21T22:18:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd2ca9e0 con 0x55d1cd37c620 2023-07-21T22:18:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd96643db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd2c3e70 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46132/0 (socket says 172.21.15.93:46132) 2023-07-21T22:18:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 -- 172.21.15.93:0/2970224890 learned_addr learned my addr 172.21.15.93:0/2970224890 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 -- 172.21.15.93:0/2970224890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 msgr2=0x55d1cd37c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd953fff700 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd37c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd37c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37c620 0x55d1cd376350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 -- 172.21.15.93:0/2970224890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37c620 msgr2=0x55d1cd376350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37c620 0x55d1cd376350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 -- 172.21.15.93:0/2970224890 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cd0f1d80 con 0x55d1cd37ca70 2023-07-21T22:18:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.226+0000 7fd958855700 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009c90 tx=0x7fd94c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a530f077cb58819 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd94a7fc700 1 -- 172.21.15.93:0/2970224890 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00e030 con 0x55d1cd37ca70 2023-07-21T22:18:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd94a7fc700 1 -- 172.21.15.93:0/2970224890 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94c00bca0 con 0x55d1cd37ca70 2023-07-21T22:18:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd94a7fc700 1 -- 172.21.15.93:0/2970224890 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c005240 con 0x55d1cd37ca70 2023-07-21T22:18:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 -- 172.21.15.93:0/2970224890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 msgr2=0x55d1cd3768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009c90 tx=0x7fd94c0092e0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 -- 172.21.15.93:0/2970224890 shutdown_connections 2023-07-21T22:18:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37ca70 0x55d1cd3768b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd37c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 --2- 172.21.15.93:0/2970224890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37c620 0x55d1cd376350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 -- 172.21.15.93:0/2970224890 >> 172.21.15.93:0/2970224890 conn(0x55d1cd367ae0 msgr2=0x55d1cd36d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 -- 172.21.15.93:0/2970224890 shutdown_connections 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.227+0000 7fd96643db80 1 -- 172.21.15.93:0/2970224890 wait complete. 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 -- start start 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 Processor -- start 2023-07-21T22:18:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 0x55d1cd37fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd380150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd263a60 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd263760 con 0x55d1cd37ca70 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd1478c0 con 0x55d1cd37c620 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd96643db80 10 client.0 _ll_register_callbacks cb 0x55d1cd261700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46134/0 (socket says 172.21.15.93:46134) 2023-07-21T22:18:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.228+0000 7fd953fff700 1 -- 172.21.15.93:0/2003373030 learned_addr learned my addr 172.21.15.93:0/2003373030 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:40.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd958855700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd380150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 0x55d1cd37fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 -- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 msgr2=0x55d1cd37fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 0x55d1cd37fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 -- 172.21.15.93:0/2003373030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 msgr2=0x55d1cd380150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd380150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 -- 172.21.15.93:0/2003373030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95410c2e0 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 0x55d1cd37fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd953fff700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd93c096610 tx=0x7fd93c09a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770016b771ab4db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c099600 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd93c099760 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1cd21d490 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93c0a20d0 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd96643db80 10 client.16272 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d1cd0dac00 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1cd21d490 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd93c095f00 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd96643db80 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.229+0000 7fd96643db80 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd96643db80 20 client.16272 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd392290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd96643db80 4 client.16272 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1cd2bce50 con 0x55d1cd38fe80 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd392290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd9337fe700 10 client.16272 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:40.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.230+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd392290 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa976cf5ec8b344a server_cookie=6a83f7c951f3309a in_seq=0 out_seq=0 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd9337fe700 1 -- 172.21.15.93:0/2003373030 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d1cd2bce50 con 0x55d1cd38fe80 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd9337fe700 10 client.16272 handle_command_reply: tid=0 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 client.16272 shutdown 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 msgr2=0x55d1cd3828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd93c096610 tx=0x7fd93c09a210 comp rx=0 tx=0).stop 2023-07-21T22:18:40.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 shutdown_connections 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd392290 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd380150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd9337fe700 0 client.16272 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3828f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 --2- 172.21.15.93:0/2003373030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37c620 0x55d1cd37fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 >> 172.21.15.93:0/2003373030 conn(0x55d1cd367ae0 msgr2=0x55d1cd370f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 shutdown_connections 2023-07-21T22:18:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 1 -- 172.21.15.93:0/2003373030 wait complete. 2023-07-21T22:18:40.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.231+0000 7fd96643db80 20 client.16272 trim_cache size 0 max 16384 2023-07-21T22:18:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 Processor -- start 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 -- start start 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd386ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37ca70 0x55d1cd382490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd36dd00 con 0x55d1cd37c620 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd377570 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd96643db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd381150 con 0x55d1cd37ca70 2023-07-21T22:18:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46144/0 (socket says 172.21.15.93:46144) 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 -- 172.21.15.93:0/3217642236 learned_addr learned my addr 172.21.15.93:0/3217642236 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd953fff700 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd386ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd958855700 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37ca70 0x55d1cd382490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 -- 172.21.15.93:0/3217642236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37ca70 msgr2=0x55d1cd382490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37ca70 0x55d1cd382490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 -- 172.21.15.93:0/3217642236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd1ce6b0 msgr2=0x55d1cd386ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd386ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.248+0000 7fd9537fe700 1 -- 172.21.15.93:0/3217642236 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cd1181f0 con 0x55d1cd37c620 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd9537fe700 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd954072c00 tx=0x7fd944015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdedab6805a74d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd94affd700 1 -- 172.21.15.93:0/3217642236 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944023040 con 0x55d1cd37c620 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd94affd700 1 -- 172.21.15.93:0/3217642236 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd944026330 con 0x55d1cd37c620 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd94affd700 1 -- 172.21.15.93:0/3217642236 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94401e660 con 0x55d1cd37c620 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 -- 172.21.15.93:0/3217642236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 msgr2=0x55d1cd37d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd954072c00 tx=0x7fd944015000 comp rx=0 tx=0).stop 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 -- 172.21.15.93:0/3217642236 shutdown_connections 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd37ca70 0x55d1cd382490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd386ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 --2- 172.21.15.93:0/3217642236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd37d5c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.249+0000 7fd96643db80 1 -- 172.21.15.93:0/3217642236 >> 172.21.15.93:0/3217642236 conn(0x55d1cd367ae0 msgr2=0x55d1cd37ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.250+0000 7fd96643db80 1 -- 172.21.15.93:0/3217642236 shutdown_connections 2023-07-21T22:18:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.250+0000 7fd96643db80 1 -- 172.21.15.93:0/3217642236 wait complete. 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.250+0000 7fd96643db80 1 -- start start 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.250+0000 7fd96643db80 1 Processor -- start 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd378080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd379800 con 0x55d1cd37c620 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd3785c0 con 0x55d1cd37ca70 2023-07-21T22:18:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cd378780 con 0x55d1cd1ce6b0 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 10 client.0 _ll_register_callbacks cb 0x7fd93c099cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46150/0 (socket says 172.21.15.93:46150) 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 -- 172.21.15.93:0/2473664704 learned_addr learned my addr 172.21.15.93:0/2473664704 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd958855700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd378080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd953fff700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 -- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 msgr2=0x55d1cd3791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 -- 172.21.15.93:0/2473664704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 msgr2=0x55d1cd378080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd378080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 -- 172.21.15.93:0/2473664704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94400f990 con 0x55d1cd37c620 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd953fff700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9537fe700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd944012ed0 tx=0x7fd944012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cd6e80a2ef103a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9337fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944011910 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd944005040 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.251+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd94c00bca0 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd944005200 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 10 client.16278 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d1cd0f0250 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94c00bca0 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd944026050 con 0x55d1cd37c620 2023-07-21T22:18:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 10 client.16278 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 20 client.16278 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd3e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 4 client.16278 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d1cd27eff0 con 0x55d1cd38fe80 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd953fff700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd3e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd9337fe700 10 client.16278 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.252+0000 7fd953fff700 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd3e7100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b871c7878b5bc5 server_cookie=774e17d772c7ed6a in_seq=0 out_seq=0 2023-07-21T22:18:40.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50411 3.75 MB/sec warmup 77 sec latency 3153.344 ms 2023-07-21T22:18:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.282+0000 7fd9337fe700 1 -- 172.21.15.93:0/2473664704 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d1cd27eff0 con 0x55d1cd38fe80 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.282+0000 7fd9337fe700 10 client.16278 handle_command_reply: tid=0 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 client.16278 shutdown 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 msgr2=0x55d1cd377b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd944012ed0 tx=0x7fd944012f00 comp rx=0 tx=0).stop 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 shutdown_connections 2023-07-21T22:18:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d1cd38fe80 0x55d1cd3e7100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1cd37ca70 0x55d1cd378080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd9337fe700 0 client.16278 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1cd1ce6b0 0x55d1cd3791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 --2- 172.21.15.93:0/2473664704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1cd37c620 0x55d1cd377b40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 >> 172.21.15.93:0/2473664704 conn(0x55d1cd367ae0 msgr2=0x55d1cd370f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 shutdown_connections 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 1 -- 172.21.15.93:0/2473664704 wait complete. 2023-07-21T22:18:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd96643db80 20 client.16278 trim_cache size 0 max 16384 2023-07-21T22:18:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:18:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:18:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:18:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:18:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 msgr2=0x7fd9400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 0x7fd9400a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9540feaf0 tx=0x7fd944009260 comp rx=0 tx=0).stop 2023-07-21T22:18:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 msgr2=0x7fd954196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.283+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e9a0 tx=0x7fd93c00ed60 comp rx=0 tx=0).stop 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 shutdown_connections 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd954103ed0 0x7fd9541a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd9400a12c0 0x7fd9400a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9540ff3b0 0x7fd954196f70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 --2- 172.21.15.93:0/1226795879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9540ffd50 0x7fd9541974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 >> 172.21.15.93:0/1226795879 conn(0x7fd954075f90 msgr2=0x7fd9540fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 shutdown_connections 2023-07-21T22:18:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:40.284+0000 7fd95a2b8700 1 -- 172.21.15.93:0/1226795879 wait complete. 2023-07-21T22:18:41.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50411 3.70 MB/sec warmup 78 sec latency 1290.881 ms 2023-07-21T22:18:41.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:41 smithi194 ceph-mon[98288]: pgmap v70: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-21T22:18:41.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:41 smithi134 ceph-mon[97873]: pgmap v70: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-21T22:18:41.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:41 smithi093 ceph-mon[96579]: pgmap v70: 97 pgs: 97 active+clean; 41 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2023-07-21T22:18:42.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50411 3.66 MB/sec warmup 79 sec latency 2290.961 ms 2023-07-21T22:18:43.311 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50411 3.61 MB/sec warmup 80 sec latency 3291.034 ms 2023-07-21T22:18:43.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:43 smithi194 ceph-mon[98288]: pgmap v71: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2023-07-21T22:18:43.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:43 smithi134 ceph-mon[97873]: pgmap v71: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2023-07-21T22:18:43.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:43 smithi093 ceph-mon[96579]: pgmap v71: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2023-07-21T22:18:44.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 50411 3.57 MB/sec warmup 81 sec latency 4291.129 ms 2023-07-21T22:18:45.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 51115 3.58 MB/sec warmup 82 sec latency 4959.828 ms 2023-07-21T22:18:45.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:45 smithi194 ceph-mon[98288]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:45.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:45 smithi134 ceph-mon[97873]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:45.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:45 smithi093 ceph-mon[96579]: pgmap v72: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:46.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 53556 3.70 MB/sec warmup 83 sec latency 11.004 ms 2023-07-21T22:18:47.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 53959 3.67 MB/sec warmup 84 sec latency 597.092 ms 2023-07-21T22:18:47.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:47 smithi194 ceph-mon[98288]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:47.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:47 smithi134 ceph-mon[97873]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:47.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:47 smithi093 ceph-mon[96579]: pgmap v73: 97 pgs: 97 active+clean; 35 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 558 KiB/s wr, 81 op/s 2023-07-21T22:18:48.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 53959 3.62 MB/sec warmup 85 sec latency 1597.240 ms 2023-07-21T22:18:49.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 53959 3.58 MB/sec warmup 86 sec latency 2597.334 ms 2023-07-21T22:18:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:49 smithi194 ceph-mon[98288]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1014 KiB/s wr, 125 op/s 2023-07-21T22:18:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:49 smithi134 ceph-mon[97873]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1014 KiB/s wr, 125 op/s 2023-07-21T22:18:49.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:49 smithi093 ceph-mon[96579]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1014 KiB/s wr, 125 op/s 2023-07-21T22:18:50.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 54520 3.56 MB/sec warmup 87 sec latency 3266.441 ms 2023-07-21T22:18:51.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 57076 3.71 MB/sec warmup 88 sec latency 13.899 ms 2023-07-21T22:18:51.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:51 smithi194 ceph-mon[98288]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 528 KiB/s wr, 71 op/s 2023-07-21T22:18:51.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:51 smithi134 ceph-mon[97873]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 528 KiB/s wr, 71 op/s 2023-07-21T22:18:51.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:51 smithi093 ceph-mon[96579]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 528 KiB/s wr, 71 op/s 2023-07-21T22:18:52.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 57571 3.68 MB/sec warmup 89 sec latency 509.726 ms 2023-07-21T22:18:53.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 57571 3.64 MB/sec warmup 90 sec latency 1509.823 ms 2023-07-21T22:18:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 -- 172.21.15.93:0/4269697550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0fed20 msgr2=0x7fdb0c0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 --2- 172.21.15.93:0/4269697550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c0ff190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009990 tx=0x7fdafc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 -- 172.21.15.93:0/4269697550 shutdown_connections 2023-07-21T22:18:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 --2- 172.21.15.93:0/4269697550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 --2- 172.21.15.93:0/4269697550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 0x7fdb0c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 --2- 172.21.15.93:0/4269697550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c0ff190 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 -- 172.21.15.93:0/4269697550 >> 172.21.15.93:0/4269697550 conn(0x7fdb0c075270 msgr2=0x7fdb0c075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 -- 172.21.15.93:0/4269697550 shutdown_connections 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 -- 172.21.15.93:0/4269697550 wait complete. 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.400+0000 7fdb11a12700 1 Processor -- start 2023-07-21T22:18:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 -- start start 2023-07-21T22:18:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 0x7fdb0c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1a20c0 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1a2230 con 0x7fdb0c104370 2023-07-21T22:18:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb11a12700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1a23f0 con 0x7fdb0c0fed20 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46164/0 (socket says 172.21.15.93:46164) 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3089300506 learned_addr learned my addr 172.21.15.93:0/3089300506 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0affd700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 0x7fdb0c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3089300506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 msgr2=0x7fdb0c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3089300506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 msgr2=0x7fdb0c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 0x7fdb0c1a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3089300506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc0095f0 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0affd700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.401+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000c00 tx=0x7fdafc00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.402+0000 7fdb10a10700 1 -- 172.21.15.93:0/3089300506 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc01d070 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.402+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c1a2670 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.402+0000 7fdb10a10700 1 -- 172.21.15.93:0/3089300506 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdafc00fc20 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.402+0000 7fdb10a10700 1 -- 172.21.15.93:0/3089300506 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc017650 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.402+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1a2b20 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.404+0000 7fdb10a10700 1 -- 172.21.15.93:0/3089300506 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdafc021410 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.404+0000 7fdb10a10700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 0x7fdaf40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.405+0000 7fdb0affd700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 0x7fdaf40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.405+0000 7fdb10a10700 1 -- 172.21.15.93:0/3089300506 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdafc0ce1f0 con 0x7fdb0c0ff6d0 2023-07-21T22:18:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.405+0000 7fdb0affd700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 0x7fdaf40a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bb90 tx=0x7fdb00005d80 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.405+0000 7fdb1db97b80 1 Processor -- start 2023-07-21T22:18:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.405+0000 7fdb1db97b80 1 -- start start 2023-07-21T22:18:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bdfc8c50 con 0x55a6be0c9210 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be01b9e0 con 0x55a6be0c9660 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be014e70 con 0x55a6bdf1f6b0 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46172/0 (socket says 172.21.15.93:46172) 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3241222398 learned_addr learned my addr 172.21.15.93:0/3241222398 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0affd700 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3241222398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 msgr2=0x55a6be0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3241222398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 msgr2=0x55a6be0c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/3241222398 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bde42d80 con 0x55a6be0c9210 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdafc097870 tx=0x7fdafc00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc059fb7ba60702 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdaf1ffb700 1 -- 172.21.15.93:0/3241222398 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc09f3a0 con 0x55a6be0c9210 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdaf1ffb700 1 -- 172.21.15.93:0/3241222398 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdafc0a3730 con 0x55a6be0c9210 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.406+0000 7fdaf1ffb700 1 -- 172.21.15.93:0/3241222398 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc09b7f0 con 0x55a6be0c9210 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- 172.21.15.93:0/3241222398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 msgr2=0x55a6be0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdafc097870 tx=0x7fdafc00b680 comp rx=0 tx=0).stop 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- 172.21.15.93:0/3241222398 shutdown_connections 2023-07-21T22:18:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/3241222398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0c2f40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- 172.21.15.93:0/3241222398 >> 172.21.15.93:0/3241222398 conn(0x55a6be0b8ae0 msgr2=0x55a6be0be0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- 172.21.15.93:0/3241222398 shutdown_connections 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- 172.21.15.93:0/3241222398 wait complete. 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.407+0000 7fdb1db97b80 1 -- start start 2023-07-21T22:18:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 Processor -- start 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bdfb4a60 con 0x55a6be0c9210 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bdfb4760 con 0x55a6be0c9660 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bde988c0 con 0x55a6bdf1f6b0 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb1db97b80 10 client.0 _ll_register_callbacks cb 0x55a6bdfb2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46178/0 (socket says 172.21.15.93:46178) 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/2795391596 learned_addr learned my addr 172.21.15.93:0/2795391596 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0affd700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 msgr2=0x55a6be0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/2795391596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 msgr2=0x55a6be0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0a7fc700 1 -- 172.21.15.93:0/2795391596 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1a2b20 con 0x55a6be0c9210 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.408+0000 7fdb0affd700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdafc09b070 tx=0x7fdafc09b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca41d07924b78dee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc09a860 con 0x55a6be0c9210 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdafc09a9c0 con 0x55a6be0c9210 2023-07-21T22:18:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6bdf6e490 con 0x55a6be0c9210 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdafc09b5d0 con 0x55a6be0c9210 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 10 client.16290 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a6bde2bc00 con 0x55a6be0c9210 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6bdf6e490 con 0x55a6be0c9210 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fdafc084b70 con 0x55a6be0c9210 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 20 client.16290 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be0dc300 0x55a6be0de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 4 client.16290 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.409+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6be00de50 con 0x55a6be0dc300 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.410+0000 7fdb0affd700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be0dc300 0x55a6be0de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.410+0000 7fdaeaffd700 10 client.16290 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.410+0000 7fdb0affd700 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be0dc300 0x55a6be0de6e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54c364fe7d56a48 server_cookie=8034746b66beb6fa in_seq=0 out_seq=0 2023-07-21T22:18:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdaeaffd700 1 -- 172.21.15.93:0/2795391596 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6be00de50 con 0x55a6be0dc300 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdaeaffd700 10 client.16290 handle_command_reply: tid=0 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 client.16290 shutdown 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 msgr2=0x55a6be0d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdafc09b070 tx=0x7fdafc09b4f0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 shutdown_connections 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be0dc300 0x55a6be0de6e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9660 0x55a6be0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdaeaffd700 0 client.16290 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2795391596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9210 0x55a6be0d2830 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 >> 172.21.15.93:0/2795391596 conn(0x55a6be0b8ae0 msgr2=0x55a6be0bd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 shutdown_connections 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2795391596 wait complete. 2023-07-21T22:18:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.411+0000 7fdb1db97b80 20 client.16290 trim_cache size 0 max 16384 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 Processor -- start 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 -- start start 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9210 0x55a6be0bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bdf6e490 con 0x55a6be0c9660 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be0becb0 con 0x55a6be0c9210 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be0bac00 con 0x55a6bdf1f6b0 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46188/0 (socket says 172.21.15.93:46188) 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/2625917939 learned_addr learned my addr 172.21.15.93:0/2625917939 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0affd700 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/2625917939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 msgr2=0x55a6be0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/2625917939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9210 msgr2=0x55a6be0bdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9210 0x55a6be0bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/2625917939 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6be0e3310 con 0x55a6be0c9660 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.426+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb040094d0 tx=0x7fdb0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19abe95e6455143 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdaf27fc700 1 -- 172.21.15.93:0/2625917939 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040157d0 con 0x55a6be0c9660 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdaf27fc700 1 -- 172.21.15.93:0/2625917939 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb0400f030 con 0x55a6be0c9660 2023-07-21T22:18:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdaf27fc700 1 -- 172.21.15.93:0/2625917939 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04016db0 con 0x55a6be0c9660 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2625917939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 msgr2=0x55a6be0be0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb040094d0 tx=0x7fdb0400b040 comp rx=0 tx=0).stop 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2625917939 shutdown_connections 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0be0a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/2625917939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6be0c9210 0x55a6be0bdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2625917939 >> 172.21.15.93:0/2625917939 conn(0x55a6be0b8ae0 msgr2=0x55a6be0c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2625917939 shutdown_connections 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- 172.21.15.93:0/2625917939 wait complete. 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.427+0000 7fdb1db97b80 1 -- start start 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 Processor -- start 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6be0c9210 0x55a6be0d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be0d1140 con 0x55a6be0c9660 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be0d12b0 con 0x55a6bdf1f6b0 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6be0d1470 con 0x55a6be0c9210 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb1db97b80 10 client.0 _ll_register_callbacks cb 0x7fdb00015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46194/0 (socket says 172.21.15.93:46194) 2023-07-21T22:18:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/94820139 learned_addr learned my addr 172.21.15.93:0/94820139 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0a7fc700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6be0c9210 0x55a6be0d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0affd700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6be0c9210 msgr2=0x55a6be0d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6be0c9210 0x55a6be0d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/94820139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6bdf1f6b0 msgr2=0x55a6be0d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.428+0000 7fdb0b7fe700 1 -- 172.21.15.93:0/94820139 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb040091b0 con 0x55a6be0c9660 2023-07-21T22:18:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb0b7fe700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009960 tx=0x7fdb04016690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643cd5a41aea5c78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04017f10 con 0x55a6be0c9660 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fdb04003a00 con 0x55a6be0c9660 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdafc0a3730 con 0x55a6be0c9660 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb04003bc0 con 0x55a6be0c9660 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 10 client.16296 fetch_fsmap learned FSMap version 32 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a6bde691f0 con 0x55a6be0c9660 2023-07-21T22:18:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdafc0a3730 con 0x55a6be0c9660 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fdb04014040 con 0x55a6be0c9660 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 20 client.16296 populate_metadata read hostname 'smithi093' 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be130b10 0x55a6be132f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 4 client.16296 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:18:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.429+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a6bdfcfff0 con 0x55a6be130b10 2023-07-21T22:18:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.430+0000 7fdb0affd700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be130b10 0x55a6be132f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.430+0000 7fdaeaffd700 10 client.16296 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.430+0000 7fdb0affd700 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be130b10 0x55a6be132f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c123cdb526a06bec server_cookie=b68dbf16cdc4ca18 in_seq=0 out_seq=0 2023-07-21T22:18:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdaeaffd700 1 -- 172.21.15.93:0/94820139 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a6bdfcfff0 con 0x55a6be130b10 2023-07-21T22:18:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdaeaffd700 10 client.16296 handle_command_reply: tid=0 2023-07-21T22:18:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 client.16296 shutdown 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 msgr2=0x55a6be0d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009960 tx=0x7fdb04016690 comp rx=0 tx=0).stop 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 shutdown_connections 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a6be130b10 0x55a6be132f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a6be0c9660 0x55a6be0d0c00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdaeaffd700 0 client.16296 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.461+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6bdf1f6b0 0x55a6be0d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb1db97b80 1 --2- 172.21.15.93:0/94820139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a6be0c9210 0x55a6be0d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 >> 172.21.15.93:0/94820139 conn(0x55a6be0b8ae0 msgr2=0x55a6be0bcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 shutdown_connections 2023-07-21T22:18:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb1db97b80 1 -- 172.21.15.93:0/94820139 wait complete. 2023-07-21T22:18:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb1db97b80 20 client.16296 trim_cache size 0 max 16384 2023-07-21T22:18:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:18:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:18:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:18:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 msgr2=0x7fdaf40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 0x7fdaf40a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bb90 tx=0x7fdb00005d80 comp rx=0 tx=0).stop 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 msgr2=0x7fdb0c197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000c00 tx=0x7fdafc00f510 comp rx=0 tx=0).stop 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 shutdown_connections 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb0c104370 0x7fdb0c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb0c0fed20 0x7fdb0c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fdaf40a12c0 0x7fdaf40a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 --2- 172.21.15.93:0/3089300506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb0c0ff6d0 0x7fdb0c197190 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 >> 172.21.15.93:0/3089300506 conn(0x7fdb0c075270 msgr2=0x7fdb0c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 shutdown_connections 2023-07-21T22:18:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:18:53.462+0000 7fdb11a12700 1 -- 172.21.15.93:0/3089300506 wait complete. 2023-07-21T22:18:53.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:53 smithi194 ceph-mon[98288]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2023-07-21T22:18:53.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:53 smithi134 ceph-mon[97873]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2023-07-21T22:18:53.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:53 smithi093 ceph-mon[96579]: pgmap v76: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2023-07-21T22:18:54.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 57571 3.60 MB/sec warmup 91 sec latency 2509.911 ms 2023-07-21T22:18:55.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 58299 3.61 MB/sec warmup 92 sec latency 3177.481 ms 2023-07-21T22:18:55.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:55 smithi194 ceph-mon[98288]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:55.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:55 smithi134 ceph-mon[97873]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:55.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:55 smithi093 ceph-mon[96579]: pgmap v77: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:56.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 60653 3.72 MB/sec warmup 93 sec latency 10.754 ms 2023-07-21T22:18:57.312 INFO:tasks.workunit.client.0.smithi093.stdout: 1 61119 3.69 MB/sec warmup 94 sec latency 537.575 ms 2023-07-21T22:18:57.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:57 smithi194 ceph-mon[98288]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:57.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:57 smithi134 ceph-mon[97873]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:57.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:57 smithi093 ceph-mon[96579]: pgmap v78: 97 pgs: 97 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1014 KiB/s wr, 98 op/s 2023-07-21T22:18:58.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 61119 3.65 MB/sec warmup 95 sec latency 1537.696 ms 2023-07-21T22:18:59.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 61119 3.61 MB/sec warmup 96 sec latency 2537.744 ms 2023-07-21T22:18:59.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:18:59 smithi194 ceph-mon[98288]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2023-07-21T22:18:59.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:18:59 smithi134 ceph-mon[97873]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2023-07-21T22:18:59.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:18:59 smithi093 ceph-mon[96579]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2023-07-21T22:19:00.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 61835 3.62 MB/sec warmup 97 sec latency 3208.061 ms 2023-07-21T22:19:01.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 63877 3.71 MB/sec warmup 98 sec latency 21.879 ms 2023-07-21T22:19:01.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:01 smithi194 ceph-mon[98288]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-07-21T22:19:01.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:01 smithi134 ceph-mon[97873]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-07-21T22:19:01.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:01 smithi093 ceph-mon[96579]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2023-07-21T22:19:02.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 64592 3.69 MB/sec warmup 99 sec latency 297.863 ms 2023-07-21T22:19:03.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 64592 3.65 MB/sec warmup 100 sec latency 1297.929 ms 2023-07-21T22:19:03.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:03 smithi194 ceph-mon[98288]: pgmap v81: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T22:19:03.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:03 smithi134 ceph-mon[97873]: pgmap v81: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T22:19:03.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:03 smithi093 ceph-mon[96579]: pgmap v81: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2023-07-21T22:19:04.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 64592 3.62 MB/sec warmup 101 sec latency 2297.974 ms 2023-07-21T22:19:05.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 65128 3.60 MB/sec warmup 102 sec latency 2966.254 ms 2023-07-21T22:19:05.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:05 smithi194 ceph-mon[98288]: pgmap v82: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:05.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:05 smithi134 ceph-mon[97873]: pgmap v82: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:05.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:05 smithi093 ceph-mon[96579]: pgmap v82: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:06.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 67705 3.73 MB/sec warmup 103 sec latency 11.995 ms 2023-07-21T22:19:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/343326168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 msgr2=0x7fe9cc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/343326168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc104770 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0099c0 tx=0x7fe9bc009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/343326168 shutdown_connections 2023-07-21T22:19:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/343326168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc104e50 0x7fe9cc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/343326168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc067c10 0x7fe9cc067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/343326168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc104770 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/343326168 >> 172.21.15.93:0/343326168 conn(0x7fe9cc074760 msgr2=0x7fe9cc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/343326168 shutdown_connections 2023-07-21T22:19:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.574+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/343326168 wait complete. 2023-07-21T22:19:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 Processor -- start 2023-07-21T22:19:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 -- start start 2023-07-21T22:19:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 0x7fe9cc0fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 0x7fe9cc1a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1a42a0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1a43e0 con 0x7fe9cc067c10 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d3d0b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1a4550 con 0x7fe9cc104e50 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 0x7fe9cc0fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46206/0 (socket says 172.21.15.93:46206) 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 -- 172.21.15.93:0/549743307 learned_addr learned my addr 172.21.15.93:0/549743307 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 0x7fe9cc1a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 -- 172.21.15.93:0/549743307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 msgr2=0x7fe9cc1a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 0x7fe9cc1a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 -- 172.21.15.93:0/549743307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 msgr2=0x7fe9cc0fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 0x7fe9cc0fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d12a6700 1 -- 172.21.15.93:0/549743307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0095f0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 0x7fe9cc0fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.575+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 0x7fe9cc1a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009fd0 tx=0x7fe9bc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9c2ffd700 1 -- 172.21.15.93:0/549743307 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc01d070 con 0x7fe9cc0685b0 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9c2ffd700 1 -- 172.21.15.93:0/549743307 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc005020 con 0x7fe9cc0685b0 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc1a47d0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9c2ffd700 1 -- 172.21.15.93:0/549743307 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc004aa0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.576+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1a4b90 con 0x7fe9cc0685b0 2023-07-21T22:19:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.578+0000 7fe9c2ffd700 1 -- 172.21.15.93:0/549743307 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe9bc0054b0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.578+0000 7fe9c2ffd700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 0x7fe9b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.578+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 0x7fe9b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.578+0000 7fe9c2ffd700 1 -- 172.21.15.93:0/549743307 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe9bc0cd7e0 con 0x7fe9cc0685b0 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.579+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 0x7fe9b80a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005950 tx=0x7fe9c800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 Processor -- start 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 -- start start 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f446f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d660 0x55689f4474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f34cc50 con 0x55689f2a36b0 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f39f9e0 con 0x55689f44d660 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f398e70 con 0x55689f44d210 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46212/0 (socket says 172.21.15.93:46212) 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/2833539816 learned_addr learned my addr 172.21.15.93:0/2833539816 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d660 0x55689f4474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/2833539816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 msgr2=0x55689f446f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f446f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f446f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/2833539816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d660 msgr2=0x55689f4474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d660 0x55689f4474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.580+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/2833539816 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55689f1c6d80 con 0x55689f2a36b0 2023-07-21T22:19:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.581+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8011790 tx=0x7fe9c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf1042052c98ae3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.581+0000 7fe9b7fff700 1 -- 172.21.15.93:0/2833539816 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c801e620 con 0x55689f2a36b0 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.581+0000 7fe9b7fff700 1 -- 172.21.15.93:0/2833539816 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c8015070 con 0x55689f2a36b0 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.581+0000 7fe9b7fff700 1 -- 172.21.15.93:0/2833539816 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c801ca10 con 0x55689f2a36b0 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2833539816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 msgr2=0x55689f44ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8011790 tx=0x7fe9c8013000 comp rx=0 tx=0).stop 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2833539816 shutdown_connections 2023-07-21T22:19:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d660 0x55689f4474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44ccd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2833539816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f446f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2833539816 >> 172.21.15.93:0/2833539816 conn(0x55689f43cae0 msgr2=0x55689f4420e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2833539816 shutdown_connections 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2833539816 wait complete. 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 -- start start 2023-07-21T22:19:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.582+0000 7fe9dfe90b80 1 Processor -- start 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f2a36b0 0x55689f4535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d660 0x55689f450fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f338a60 con 0x55689f44d210 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f338760 con 0x55689f2a36b0 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f21c8c0 con 0x55689f44d660 2023-07-21T22:19:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 10 client.0 _ll_register_callbacks cb 0x55689f336700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46220/0 (socket says 172.21.15.93:46220) 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 -- 172.21.15.93:0/2884995013 learned_addr learned my addr 172.21.15.93:0/2884995013 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f2a36b0 0x55689f4535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d660 0x55689f450fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 -- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d660 msgr2=0x55689f450fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d660 0x55689f450fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 -- 172.21.15.93:0/2884995013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f2a36b0 msgr2=0x55689f4535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f2a36b0 0x55689f4535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 -- 172.21.15.93:0/2884995013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c8011440 con 0x55689f44d210 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc09a660 tx=0x7fe9bc09a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55a292dcf25dca2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0a2d20 con 0x55689f44d210 2023-07-21T22:19:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe9bc09add0 con 0x55689f44d210 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.583+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55689f2f2490 con 0x55689f44d210 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc099e50 con 0x55689f44d210 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 10 client.16308 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55689f1afc00 con 0x55689f44d210 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55689f2f2490 con 0x55689f44d210 2023-07-21T22:19:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe9bc0b4020 con 0x55689f44d210 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 20 client.16308 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f4606f0 0x55689f462b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 4 client.16308 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55689f391e50 con 0x55689f4606f0 2023-07-21T22:19:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.584+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f4606f0 0x55689f462b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.585+0000 7fe9b57fa700 10 client.16308 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.585+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f4606f0 0x55689f462b00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7d97b994f57e7a server_cookie=ca66634ce68414 in_seq=0 out_seq=0 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9b57fa700 1 -- 172.21.15.93:0/2884995013 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55689f391e50 con 0x55689f4606f0 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9b57fa700 10 client.16308 handle_command_reply: tid=0 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 client.16308 shutdown 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 msgr2=0x55689f4508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc09a660 tx=0x7fe9bc09a9e0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 shutdown_connections 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f4606f0 0x55689f462b00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d660 0x55689f450fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f2a36b0 0x55689f4535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9b57fa700 0 client.16308 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/2884995013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f44d210 0x55689f4508a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 >> 172.21.15.93:0/2884995013 conn(0x55689f43cae0 msgr2=0x55689f4413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 shutdown_connections 2023-07-21T22:19:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/2884995013 wait complete. 2023-07-21T22:19:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.586+0000 7fe9dfe90b80 20 client.16308 trim_cache size 0 max 16384 2023-07-21T22:19:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 Processor -- start 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 -- start start 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f43cae0 0x55689f457b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 0x55689f441b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f43ec00 con 0x55689f2a36b0 2023-07-21T22:19:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f44c900 con 0x55689f44d210 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f43e4f0 con 0x55689f43cae0 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f43cae0 0x55689f457b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9d12a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46228/0 (socket says 172.21.15.93:46228) 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.601+0000 7fe9d12a6700 1 -- 172.21.15.93:0/133386884 learned_addr learned my addr 172.21.15.93:0/133386884 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 0x55689f441b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 -- 172.21.15.93:0/133386884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f43cae0 msgr2=0x55689f457b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f43cae0 0x55689f457b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 -- 172.21.15.93:0/133386884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 msgr2=0x55689f441b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 0x55689f441b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 -- 172.21.15.93:0/133386884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55689f467730 con 0x55689f2a36b0 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 0x55689f441b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0cb550 tx=0x7fe9bc09eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6ec3b8e4300636 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9c0ff9700 1 -- 172.21.15.93:0/133386884 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0a2d20 con 0x55689f2a36b0 2023-07-21T22:19:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9c0ff9700 1 -- 172.21.15.93:0/133386884 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc09add0 con 0x55689f2a36b0 2023-07-21T22:19:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.602+0000 7fe9c0ff9700 1 -- 172.21.15.93:0/133386884 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc09a0b0 con 0x55689f2a36b0 2023-07-21T22:19:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/133386884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 msgr2=0x55689f44c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0cb550 tx=0x7fe9bc09eb10 comp rx=0 tx=0).stop 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/133386884 shutdown_connections 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f44d210 0x55689f441b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f43cae0 0x55689f457b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/133386884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f44c050 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/133386884 >> 172.21.15.93:0/133386884 conn(0x55689f44ae10 msgr2=0x55689f44e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/133386884 shutdown_connections 2023-07-21T22:19:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/133386884 wait complete. 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 -- start start 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.603+0000 7fe9dfe90b80 1 Processor -- start 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f43cae0 0x55689f456760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f456ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f4b45c0 con 0x55689f2a36b0 2023-07-21T22:19:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f4b4700 con 0x55689f43cae0 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689f4b4870 con 0x55689f44d210 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9dfe90b80 10 client.0 _ll_register_callbacks cb 0x7fe9bc09a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46232/0 (socket says 172.21.15.93:46232) 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/1353264751 learned_addr learned my addr 172.21.15.93:0/1353264751 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f43cae0 0x55689f456760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d22a8700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f456ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 msgr2=0x55689f456ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f456ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/1353264751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f43cae0 msgr2=0x55689f456760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f43cae0 0x55689f456760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 -- 172.21.15.93:0/1353264751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55689f1afc00 con 0x55689f2a36b0 2023-07-21T22:19:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9d1aa7700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8010c00 tx=0x7fe9c80050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52ade808e437dca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.604+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c802d070 con 0x55689f2a36b0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c8015070 con 0x55689f2a36b0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe9c8017070 con 0x55689f2a36b0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c8005240 con 0x55689f2a36b0 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 10 client.16314 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55689f1ed1f0 con 0x55689f2a36b0 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c8015070 con 0x55689f2a36b0 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe9c803b020 con 0x55689f2a36b0 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 10 client.16314 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 20 client.16314 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f44d660 0x55689f4b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 4 client.16314 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:19:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55689f353ff0 con 0x55689f44d660 2023-07-21T22:19:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.605+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f44d660 0x55689f4b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.606+0000 7fe9b57fa700 10 client.16314 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.606+0000 7fe9d12a6700 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f44d660 0x55689f4b7e70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421d33138d95088d server_cookie=ea15138a731b47c8 in_seq=0 out_seq=0 2023-07-21T22:19:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.636+0000 7fe9b57fa700 1 -- 172.21.15.93:0/1353264751 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55689f353ff0 con 0x55689f44d660 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.636+0000 7fe9b57fa700 10 client.16314 handle_command_reply: tid=0 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.636+0000 7fe9dfe90b80 1 client.16314 shutdown 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.636+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 msgr2=0x55689f456220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.636+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8010c00 tx=0x7fe9c80050d0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 shutdown_connections 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55689f44d660 0x55689f4b7e70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55689f44d210 0x55689f456ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9b57fa700 0 client.16314 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55689f2a36b0 0x55689f456220 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 --2- 172.21.15.93:0/1353264751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55689f43cae0 0x55689f456760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 >> 172.21.15.93:0/1353264751 conn(0x55689f44ae10 msgr2=0x55689f44e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 shutdown_connections 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 1 -- 172.21.15.93:0/1353264751 wait complete. 2023-07-21T22:19:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9dfe90b80 20 client.16314 trim_cache size 0 max 16384 2023-07-21T22:19:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:19:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:19:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:19:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:19:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 msgr2=0x7fe9b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 0x7fe9b80a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005950 tx=0x7fe9c800b410 comp rx=0 tx=0).stop 2023-07-21T22:19:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 msgr2=0x7fe9cc0ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009fd0 tx=0x7fe9bc009580 comp rx=0 tx=0).stop 2023-07-21T22:19:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 shutdown_connections 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe9cc104e50 0x7fe9cc1a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9cc067c10 0x7fe9cc0fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe9b80a12c0 0x7fe9b80a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 --2- 172.21.15.93:0/549743307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9cc0685b0 0x7fe9cc0ff180 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.637+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 >> 172.21.15.93:0/549743307 conn(0x7fe9cc074760 msgr2=0x7fe9cc103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.638+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 shutdown_connections 2023-07-21T22:19:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:06.638+0000 7fe9d3d0b700 1 -- 172.21.15.93:0/549743307 wait complete. 2023-07-21T22:19:07.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 68211 3.70 MB/sec warmup 104 sec latency 481.177 ms 2023-07-21T22:19:07.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:07 smithi194 ceph-mon[98288]: pgmap v83: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:07.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:07 smithi134 ceph-mon[97873]: pgmap v83: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:07.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:07 smithi093 ceph-mon[96579]: pgmap v83: 97 pgs: 97 active+clean; 45 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 1.0 MiB/s wr, 150 op/s 2023-07-21T22:19:08.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 68211 3.66 MB/sec warmup 105 sec latency 1481.212 ms 2023-07-21T22:19:09.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 68211 3.63 MB/sec warmup 106 sec latency 2481.299 ms 2023-07-21T22:19:09.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:09 smithi194 ceph-mon[98288]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-07-21T22:19:09.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:09 smithi134 ceph-mon[97873]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-07-21T22:19:09.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:09 smithi093 ceph-mon[96579]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.5 MiB/s wr, 223 op/s 2023-07-21T22:19:10.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 68908 3.64 MB/sec warmup 107 sec latency 3151.307 ms 2023-07-21T22:19:11.313 INFO:tasks.workunit.client.0.smithi093.stdout: 1 71302 3.73 MB/sec warmup 108 sec latency 10.155 ms 2023-07-21T22:19:11.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:11 smithi194 ceph-mon[98288]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-07-21T22:19:11.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:11 smithi134 ceph-mon[97873]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-07-21T22:19:11.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:11 smithi093 ceph-mon[96579]: pgmap v85: 97 pgs: 97 active+clean; 43 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2023-07-21T22:19:12.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 71629 3.70 MB/sec warmup 109 sec latency 658.492 ms 2023-07-21T22:19:13.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 71629 3.67 MB/sec warmup 110 sec latency 1658.577 ms 2023-07-21T22:19:13.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:13 smithi194 ceph-mon[98288]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-07-21T22:19:13.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:13 smithi134 ceph-mon[97873]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-07-21T22:19:13.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:13 smithi093 ceph-mon[96579]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2023-07-21T22:19:14.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 71629 3.64 MB/sec warmup 111 sec latency 2658.669 ms 2023-07-21T22:19:15.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 72150 3.62 MB/sec warmup 112 sec latency 3327.962 ms 2023-07-21T22:19:15.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:15 smithi194 ceph-mon[98288]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:15.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:15 smithi134 ceph-mon[97873]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:15.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:15 smithi093 ceph-mon[96579]: pgmap v87: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:16.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 74816 3.74 MB/sec warmup 113 sec latency 10.795 ms 2023-07-21T22:19:17.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 75210 3.71 MB/sec warmup 114 sec latency 565.853 ms 2023-07-21T22:19:17.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:17 smithi194 ceph-mon[98288]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:17.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:17 smithi134 ceph-mon[97873]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:17.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:17 smithi093 ceph-mon[96579]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 767 KiB/s wr, 120 op/s 2023-07-21T22:19:18.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 75210 3.68 MB/sec warmup 115 sec latency 1565.969 ms 2023-07-21T22:19:19.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 75210 3.64 MB/sec warmup 116 sec latency 2566.012 ms 2023-07-21T22:19:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:19 smithi194 ceph-mon[98288]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-07-21T22:19:19.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:19 smithi134 ceph-mon[97873]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-07-21T22:19:19.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:19 smithi093 ceph-mon[96579]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2023-07-21T22:19:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 -- 172.21.15.93:0/4285648480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 msgr2=0x7f142c105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 --2- 172.21.15.93:0/4285648480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c105f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f141c009990 tx=0x7f141c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 -- 172.21.15.93:0/4285648480 shutdown_connections 2023-07-21T22:19:19.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 --2- 172.21.15.93:0/4285648480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c100500 0x7f142c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 --2- 172.21.15.93:0/4285648480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c105f50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 --2- 172.21.15.93:0/4285648480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c0ffb50 0x7f142c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 -- 172.21.15.93:0/4285648480 >> 172.21.15.93:0/4285648480 conn(0x7f142c0fb650 msgr2=0x7f142c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 -- 172.21.15.93:0/4285648480 shutdown_connections 2023-07-21T22:19:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.757+0000 7f1430b6a700 1 -- 172.21.15.93:0/4285648480 wait complete. 2023-07-21T22:19:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 Processor -- start 2023-07-21T22:19:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 -- start start 2023-07-21T22:19:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c0ffb50 0x7f142c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c100500 0x7f142c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c196130 con 0x7f142c105b80 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1962a0 con 0x7f142c100500 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1430b6a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c196460 con 0x7f142c0ffb50 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46250/0 (socket says 172.21.15.93:46250) 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 -- 172.21.15.93:0/3269829746 learned_addr learned my addr 172.21.15.93:0/3269829746 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f1429d9b700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c100500 0x7f142c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142a59c700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c0ffb50 0x7f142c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 -- 172.21.15.93:0/3269829746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c0ffb50 msgr2=0x7f142c19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c0ffb50 0x7f142c19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 -- 172.21.15.93:0/3269829746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c100500 msgr2=0x7f142c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c100500 0x7f142c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.758+0000 7f142ad9d700 1 -- 172.21.15.93:0/3269829746 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c105b80 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f142ad9d700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f142400ada0 tx=0x7f142400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f141b7fe700 1 -- 172.21.15.93:0/3269829746 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400f070 con 0x7f142c105b80 2023-07-21T22:19:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c196740 con 0x7f142c105b80 2023-07-21T22:19:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f141b7fe700 1 -- 172.21.15.93:0/3269829746 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1424007b30 con 0x7f142c105b80 2023-07-21T22:19:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f141b7fe700 1 -- 172.21.15.93:0/3269829746 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424005280 con 0x7f142c105b80 2023-07-21T22:19:19.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.759+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1a2c50 con 0x7f142c105b80 2023-07-21T22:19:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.761+0000 7f141b7fe700 1 -- 172.21.15.93:0/3269829746 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f142400c8d0 con 0x7f142c105b80 2023-07-21T22:19:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.762+0000 7f141b7fe700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 0x7f14140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.762+0000 7f141b7fe700 1 -- 172.21.15.93:0/3269829746 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14240cc200 con 0x7f142c105b80 2023-07-21T22:19:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.762+0000 7f142a59c700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 0x7f14140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f142a59c700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 0x7f14140a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b5c0 tx=0x7f141c005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 Processor -- start 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 -- start start 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be5246b0 0x5615be6cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be5cdc50 con 0x5615be6c7f40 2023-07-21T22:19:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6209e0 con 0x5615be6c88d0 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.763+0000 7f143ccefb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be619e70 con 0x5615be5246b0 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46256/0 (socket says 172.21.15.93:46256) 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 -- 172.21.15.93:0/2985646225 learned_addr learned my addr 172.21.15.93:0/2985646225 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f142ad9d700 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f142a59c700 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be5246b0 0x5615be6cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 -- 172.21.15.93:0/2985646225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be5246b0 msgr2=0x5615be6cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be5246b0 0x5615be6cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 -- 172.21.15.93:0/2985646225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 msgr2=0x5615be6dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 -- 172.21.15.93:0/2985646225 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615be447d80 con 0x5615be6c7f40 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1429d9b700 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f142000bd30 tx=0x7f14200106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dfca8caf59d9e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1418ff9700 1 -- 172.21.15.93:0/2985646225 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420010e10 con 0x5615be6c7f40 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1418ff9700 1 -- 172.21.15.93:0/2985646225 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1420015070 con 0x5615be6c7f40 2023-07-21T22:19:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.764+0000 7f1418ff9700 1 -- 172.21.15.93:0/2985646225 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420003df0 con 0x5615be6c7f40 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- 172.21.15.93:0/2985646225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 msgr2=0x5615be6c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f142000bd30 tx=0x7f14200106b0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- 172.21.15.93:0/2985646225 shutdown_connections 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be5246b0 0x5615be6cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 --2- 172.21.15.93:0/2985646225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6c8390 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- 172.21.15.93:0/2985646225 >> 172.21.15.93:0/2985646225 conn(0x5615be6bdae0 msgr2=0x5615be6c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- 172.21.15.93:0/2985646225 shutdown_connections 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- 172.21.15.93:0/2985646225 wait complete. 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.765+0000 7f143ccefb80 1 -- start start 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 Processor -- start 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c7f40 0x5615be6d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be5b9a60 con 0x5615be5246b0 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be5b9760 con 0x5615be6c7f40 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be49d8c0 con 0x5615be6c88d0 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f143ccefb80 10 client.0 _ll_register_callbacks cb 0x5615be5b7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46260/0 (socket says 172.21.15.93:46260) 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 -- 172.21.15.93:0/1148245990 learned_addr learned my addr 172.21.15.93:0/1148245990 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142ad9d700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f1429d9b700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c7f40 0x5615be6d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 -- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 msgr2=0x5615be6da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 -- 172.21.15.93:0/1148245990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c7f40 msgr2=0x5615be6d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c7f40 0x5615be6d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 -- 172.21.15.93:0/1148245990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142000b960 con 0x5615be5246b0 2023-07-21T22:19:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f142a59c700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f141c015000 tx=0x7f141c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a132be13919463 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.766+0000 7f1411ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c00f660 con 0x5615be5246b0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f141c00f7c0 con 0x5615be5246b0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615be573490 con 0x5615be5246b0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f141c00f980 con 0x5615be5246b0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 10 client.16326 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5615be430c00 con 0x5615be5246b0 2023-07-21T22:19:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141c03c020 con 0x5615be5246b0 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f141c023b40 con 0x5615be5246b0 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 10 client.16326 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 20 client.16326 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be6ea400 0x5615be6ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 4 client.16326 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.767+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615be612e50 con 0x5615be6ea400 2023-07-21T22:19:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.768+0000 7f1429d9b700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be6ea400 0x5615be6ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.768+0000 7f1411ffb700 10 client.16326 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.768+0000 7f1429d9b700 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be6ea400 0x5615be6ec810 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b35a2bbb000bb2 server_cookie=81d209f401d10d3d in_seq=0 out_seq=0 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f1411ffb700 1 -- 172.21.15.93:0/1148245990 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615be612e50 con 0x5615be6ea400 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f1411ffb700 10 client.16326 handle_command_reply: tid=0 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 client.16326 shutdown 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 msgr2=0x5615be6dcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f141c015000 tx=0x7f141c009310 comp rx=0 tx=0).stop 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 shutdown_connections 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be6ea400 0x5615be6ec810 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f1411ffb700 0 client.16326 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6dcb70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 --2- 172.21.15.93:0/1148245990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c7f40 0x5615be6d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 >> 172.21.15.93:0/1148245990 conn(0x5615be6bdae0 msgr2=0x5615be6c23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 shutdown_connections 2023-07-21T22:19:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 1 -- 172.21.15.93:0/1148245990 wait complete. 2023-07-21T22:19:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.769+0000 7f143ccefb80 20 client.16326 trim_cache size 0 max 16384 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 Processor -- start 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 -- start start 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be5246b0 0x5615be6c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6daab0 con 0x5615be6c7f40 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6e1930 con 0x5615be5246b0 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f143ccefb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6d9330 con 0x5615be6c88d0 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46270/0 (socket says 172.21.15.93:46270) 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 -- 172.21.15.93:0/929123838 learned_addr learned my addr 172.21.15.93:0/929123838 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f142ad9d700 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 -- 172.21.15.93:0/929123838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 msgr2=0x5615be6c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 -- 172.21.15.93:0/929123838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be5246b0 msgr2=0x5615be6c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be5246b0 0x5615be6c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 -- 172.21.15.93:0/929123838 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615be46e1f0 con 0x5615be6c7f40 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f142ad9d700 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f1429d9b700 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f142000b900 tx=0x7f1420005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7ec8a05d50e13f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.784+0000 7f14197fa700 1 -- 172.21.15.93:0/929123838 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420020040 con 0x5615be6c7f40 2023-07-21T22:19:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f14197fa700 1 -- 172.21.15.93:0/929123838 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1420021040 con 0x5615be6c7f40 2023-07-21T22:19:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f14197fa700 1 -- 172.21.15.93:0/929123838 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200038b0 con 0x5615be6c7f40 2023-07-21T22:19:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- 172.21.15.93:0/929123838 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 msgr2=0x5615be6e13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f142000b900 tx=0x7f1420005950 comp rx=0 tx=0).stop 2023-07-21T22:19:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- 172.21.15.93:0/929123838 shutdown_connections 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c88d0 0x5615be6c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be5246b0 0x5615be6c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 --2- 172.21.15.93:0/929123838 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be6c7f40 0x5615be6e13f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- 172.21.15.93:0/929123838 >> 172.21.15.93:0/929123838 conn(0x5615be6bdae0 msgr2=0x5615be6f1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- 172.21.15.93:0/929123838 shutdown_connections 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- 172.21.15.93:0/929123838 wait complete. 2023-07-21T22:19:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 -- start start 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.785+0000 7f143ccefb80 1 Processor -- start 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c7f40 0x5615be6e1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6dfae0 con 0x5615be5246b0 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6dfc50 con 0x5615be6c88d0 2023-07-21T22:19:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615be6dfe10 con 0x5615be6c7f40 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 10 client.0 _ll_register_callbacks cb 0x7f141c003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46274/0 (socket says 172.21.15.93:46274) 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 -- 172.21.15.93:0/3659870247 learned_addr learned my addr 172.21.15.93:0/3659870247 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f1429d9b700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c7f40 0x5615be6e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142ad9d700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 -- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c7f40 msgr2=0x5615be6e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c7f40 0x5615be6e1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 -- 172.21.15.93:0/3659870247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 msgr2=0x5615be6df3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 -- 172.21.15.93:0/3659870247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615be447d80 con 0x5615be5246b0 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f142a59c700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f141c023ea0 tx=0x7f141c02fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b33a06d08324a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f1411ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c023870 con 0x5615be5246b0 2023-07-21T22:19:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f141c03d080 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.786+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1420015070 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f141c005940 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 10 client.16332 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5615be446250 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1420015070 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f141c00f920 con 0x5615be5246b0 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 10 client.16332 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 20 client.16332 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be73f090 0x5615be7414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 4 client.16332 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615be5d4ff0 con 0x5615be73f090 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.787+0000 7f1429d9b700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be73f090 0x5615be7414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.788+0000 7f1411ffb700 10 client.16332 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.788+0000 7f1429d9b700 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be73f090 0x5615be7414a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ee3e6616cce4aa server_cookie=fbc10dd70fe6d985 in_seq=0 out_seq=0 2023-07-21T22:19:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.825+0000 7f1411ffb700 1 -- 172.21.15.93:0/3659870247 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615be5d4ff0 con 0x5615be73f090 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.825+0000 7f1411ffb700 10 client.16332 handle_command_reply: tid=0 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.825+0000 7f143ccefb80 1 client.16332 shutdown 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 msgr2=0x5615be6e0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f141c023ea0 tx=0x7f141c02fbd0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 shutdown_connections 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5615be73f090 0x5615be7414a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615be6c88d0 0x5615be6df3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1411ffb700 0 client.16332 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615be5246b0 0x5615be6e0c30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 --2- 172.21.15.93:0/3659870247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615be6c7f40 0x5615be6e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 >> 172.21.15.93:0/3659870247 conn(0x5615be6bdae0 msgr2=0x5615be6c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 shutdown_connections 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 1 -- 172.21.15.93:0/3659870247 wait complete. 2023-07-21T22:19:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f143ccefb80 20 client.16332 trim_cache size 0 max 16384 2023-07-21T22:19:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:19:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:19:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:19:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:19:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 msgr2=0x7f14140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 0x7f14140a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b5c0 tx=0x7f141c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 msgr2=0x7f142c195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f142400ada0 tx=0x7f142400c330 comp rx=0 tx=0).stop 2023-07-21T22:19:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 shutdown_connections 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14140a12c0 0x7f14140a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f142c105b80 0x7f142c195ac0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f142c0ffb50 0x7f142c19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 --2- 172.21.15.93:0/3269829746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f142c100500 0x7f142c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.826+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 >> 172.21.15.93:0/3269829746 conn(0x7f142c0fb650 msgr2=0x7f142c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.827+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 shutdown_connections 2023-07-21T22:19:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:19.827+0000 7f1430b6a700 1 -- 172.21.15.93:0/3269829746 wait complete. 2023-07-21T22:19:20.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 75795 3.63 MB/sec warmup 117 sec latency 3233.529 ms 2023-07-21T22:19:21.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 78390 3.74 MB/sec warmup 118 sec latency 10.836 ms 2023-07-21T22:19:21.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:21 smithi194 ceph-mon[98288]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 539 KiB/s wr, 92 op/s 2023-07-21T22:19:21.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:21 smithi134 ceph-mon[97873]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 539 KiB/s wr, 92 op/s 2023-07-21T22:19:21.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:21 smithi093 ceph-mon[96579]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 539 KiB/s wr, 92 op/s 2023-07-21T22:19:22.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 80455 3.77 MB/sec warmup 119 sec latency 11.139 ms 2023-07-21T22:19:23.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:23 smithi134 ceph-mon[97873]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2023-07-21T22:19:23.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:23 smithi093 ceph-mon[96579]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2023-07-21T22:19:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:23 smithi194 ceph-mon[98288]: pgmap v91: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.1 MiB/s wr, 168 op/s 2023-07-21T22:19:24.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 82426 0.42 MB/sec execute 1 sec latency 590.470 ms 2023-07-21T22:19:25.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 83128 2.55 MB/sec execute 2 sec latency 1263.011 ms 2023-07-21T22:19:25.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:25 smithi134 ceph-mon[97873]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:25 smithi093 ceph-mon[96579]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:25 smithi194 ceph-mon[98288]: pgmap v92: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:26.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 85541 6.37 MB/sec execute 3 sec latency 13.519 ms 2023-07-21T22:19:27.314 INFO:tasks.workunit.client.0.smithi093.stdout: 1 85843 4.81 MB/sec execute 4 sec latency 675.982 ms 2023-07-21T22:19:27.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:27 smithi134 ceph-mon[97873]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:27.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:27 smithi093 ceph-mon[96579]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:27.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:27 smithi194 ceph-mon[98288]: pgmap v93: 97 pgs: 97 active+clean; 47 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 889 KiB/s wr, 121 op/s 2023-07-21T22:19:28.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 85843 3.85 MB/sec execute 5 sec latency 1676.061 ms 2023-07-21T22:19:29.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 85843 3.21 MB/sec execute 6 sec latency 2676.169 ms 2023-07-21T22:19:29.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:29 smithi134 ceph-mon[97873]: pgmap v94: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.5 MiB/s wr, 204 op/s 2023-07-21T22:19:29.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:29 smithi093 ceph-mon[96579]: pgmap v94: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.5 MiB/s wr, 204 op/s 2023-07-21T22:19:29.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:29 smithi194 ceph-mon[98288]: pgmap v94: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.5 MiB/s wr, 204 op/s 2023-07-21T22:19:30.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 86331 3.02 MB/sec execute 7 sec latency 3347.881 ms 2023-07-21T22:19:31.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 88785 4.60 MB/sec execute 8 sec latency 11.160 ms 2023-07-21T22:19:31.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:31 smithi134 ceph-mon[97873]: pgmap v95: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2023-07-21T22:19:31.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:31 smithi093 ceph-mon[96579]: pgmap v95: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2023-07-21T22:19:31.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:31 smithi194 ceph-mon[98288]: pgmap v95: 97 pgs: 97 active+clean; 51 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.2 MiB/s wr, 159 op/s 2023-07-21T22:19:32.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 89392 4.26 MB/sec execute 9 sec latency 361.969 ms 2023-07-21T22:19:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1394370188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d28104db0 msgr2=0x7f6d2810c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1394370188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d28104db0 0x7f6d2810c980 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6d240099c0 tx=0x7f6d24009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1394370188 shutdown_connections 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1394370188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d28104db0 0x7f6d2810c980 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1394370188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d280fe260 0x7f6d280fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1394370188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 0x7f6d281046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1394370188 >> 172.21.15.93:0/1394370188 conn(0x7f6d28075f90 msgr2=0x7f6d28076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1394370188 shutdown_connections 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.945+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1394370188 wait complete. 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 Processor -- start 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 -- start start 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 0x7f6d2819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d28104db0 0x7f6d28195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281960a0 con 0x7f6d280fe260 2023-07-21T22:19:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28196210 con 0x7f6d280fec00 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2f75a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281963d0 con 0x7f6d28104db0 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2d4f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2d4f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46288/0 (socket says 172.21.15.93:46288) 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/1255020689 learned_addr learned my addr 172.21.15.93:0/1255020689 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.946+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 0x7f6d2819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d28104db0 0x7f6d28195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/1255020689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d28104db0 msgr2=0x7f6d28195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d28104db0 0x7f6d28195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/1255020689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 msgr2=0x7f6d2819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 0x7f6d2819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/1255020689 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240095f0 con 0x7f6d280fe260 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6d200060b0 tx=0x7f6d2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d1e7fc700 1 -- 172.21.15.93:0/1255020689 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000dd20 con 0x7f6d280fe260 2023-07-21T22:19:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d1e7fc700 1 -- 172.21.15.93:0/1255020689 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2000f040 con 0x7f6d280fe260 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d281966b0 con 0x7f6d280fe260 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d1e7fc700 1 -- 172.21.15.93:0/1255020689 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20014600 con 0x7f6d280fe260 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.947+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d281a2bc0 con 0x7f6d280fe260 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.949+0000 7f6d1e7fc700 1 -- 172.21.15.93:0/1255020689 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6d20015400 con 0x7f6d280fe260 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.950+0000 7f6d1e7fc700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 0x7f6d140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.950+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 0x7f6d140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.950+0000 7f6d1e7fc700 1 -- 172.21.15.93:0/1255020689 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6d200cc5d0 con 0x7f6d280fe260 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.950+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 0x7f6d140a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d28106240 tx=0x7f6d1800a300 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 Processor -- start 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 -- start start 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72620 0x5636d1e6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1d6dc50 con 0x5636d1e72a70 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1dc09e0 con 0x5636d1e72620 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1db9e70 con 0x5636d1cc46b0 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46300/0 (socket says 172.21.15.93:46300) 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 -- 172.21.15.93:0/4118321141 learned_addr learned my addr 172.21.15.93:0/4118321141 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2d4f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72620 0x5636d1e6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 -- 172.21.15.93:0/4118321141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 msgr2=0x5636d1e720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 -- 172.21.15.93:0/4118321141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72620 msgr2=0x5636d1e6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72620 0x5636d1e6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 -- 172.21.15.93:0/4118321141 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d1be7d80 con 0x5636d1e72a70 2023-07-21T22:19:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.951+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6d240099c0 tx=0x7f6d24005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8094d2b8d2dba244 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.952+0000 7f6d137fe700 1 -- 172.21.15.93:0/4118321141 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2401d050 con 0x5636d1e72a70 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.952+0000 7f6d137fe700 1 -- 172.21.15.93:0/4118321141 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2401e040 con 0x5636d1e72a70 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.952+0000 7f6d137fe700 1 -- 172.21.15.93:0/4118321141 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24003e00 con 0x5636d1e72a70 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.952+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/4118321141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 msgr2=0x5636d1e6c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.952+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6d240099c0 tx=0x7f6d24005be0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/4118321141 shutdown_connections 2023-07-21T22:19:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e6c8b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/4118321141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72620 0x5636d1e6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/4118321141 >> 172.21.15.93:0/4118321141 conn(0x5636d1e5dae0 msgr2=0x5636d1e630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/4118321141 shutdown_connections 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/4118321141 wait complete. 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- start start 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 Processor -- start 2023-07-21T22:19:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1cc46b0 0x5636d1e7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e72a70 0x5636d1e75c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1d59a60 con 0x5636d1e72620 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1d59760 con 0x5636d1cc46b0 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.953+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1c3d8c0 con 0x5636d1e72a70 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d3b8dfb80 10 client.0 _ll_register_callbacks cb 0x5636d1d57700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46304/0 (socket says 172.21.15.93:46304) 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/2821454284 learned_addr learned my addr 172.21.15.93:0/2821454284 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1cc46b0 0x5636d1e7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e72a70 0x5636d1e75c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e72a70 msgr2=0x5636d1e75c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e72a70 0x5636d1e75c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/2821454284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1cc46b0 msgr2=0x5636d1e7b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1cc46b0 0x5636d1e7b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/2821454284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2810d1c0 con 0x5636d1e72620 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6d1801d9b0 tx=0x7f6d180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5ccbf16d30e050 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1801b850 con 0x5636d1e72620 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6d1801b9b0 con 0x5636d1e72620 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636d1d13490 con 0x5636d1e72620 2023-07-21T22:19:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d1801bb70 con 0x5636d1e72620 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d3b8dfb80 10 client.16344 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5636d1bd0c00 con 0x5636d1e72620 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.954+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d18032020 con 0x5636d1e72620 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6d180253f0 con 0x5636d1e72620 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 10 client.16344 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 20 client.16344 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1e85f90 0x5636d1e883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 4 client.16344 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636d1db2e50 con 0x5636d1e85f90 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1e85f90 0x5636d1e883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d10ff9700 10 client.16344 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.955+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1e85f90 0x5636d1e883a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46928280df22c9a1 server_cookie=584681a998bc98a4 in_seq=0 out_seq=0 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2821454284 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5636d1db2e50 con 0x5636d1e85f90 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d10ff9700 10 client.16344 handle_command_reply: tid=0 2023-07-21T22:19:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 client.16344 shutdown 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 msgr2=0x5636d1e7bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6d1801d9b0 tx=0x7f6d180047c0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 shutdown_connections 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1e85f90 0x5636d1e883a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e72a70 0x5636d1e75c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d10ff9700 0 client.16344 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1cc46b0 0x5636d1e7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2821454284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72620 0x5636d1e7bf10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 >> 172.21.15.93:0/2821454284 conn(0x5636d1e5dae0 msgr2=0x5636d1e66f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 shutdown_connections 2023-07-21T22:19:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2821454284 wait complete. 2023-07-21T22:19:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.956+0000 7f6d3b8dfb80 20 client.16344 trim_cache size 0 max 16384 2023-07-21T22:19:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 Processor -- start 2023-07-21T22:19:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 -- start start 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e75d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 0x5636d1e62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e77200 con 0x5636d1e72a70 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e83530 con 0x5636d1e62b60 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e71d10 con 0x5636d1cc46b0 2023-07-21T22:19:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46310/0 (socket says 172.21.15.93:46310) 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/1689521705 learned_addr learned my addr 172.21.15.93:0/1689521705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e75d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 0x5636d1e62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/1689521705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 msgr2=0x5636d1e75d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e75d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/1689521705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 msgr2=0x5636d1e62f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 0x5636d1e62f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 -- 172.21.15.93:0/1689521705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636d1c0e1f0 con 0x5636d1e72a70 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 0x5636d1e62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.971+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6d18025750 tx=0x7f6d1800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4efc6527fce5222 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d13fff700 1 -- 172.21.15.93:0/1689521705 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18025970 con 0x5636d1e72a70 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d13fff700 1 -- 172.21.15.93:0/1689521705 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d18032070 con 0x5636d1e72a70 2023-07-21T22:19:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d13fff700 1 -- 172.21.15.93:0/1689521705 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18010690 con 0x5636d1e72a70 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/1689521705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 msgr2=0x5636d1e71470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6d18025750 tx=0x7f6d1800a8f0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/1689521705 shutdown_connections 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e62b60 0x5636d1e62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1cc46b0 0x5636d1e75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/1689521705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1e72a70 0x5636d1e71470 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/1689521705 >> 172.21.15.93:0/1689521705 conn(0x5636d1e5dae0 msgr2=0x5636d1e70d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/1689521705 shutdown_connections 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/1689521705 wait complete. 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.972+0000 7f6d3b8dfb80 1 -- start start 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 Processor -- start 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e62b60 0x5636d1e79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72a70 0x5636d1e7a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e7a980 con 0x5636d1cc46b0 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e7aaf0 con 0x5636d1e72a70 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636d1e7acb0 con 0x5636d1e62b60 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d3b8dfb80 10 client.0 _ll_register_callbacks cb 0x7f6d1800ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46316/0 (socket says 172.21.15.93:46316) 2023-07-21T22:19:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/2089049822 learned_addr learned my addr 172.21.15.93:0/2089049822 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e62b60 0x5636d1e79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2dcf7700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72a70 0x5636d1e7a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e62b60 msgr2=0x5636d1e79eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e62b60 0x5636d1e79eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/2089049822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72a70 msgr2=0x5636d1e7a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72a70 0x5636d1e7a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.973+0000 7f6d2d4f6700 1 -- 172.21.15.93:0/2089049822 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636d1bd0c00 con 0x5636d1cc46b0 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d2d4f6700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d200ca340 tx=0x7f6d20084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1beafa35523379 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200a1b10 con 0x5636d1cc46b0 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f6d20095dd0 con 0x5636d1cc46b0 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d2401e040 con 0x5636d1cc46b0 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d20095f90 con 0x5636d1cc46b0 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 10 client.16350 fetch_fsmap learned FSMap version 32 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5636d1be6250 con 0x5636d1cc46b0 2023-07-21T22:19:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d2007c020 con 0x5636d1cc46b0 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6d20098bb0 con 0x5636d1cc46b0 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 10 client.16350 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 20 client.16350 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1edb3a0 0x5636d1edd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 4 client.16350 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636d1d74ff0 con 0x5636d1edb3a0 2023-07-21T22:19:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.974+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1edb3a0 0x5636d1edd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.975+0000 7f6d10ff9700 10 client.16350 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:32.975+0000 7f6d2ccf5700 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1edb3a0 0x5636d1edd7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691530abb809881b server_cookie=a8eb2d072c08537b in_seq=0 out_seq=0 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d10ff9700 1 -- 172.21.15.93:0/2089049822 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636d1d74ff0 con 0x5636d1edb3a0 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d10ff9700 10 client.16350 handle_command_reply: tid=0 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 client.16350 shutdown 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 msgr2=0x5636d1e7bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d200ca340 tx=0x7f6d20084040 comp rx=0 tx=0).stop 2023-07-21T22:19:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 shutdown_connections 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5636d1edb3a0 0x5636d1edd7b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636d1e72a70 0x5636d1e7a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636d1cc46b0 0x5636d1e7bbc0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d10ff9700 0 client.16350 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 --2- 172.21.15.93:0/2089049822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636d1e62b60 0x5636d1e79eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 >> 172.21.15.93:0/2089049822 conn(0x5636d1e5dae0 msgr2=0x5636d1e70d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 shutdown_connections 2023-07-21T22:19:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 1 -- 172.21.15.93:0/2089049822 wait complete. 2023-07-21T22:19:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.004+0000 7f6d3b8dfb80 20 client.16350 trim_cache size 0 max 16384 2023-07-21T22:19:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:19:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:19:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:19:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:19:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 msgr2=0x7f6d140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 0x7f6d140a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d28106240 tx=0x7f6d1800a300 comp rx=0 tx=0).stop 2023-07-21T22:19:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 msgr2=0x7f6d2819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6d200060b0 tx=0x7f6d2000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 shutdown_connections 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d28104db0 0x7f6d28195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f6d140a1340 0x7f6d140a37f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d280fe260 0x7f6d2819bbd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 --2- 172.21.15.93:0/1255020689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d280fec00 0x7f6d2819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 >> 172.21.15.93:0/1255020689 conn(0x7f6d28075f90 msgr2=0x7f6d280fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 shutdown_connections 2023-07-21T22:19:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:33.005+0000 7f6d2f75a700 1 -- 172.21.15.93:0/1255020689 wait complete. 2023-07-21T22:19:33.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 89392 3.84 MB/sec execute 10 sec latency 1362.037 ms 2023-07-21T22:19:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: pgmap v96: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-07-21T22:19:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:33 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: pgmap v96: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:33 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: pgmap v96: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2023-07-21T22:19:33.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:33.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:33.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:33 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:34.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 89392 3.49 MB/sec execute 11 sec latency 2362.129 ms 2023-07-21T22:19:34.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:34 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:34.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:34 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:34.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:34 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:34.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:34 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:34.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:34 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:34.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:34 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:34.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:34 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:34.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:34 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:34.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:34 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:19:35.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 89905 3.36 MB/sec execute 12 sec latency 3028.662 ms 2023-07-21T22:19:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:35 smithi134 ceph-mon[97873]: pgmap v97: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:35.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:35 smithi093 ceph-mon[96579]: pgmap v97: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:35.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:35 smithi194 ceph-mon[98288]: pgmap v97: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:36.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 92503 4.39 MB/sec execute 13 sec latency 11.451 ms 2023-07-21T22:19:37.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 92992 4.12 MB/sec execute 14 sec latency 466.201 ms 2023-07-21T22:19:37.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:37 smithi134 ceph-mon[97873]: pgmap v98: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:37.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:37 smithi093 ceph-mon[96579]: pgmap v98: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:37.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:37 smithi194 ceph-mon[98288]: pgmap v98: 97 pgs: 97 active+clean; 53 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:19:38.315 INFO:tasks.workunit.client.0.smithi093.stdout: 1 92992 3.84 MB/sec execute 15 sec latency 1466.297 ms 2023-07-21T22:19:39.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 92992 3.60 MB/sec execute 16 sec latency 2466.345 ms 2023-07-21T22:19:39.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:39 smithi134 ceph-mon[97873]: pgmap v99: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-07-21T22:19:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:39 smithi093 ceph-mon[96579]: pgmap v99: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-07-21T22:19:39.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:39 smithi194 ceph-mon[98288]: pgmap v99: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2023-07-21T22:19:40.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93060 3.40 MB/sec execute 17 sec latency 3136.906 ms 2023-07-21T22:19:41.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93060 3.21 MB/sec execute 18 sec latency 1280.505 ms 2023-07-21T22:19:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:41 smithi134 ceph-mon[97873]: pgmap v100: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1006 KiB/s wr, 128 op/s 2023-07-21T22:19:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:41 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:41 smithi093 ceph-mon[96579]: pgmap v100: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1006 KiB/s wr, 128 op/s 2023-07-21T22:19:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:41 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:41.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:41 smithi194 ceph-mon[98288]: pgmap v100: 97 pgs: 97 active+clean; 51 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1006 KiB/s wr, 128 op/s 2023-07-21T22:19:41.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:41 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:42.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93060 3.04 MB/sec execute 19 sec latency 2280.591 ms 2023-07-21T22:19:43.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93060 2.89 MB/sec execute 20 sec latency 3280.682 ms 2023-07-21T22:19:43.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:43 smithi134 ceph-mon[97873]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2023-07-21T22:19:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:43 smithi093 ceph-mon[96579]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2023-07-21T22:19:43.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:43 smithi194 ceph-mon[98288]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2023-07-21T22:19:44.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93060 2.75 MB/sec execute 21 sec latency 4280.781 ms 2023-07-21T22:19:45.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 93765 2.84 MB/sec execute 22 sec latency 4947.807 ms 2023-07-21T22:19:45.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:45 smithi134 ceph-mon[97873]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:45.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:45 smithi093 ceph-mon[96579]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:45.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:45 smithi194 ceph-mon[98288]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 -- 172.21.15.93:0/610655848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 msgr2=0x7f29cc10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 --2- 172.21.15.93:0/610655848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc10b860 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009990 tx=0x7f29c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 -- 172.21.15.93:0/610655848 shutdown_connections 2023-07-21T22:19:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 --2- 172.21.15.93:0/610655848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc10b860 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 --2- 172.21.15.93:0/610655848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc104a30 0x7f29cc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 --2- 172.21.15.93:0/610655848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc0fea30 0x7f29cc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 -- 172.21.15.93:0/610655848 >> 172.21.15.93:0/610655848 conn(0x7f29cc074760 msgr2=0x7f29cc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 -- 172.21.15.93:0/610655848 shutdown_connections 2023-07-21T22:19:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.117+0000 7f29d35b1700 1 -- 172.21.15.93:0/610655848 wait complete. 2023-07-21T22:19:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 Processor -- start 2023-07-21T22:19:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 -- start start 2023-07-21T22:19:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc0fea30 0x7f29cc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc104a30 0x7f29cc194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc194e90 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc195000 con 0x7f29cc0fea30 2023-07-21T22:19:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d35b1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc1951c0 con 0x7f29cc104a30 2023-07-21T22:19:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46332/0 (socket says 172.21.15.93:46332) 2023-07-21T22:19:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d134d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc0fea30 0x7f29cc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 -- 172.21.15.93:0/2099967003 learned_addr learned my addr 172.21.15.93:0/2099967003 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d1b4e700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc104a30 0x7f29cc194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 -- 172.21.15.93:0/2099967003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc104a30 msgr2=0x7f29cc194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc104a30 0x7f29cc194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.118+0000 7f29d0b4c700 1 -- 172.21.15.93:0/2099967003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc0fea30 msgr2=0x7f29cc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc0fea30 0x7f29cc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29d0b4c700 1 -- 172.21.15.93:0/2099967003 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80095f0 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e9a0 tx=0x7f29bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29c27fc700 1 -- 172.21.15.93:0/2099967003 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc009790 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29c27fc700 1 -- 172.21.15.93:0/2099967003 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29bc010430 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc1954a0 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29c27fc700 1 -- 172.21.15.93:0/2099967003 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc018680 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.119+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc1a19b0 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.122+0000 7f29c27fc700 1 -- 172.21.15.93:0/2099967003 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f29bc0105a0 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.122+0000 7f29c27fc700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 0x7f29b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29d134d700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 0x7f29b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29c27fc700 1 -- 172.21.15.93:0/2099967003 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f29bc014070 con 0x7f29cc0ff3e0 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29d134d700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 0x7f29b80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0ffba0 tx=0x7f29c400b040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 Processor -- start 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 -- start start 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18210 0x558bffa11f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff917c50 con 0x558bff86e6b0 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff96a9e0 con 0x558bffa18660 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.123+0000 7f29df736b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff963e70 con 0x558bffa18210 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46338/0 (socket says 172.21.15.93:46338) 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 -- 172.21.15.93:0/1585510274 learned_addr learned my addr 172.21.15.93:0/1585510274 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18210 0x558bffa11f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d1b4e700 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 -- 172.21.15.93:0/1585510274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18210 msgr2=0x558bffa11f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18210 0x558bffa11f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 -- 172.21.15.93:0/1585510274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 msgr2=0x558bffa124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 -- 172.21.15.93:0/1585510274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff791d80 con 0x558bff86e6b0 2023-07-21T22:19:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29d134d700 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29c4017760 tx=0x7f29c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27931b83b79aea67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29b77fe700 1 -- 172.21.15.93:0/1585510274 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400e040 con 0x558bff86e6b0 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29b77fe700 1 -- 172.21.15.93:0/1585510274 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c400f040 con 0x558bff86e6b0 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.124+0000 7f29b77fe700 1 -- 172.21.15.93:0/1585510274 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c40219f0 con 0x558bff86e6b0 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- 172.21.15.93:0/1585510274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 msgr2=0x558bffa17cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29c4017760 tx=0x7f29c4014000 comp rx=0 tx=0).stop 2023-07-21T22:19:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- 172.21.15.93:0/1585510274 shutdown_connections 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa17cd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 --2- 172.21.15.93:0/1585510274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18210 0x558bffa11f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- 172.21.15.93:0/1585510274 >> 172.21.15.93:0/1585510274 conn(0x558bffa07ae0 msgr2=0x558bffa0d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- 172.21.15.93:0/1585510274 shutdown_connections 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- 172.21.15.93:0/1585510274 wait complete. 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 -- start start 2023-07-21T22:19:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.125+0000 7f29df736b80 1 Processor -- start 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 0x558bffa1e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff903a60 con 0x558bffa18210 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff903760 con 0x558bff86e6b0 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff7e78c0 con 0x558bffa18660 2023-07-21T22:19:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29df736b80 10 client.0 _ll_register_callbacks cb 0x558bff901700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d134d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 0x558bffa1e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d1b4e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46346/0 (socket says 172.21.15.93:46346) 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 -- 172.21.15.93:0/4119077165 learned_addr learned my addr 172.21.15.93:0/4119077165 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 -- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 msgr2=0x558bffa1bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 -- 172.21.15.93:0/4119077165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 msgr2=0x558bffa1e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 0x558bffa1e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 -- 172.21.15.93:0/4119077165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c4017440 con 0x558bffa18210 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d134d700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 0x558bffa1e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f29bc099f40 tx=0x7f29bc09a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3d6061a6127030 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.126+0000 7f29b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0996a0 con 0x558bffa18210 2023-07-21T22:19:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f29bc0a2400 con 0x558bffa18210 2023-07-21T22:19:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bff8bd490 con 0x558bffa18210 2023-07-21T22:19:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29bc0a1f70 con 0x558bffa18210 2023-07-21T22:19:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 10 client.16362 fetch_fsmap learned FSMap version 33 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558bff77ac00 con 0x558bffa18210 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558bff8bd490 con 0x558bffa18210 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f29bc099030 con 0x558bffa18210 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 20 client.16362 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa2bb20 0x558bffa2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 4 client.16362 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bff95ce50 con 0x558bffa2bb20 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.127+0000 7f29d134d700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa2bb20 0x558bffa2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29b4ff9700 10 client.16362 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29d134d700 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa2bb20 0x558bffa2df30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c21a10fd72523ba server_cookie=89b948b31fc4f58e in_seq=0 out_seq=0 2023-07-21T22:19:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29b4ff9700 1 -- 172.21.15.93:0/4119077165 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558bff95ce50 con 0x558bffa2bb20 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29b4ff9700 10 client.16362 handle_command_reply: tid=0 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 client.16362 shutdown 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 msgr2=0x558bffa1b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f29bc099f40 tx=0x7f29bc09a0f0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 shutdown_connections 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa2bb20 0x558bffa2df30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29b4ff9700 0 client.16362 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bff86e6b0 0x558bffa1e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 --2- 172.21.15.93:0/4119077165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa1b7d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.128+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 >> 172.21.15.93:0/4119077165 conn(0x558bffa07ae0 msgr2=0x558bffa0c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.129+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 shutdown_connections 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.129+0000 7f29df736b80 1 -- 172.21.15.93:0/4119077165 wait complete. 2023-07-21T22:19:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.129+0000 7f29df736b80 20 client.16362 trim_cache size 0 max 16384 2023-07-21T22:19:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.143+0000 7f29df736b80 1 Processor -- start 2023-07-21T22:19:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.143+0000 7f29df736b80 1 -- start start 2023-07-21T22:19:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.143+0000 7f29df736b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.143+0000 7f29df736b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18210 0x558bffa19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29df736b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29df736b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa12f60 con 0x558bff86e6b0 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29df736b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa164a0 con 0x558bffa18210 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29df736b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa178c0 con 0x558bffa18660 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46352/0 (socket says 172.21.15.93:46352) 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 -- 172.21.15.93:0/2301993401 learned_addr learned my addr 172.21.15.93:0/2301993401 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18210 0x558bffa19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d1b4e700 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 -- 172.21.15.93:0/2301993401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 msgr2=0x558bffa1e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 -- 172.21.15.93:0/2301993401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18210 msgr2=0x558bffa19140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18210 0x558bffa19140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 -- 172.21.15.93:0/2301993401 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7b81f0 con 0x558bff86e6b0 2023-07-21T22:19:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29d134d700 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f29c40060b0 tx=0x7f29c4017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0077453b2b11c63 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29b7fff700 1 -- 172.21.15.93:0/2301993401 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c402f040 con 0x558bff86e6b0 2023-07-21T22:19:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29b7fff700 1 -- 172.21.15.93:0/2301993401 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c4034540 con 0x558bff86e6b0 2023-07-21T22:19:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.144+0000 7f29b7fff700 1 -- 172.21.15.93:0/2301993401 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c400e040 con 0x558bff86e6b0 2023-07-21T22:19:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- 172.21.15.93:0/2301993401 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 msgr2=0x558bffa22b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f29c40060b0 tx=0x7f29c4017ea0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- 172.21.15.93:0/2301993401 shutdown_connections 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bffa18660 0x558bffa1e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bff86e6b0 0x558bffa22b80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 --2- 172.21.15.93:0/2301993401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18210 0x558bffa19140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- 172.21.15.93:0/2301993401 >> 172.21.15.93:0/2301993401 conn(0x558bffa07ae0 msgr2=0x558bffa18a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- 172.21.15.93:0/2301993401 shutdown_connections 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- 172.21.15.93:0/2301993401 wait complete. 2023-07-21T22:19:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 -- start start 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.145+0000 7f29df736b80 1 Processor -- start 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa14490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa14b30 con 0x558bffa18210 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa14c70 con 0x558bffa18660 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bffa14e30 con 0x558bff86e6b0 2023-07-21T22:19:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 10 client.0 _ll_register_callbacks cb 0x7f29bc0999a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d134d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46360/0 (socket says 172.21.15.93:46360) 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d134d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:33438/0 (socket says 172.21.15.93:33438) 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 -- 172.21.15.93:0/400783503 learned_addr learned my addr 172.21.15.93:0/400783503 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d1b4e700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa14490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 -- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 msgr2=0x558bffa13920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 -- 172.21.15.93:0/400783503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 msgr2=0x558bffa14490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa14490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 -- 172.21.15.93:0/400783503 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bff791d80 con 0x558bffa18210 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d134d700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29d0b4c700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f29bc099cb0 tx=0x7f29bc0a2200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5895ee59d450557b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0962f0 con 0x558bffa18210 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f29bc0a2400 con 0x558bffa18210 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.146+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29c400f040 con 0x558bffa18210 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29bc0ca2e0 con 0x558bffa18210 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 10 client.16368 fetch_fsmap learned FSMap version 33 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558bff790250 con 0x558bffa18210 2023-07-21T22:19:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29c400f040 con 0x558bffa18210 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f29bc0990f0 con 0x558bffa18210 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 20 client.16368 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa216d0 0x558bffa21ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 4 client.16368 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bff91eff0 con 0x558bffa216d0 2023-07-21T22:19:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.147+0000 7f29d134d700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa216d0 0x558bffa21ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.148+0000 7f29b4ff9700 10 client.16368 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.148+0000 7f29d134d700 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa216d0 0x558bffa21ad0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1941ff9012daeb2f server_cookie=8d6f5817dc461b8f in_seq=0 out_seq=0 2023-07-21T22:19:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29b4ff9700 1 -- 172.21.15.93:0/400783503 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bff91eff0 con 0x558bffa216d0 2023-07-21T22:19:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29b4ff9700 10 client.16368 handle_command_reply: tid=0 2023-07-21T22:19:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 client.16368 shutdown 2023-07-21T22:19:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 msgr2=0x558bffa13e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f29bc099cb0 tx=0x7f29bc0a2200 comp rx=0 tx=0).stop 2023-07-21T22:19:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 shutdown_connections 2023-07-21T22:19:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bffa216d0 0x558bffa21ad0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bffa18660 0x558bffa14490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29b4ff9700 0 client.16368 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bff86e6b0 0x558bffa13920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 --2- 172.21.15.93:0/400783503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bffa18210 0x558bffa13e60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 >> 172.21.15.93:0/400783503 conn(0x558bffa07ae0 msgr2=0x558bffa17140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 shutdown_connections 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 1 -- 172.21.15.93:0/400783503 wait complete. 2023-07-21T22:19:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.181+0000 7f29df736b80 20 client.16368 trim_cache size 0 max 16384 2023-07-21T22:19:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:19:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:19:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:19:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:19:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 msgr2=0x7f29b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 0x7f29b80a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0ffba0 tx=0x7f29c400b040 comp rx=0 tx=0).stop 2023-07-21T22:19:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 msgr2=0x7f29cc19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e9a0 tx=0x7f29bc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 shutdown_connections 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f29cc104a30 0x7f29cc194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29cc0fea30 0x7f29cc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f29b80a12c0 0x7f29b80a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 --2- 172.21.15.93:0/2099967003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29cc0ff3e0 0x7f29cc19af00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 >> 172.21.15.93:0/2099967003 conn(0x7f29cc074760 msgr2=0x7f29cc0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 shutdown_connections 2023-07-21T22:19:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:46.182+0000 7f29d35b1700 1 -- 172.21.15.93:0/2099967003 wait complete. 2023-07-21T22:19:46.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 96117 3.31 MB/sec execute 23 sec latency 28.721 ms 2023-07-21T22:19:46.880 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:19:46 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:19:46.880 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:19:46 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:19:46.880 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:19:46 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/1536493429 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636cb4f71e0 con 0x5636cb4ccc00 2023-07-21T22:19:46.880 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:19:46 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:19:47.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 96483 3.19 MB/sec execute 24 sec latency 623.713 ms 2023-07-21T22:19:47.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: pgmap v103: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:19:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/4062227106,v1:172.21.15.134:6837/4062227106] up:boot 2023-07-21T22:19:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:47 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:19:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: pgmap v103: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:19:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/4062227106,v1:172.21.15.134:6837/4062227106] up:boot 2023-07-21T22:19:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:47 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:19:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: pgmap v103: 97 pgs: 97 active+clean; 52 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 746 KiB/s wr, 93 op/s 2023-07-21T22:19:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:19:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/4062227106,v1:172.21.15.134:6837/4062227106] up:boot 2023-07-21T22:19:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:47 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:19:48.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 96483 3.06 MB/sec execute 25 sec latency 1623.814 ms 2023-07-21T22:19:49.316 INFO:tasks.workunit.client.0.smithi093.stdout: 1 96483 2.95 MB/sec execute 26 sec latency 2623.903 ms 2023-07-21T22:19:49.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:49 smithi134 ceph-mon[97873]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:49.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:49 smithi093 ceph-mon[96579]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:49.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:49 smithi194 ceph-mon[98288]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:50.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 97000 2.91 MB/sec execute 27 sec latency 3294.626 ms 2023-07-21T22:19:51.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 99578 3.40 MB/sec execute 28 sec latency 44.589 ms 2023-07-21T22:19:51.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:51 smithi093 ceph-mon[96579]: pgmap v106: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:51.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:51 smithi194 ceph-mon[98288]: pgmap v106: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:51.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:51 smithi134 ceph-mon[97873]: pgmap v106: 97 pgs: 97 active+clean; 49 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 943 KiB/s wr, 100 op/s 2023-07-21T22:19:52.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 100028 3.30 MB/sec execute 29 sec latency 504.377 ms 2023-07-21T22:19:53.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 100028 3.19 MB/sec execute 30 sec latency 1504.490 ms 2023-07-21T22:19:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:53 smithi093 ceph-mon[96579]: pgmap v107: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:53.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:53 smithi194 ceph-mon[98288]: pgmap v107: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:53.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:53 smithi134 ceph-mon[97873]: pgmap v107: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:54.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 100028 3.09 MB/sec execute 31 sec latency 2504.623 ms 2023-07-21T22:19:55.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 100569 3.05 MB/sec execute 32 sec latency 3171.387 ms 2023-07-21T22:19:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:55 smithi093 ceph-mon[96579]: pgmap v108: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:55 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:55.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:55 smithi194 ceph-mon[98288]: pgmap v108: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:55.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:55 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:55.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:55 smithi134 ceph-mon[97873]: pgmap v108: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:55.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:55 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:19:56.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 103180 3.47 MB/sec execute 33 sec latency 10.427 ms 2023-07-21T22:19:57.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 103700 3.39 MB/sec execute 34 sec latency 477.084 ms 2023-07-21T22:19:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:57 smithi093 ceph-mon[96579]: pgmap v109: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:57.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:57 smithi194 ceph-mon[98288]: pgmap v109: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:57.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:57 smithi134 ceph-mon[97873]: pgmap v109: 97 pgs: 97 active+clean; 51 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2023-07-21T22:19:58.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 103700 3.29 MB/sec execute 35 sec latency 1477.191 ms 2023-07-21T22:19:59.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 103700 3.20 MB/sec execute 36 sec latency 2477.227 ms 2023-07-21T22:19:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.299+0000 7f865c2b8700 1 -- 172.21.15.93:0/3281347519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 msgr2=0x7f865410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.299+0000 7f865c2b8700 1 --2- 172.21.15.93:0/3281347519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f865410b870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f864c00b0a0 tx=0x7f864c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- 172.21.15.93:0/3281347519 shutdown_connections 2023-07-21T22:19:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- 172.21.15.93:0/3281347519 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f865410b870 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- 172.21.15.93:0/3281347519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8654104b40 0x7f8654104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- 172.21.15.93:0/3281347519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86540ff080 0x7f86540ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- 172.21.15.93:0/3281347519 >> 172.21.15.93:0/3281347519 conn(0x7f8654074d40 msgr2=0x7f8654075140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- 172.21.15.93:0/3281347519 shutdown_connections 2023-07-21T22:19:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- 172.21.15.93:0/3281347519 wait complete. 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 Processor -- start 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- start start 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86540ff080 0x7f8654100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 0x7f86541a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654101390 con 0x7f86540ffa30 2023-07-21T22:19:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a44a0 con 0x7f8654104b40 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.300+0000 7f865c2b8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a45e0 con 0x7f86540ff080 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46374/0 (socket says 172.21.15.93:46374) 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 -- 172.21.15.93:0/2182293010 learned_addr learned my addr 172.21.15.93:0/2182293010 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f865a054700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86540ff080 0x7f8654100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f865a855700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 0x7f86541a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 -- 172.21.15.93:0/2182293010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86540ff080 msgr2=0x7f8654100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86540ff080 0x7f8654100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 -- 172.21.15.93:0/2182293010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 msgr2=0x7f86541a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 0x7f86541a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 -- 172.21.15.93:0/2182293010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009d00 con 0x7f86540ffa30 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f865a855700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 0x7f86541a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f8659853700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f864400c920 tx=0x7f864400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f864b7fe700 1 -- 172.21.15.93:0/2182293010 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644007b00 con 0x7f86540ffa30 2023-07-21T22:19:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f864b7fe700 1 -- 172.21.15.93:0/2182293010 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864400ce70 con 0x7f86540ffa30 2023-07-21T22:19:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86541a4840 con 0x7f86540ffa30 2023-07-21T22:19:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f864b7fe700 1 -- 172.21.15.93:0/2182293010 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440054e0 con 0x7f86540ffa30 2023-07-21T22:19:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.301+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86541a4d20 con 0x7f86540ffa30 2023-07-21T22:19:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.303+0000 7f864b7fe700 1 -- 172.21.15.93:0/2182293010 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8644004750 con 0x7f86540ffa30 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.304+0000 7f864b7fe700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 0x7f86400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.304+0000 7f865a054700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 0x7f86400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.304+0000 7f864b7fe700 1 -- 172.21.15.93:0/2182293010 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f86440cc8c0 con 0x7f86540ffa30 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.305+0000 7f865a054700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 0x7f86400a37f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650005950 tx=0x7f865000b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.305+0000 7f866843db80 1 Processor -- start 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.305+0000 7f866843db80 1 -- start start 2023-07-21T22:19:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.305+0000 7f866843db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 0x55beb3d7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.305+0000 7f866843db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f866843db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f866843db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3c7fc50 con 0x55beb3d80660 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f866843db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3cd29e0 con 0x55beb3d80210 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f866843db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3ccbe70 con 0x55beb3bd66b0 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46384/0 (socket says 172.21.15.93:46384) 2023-07-21T22:19:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 -- 172.21.15.93:0/2581807547 learned_addr learned my addr 172.21.15.93:0/2581807547 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a054700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 0x55beb3d7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f8659853700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 -- 172.21.15.93:0/2581807547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 msgr2=0x55beb3d7fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 0x55beb3d7fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 -- 172.21.15.93:0/2581807547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 msgr2=0x55beb3d79f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d79f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 -- 172.21.15.93:0/2581807547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb3af9d80 con 0x55beb3d80660 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a054700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 0x55beb3d7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f865a855700 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f864c00b830 tx=0x7f864c0091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03eb0842675e7d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f8648dac700 1 -- 172.21.15.93:0/2581807547 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00e050 con 0x55beb3d80660 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f8648dac700 1 -- 172.21.15.93:0/2581807547 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864c0079a0 con 0x55beb3d80660 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.306+0000 7f8648dac700 1 -- 172.21.15.93:0/2581807547 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c005610 con 0x55beb3d80660 2023-07-21T22:19:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- 172.21.15.93:0/2581807547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 msgr2=0x55beb3d7a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f864c00b830 tx=0x7f864c0091b0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- 172.21.15.93:0/2581807547 shutdown_connections 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3d80660 0x55beb3d7a4a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3bd66b0 0x55beb3d7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 --2- 172.21.15.93:0/2581807547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- 172.21.15.93:0/2581807547 >> 172.21.15.93:0/2581807547 conn(0x55beb3d6fae0 msgr2=0x55beb3d750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- 172.21.15.93:0/2581807547 shutdown_connections 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- 172.21.15.93:0/2581807547 wait complete. 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 -- start start 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.307+0000 7f866843db80 1 Processor -- start 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80660 0x55beb3d83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3c6ba60 con 0x55beb3bd66b0 2023-07-21T22:19:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3c6b760 con 0x55beb3d80210 2023-07-21T22:19:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3b4f8c0 con 0x55beb3d80660 2023-07-21T22:19:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 10 client.0 _ll_register_callbacks cb 0x55beb3c69700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46386/0 (socket says 172.21.15.93:46386) 2023-07-21T22:19:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 -- 172.21.15.93:0/1853002349 learned_addr learned my addr 172.21.15.93:0/1853002349 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a855700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80660 0x55beb3d83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 -- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80660 msgr2=0x55beb3d83e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f8659853700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80660 0x55beb3d83e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 -- 172.21.15.93:0/1853002349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 msgr2=0x55beb3d837e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d837e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 -- 172.21.15.93:0/1853002349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865410c170 con 0x55beb3bd66b0 2023-07-21T22:19:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f865a054700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8650011740 tx=0x7f86500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d9de2f75adeafe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f863dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650004a30 con 0x55beb3bd66b0 2023-07-21T22:19:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8650017070 con 0x55beb3bd66b0 2023-07-21T22:19:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.308+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beb3c25490 con 0x55beb3bd66b0 2023-07-21T22:19:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8650010eb0 con 0x55beb3bd66b0 2023-07-21T22:19:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 10 client.16386 fetch_fsmap learned FSMap version 35 2023-07-21T22:19:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55beb3ae2c00 con 0x55beb3bd66b0 2023-07-21T22:19:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beb3c25490 con 0x55beb3bd66b0 2023-07-21T22:19:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f865001f6d0 con 0x55beb3bd66b0 2023-07-21T22:19:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 10 client.16386 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T22:19:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 20 client.16386 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3d93b40 0x55beb3d95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 4 client.16386 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55beb3cc4e50 con 0x55beb3d93b40 2023-07-21T22:19:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.309+0000 7f8659853700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3d93b40 0x55beb3d95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.310+0000 7f863dffb700 10 client.16386 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.310+0000 7f8659853700 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3d93b40 0x55beb3d95f50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8d27a95219e378 server_cookie=691b1f8f42ff0064 in_seq=0 out_seq=0 2023-07-21T22:19:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.310+0000 7f863dffb700 1 -- 172.21.15.93:0/1853002349 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55beb3cc4e50 con 0x55beb3d93b40 2023-07-21T22:19:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.310+0000 7f863dffb700 10 client.16386 handle_command_reply: tid=0 2023-07-21T22:19:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.310+0000 7f866843db80 1 client.16386 shutdown 2023-07-21T22:19:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 msgr2=0x55beb3d864f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8650011740 tx=0x7f86500047c0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 shutdown_connections 2023-07-21T22:19:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3d93b40 0x55beb3d95f50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80660 0x55beb3d83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d864f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f863dffb700 0 client.16386 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 --2- 172.21.15.93:0/1853002349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80210 0x55beb3d837e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 >> 172.21.15.93:0/1853002349 conn(0x55beb3d6fae0 msgr2=0x55beb3d743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 shutdown_connections 2023-07-21T22:19:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 1 -- 172.21.15.93:0/1853002349 wait complete. 2023-07-21T22:19:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.311+0000 7f866843db80 20 client.16386 trim_cache size 0 max 16384 2023-07-21T22:19:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.325+0000 7f866843db80 1 Processor -- start 2023-07-21T22:19:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 -- start start 2023-07-21T22:19:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7acd0 con 0x55beb3bd66b0 2023-07-21T22:19:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7a830 con 0x55beb3d80660 2023-07-21T22:19:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f866843db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7e420 con 0x55beb3d80210 2023-07-21T22:19:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46394/0 (socket says 172.21.15.93:46394) 2023-07-21T22:19:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 -- 172.21.15.93:0/3077532543 learned_addr learned my addr 172.21.15.93:0/3077532543 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f8659853700 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a855700 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 -- 172.21.15.93:0/3077532543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 msgr2=0x55beb3d86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d86090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 -- 172.21.15.93:0/3077532543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 msgr2=0x55beb3d74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d74b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 -- 172.21.15.93:0/3077532543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb3b201f0 con 0x55beb3bd66b0 2023-07-21T22:19:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f865a054700 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef40daaa58a83c56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f86495ad700 1 -- 172.21.15.93:0/3077532543 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865001f840 con 0x55beb3bd66b0 2023-07-21T22:19:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f86495ad700 1 -- 172.21.15.93:0/3077532543 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8650015070 con 0x55beb3bd66b0 2023-07-21T22:19:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.326+0000 7f86495ad700 1 -- 172.21.15.93:0/3077532543 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650030e80 con 0x55beb3bd66b0 2023-07-21T22:19:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- 172.21.15.93:0/3077532543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 msgr2=0x55beb3d8aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650010810 comp rx=0 tx=0).stop 2023-07-21T22:19:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- 172.21.15.93:0/3077532543 shutdown_connections 2023-07-21T22:19:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d8aaf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 --2- 172.21.15.93:0/3077532543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- 172.21.15.93:0/3077532543 >> 172.21.15.93:0/3077532543 conn(0x55beb3d6fae0 msgr2=0x55beb3d7d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- 172.21.15.93:0/3077532543 shutdown_connections 2023-07-21T22:19:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- 172.21.15.93:0/3077532543 wait complete. 2023-07-21T22:19:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 -- start start 2023-07-21T22:19:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.327+0000 7f866843db80 1 Processor -- start 2023-07-21T22:19:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d7bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d7c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7ca70 con 0x55beb3bd66b0 2023-07-21T22:19:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7cbe0 con 0x55beb3d80660 2023-07-21T22:19:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb3d7cda0 con 0x55beb3d80210 2023-07-21T22:19:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f866843db80 10 client.0 _ll_register_callbacks cb 0x7f8650010cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46400/0 (socket says 172.21.15.93:46400) 2023-07-21T22:19:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 -- 172.21.15.93:0/2497870799 learned_addr learned my addr 172.21.15.93:0/2497870799 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:19:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f8659853700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d7bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a855700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d7c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 -- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 msgr2=0x55beb3d7bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d7bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 -- 172.21.15.93:0/2497870799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 msgr2=0x55beb3d7c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d7c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 -- 172.21.15.93:0/2497870799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55beb3ae2c00 con 0x55beb3bd66b0 2023-07-21T22:19:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f865a054700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8650011740 tx=0x7f8650010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4da438bd7efdd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.328+0000 7f863dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:19:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650030670 con 0x55beb3bd66b0 2023-07-21T22:19:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8650017040 con 0x55beb3bd66b0 2023-07-21T22:19:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864c0079a0 con 0x55beb3bd66b0 2023-07-21T22:19:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86500307d0 con 0x55beb3bd66b0 2023-07-21T22:19:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 10 client.16392 fetch_fsmap learned FSMap version 35 2023-07-21T22:19:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55beb3af8250 con 0x55beb3bd66b0 2023-07-21T22:19:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864c0079a0 con 0x55beb3bd66b0 2023-07-21T22:19:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f86500047c0 con 0x55beb3bd66b0 2023-07-21T22:19:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 10 client.16392 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T22:19:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:19:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 20 client.16392 populate_metadata read hostname 'smithi093' 2023-07-21T22:19:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3de8520 0x55beb3dea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 4 client.16392 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:19:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.329+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55beb3c86ff0 con 0x55beb3de8520 2023-07-21T22:19:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.330+0000 7f8659853700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3de8520 0x55beb3dea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.330+0000 7f863dffb700 10 client.16392 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.330+0000 7f8659853700 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3de8520 0x55beb3dea930 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c879e3e852f0eaa server_cookie=81a12eb4f3802e2f in_seq=0 out_seq=0 2023-07-21T22:19:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f863dffb700 1 -- 172.21.15.93:0/2497870799 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55beb3c86ff0 con 0x55beb3de8520 2023-07-21T22:19:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f863dffb700 10 client.16392 handle_command_reply: tid=0 2023-07-21T22:19:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 client.16392 shutdown 2023-07-21T22:19:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 msgr2=0x55beb3d7b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8650011740 tx=0x7f8650010fd0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 shutdown_connections 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55beb3de8520 0x55beb3dea930 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55beb3d80660 0x55beb3d7c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f863dffb700 0 client.16392 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55beb3bd66b0 0x55beb3d7b730 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 --2- 172.21.15.93:0/2497870799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55beb3d80210 0x55beb3d7bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 >> 172.21.15.93:0/2497870799 conn(0x55beb3d6fae0 msgr2=0x55beb3d80aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 shutdown_connections 2023-07-21T22:19:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 1 -- 172.21.15.93:0/2497870799 wait complete. 2023-07-21T22:19:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.365+0000 7f866843db80 20 client.16392 trim_cache size 0 max 16384 2023-07-21T22:19:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:19:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:19:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:19:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:19:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 msgr2=0x7f86400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 0x7f86400a37f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650005950 tx=0x7f865000b410 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 msgr2=0x7f8654100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f864400c920 tx=0x7f864400cc30 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 shutdown_connections 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8654104b40 0x7f86541a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f86540ff080 0x7f8654100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f86400a1340 0x7f86400a37f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 --2- 172.21.15.93:0/2182293010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f86540ffa30 0x7f8654100b60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 >> 172.21.15.93:0/2182293010 conn(0x7f8654074d40 msgr2=0x7f86540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 shutdown_connections 2023-07-21T22:19:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:19:59.366+0000 7f865c2b8700 1 -- 172.21.15.93:0/2182293010 wait complete. 2023-07-21T22:19:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:19:59 smithi093 ceph-mon[96579]: pgmap v110: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-07-21T22:19:59.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:19:59 smithi194 ceph-mon[98288]: pgmap v110: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-07-21T22:19:59.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:19:59 smithi134 ceph-mon[97873]: pgmap v110: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2023-07-21T22:20:00.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 104440 3.24 MB/sec execute 37 sec latency 3143.412 ms 2023-07-21T22:20:00.397 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:19:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:20:00.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:19:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-07-21T22:20:00.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:19:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : mds.f(mds.3): Metadata damage detected 2023-07-21T22:20:00.398 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:19:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : mds.c(mds.2): Metadata damage detected 2023-07-21T22:20:00.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-mon[96579]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:20:00.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-mon[96579]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-07-21T22:20:00.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-mon[96579]: mds.f(mds.3): Metadata damage detected 2023-07-21T22:20:00.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:00 smithi093 ceph-mon[96579]: mds.c(mds.2): Metadata damage detected 2023-07-21T22:20:00.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:00 smithi194 ceph-mon[98288]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:20:00.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:00 smithi194 ceph-mon[98288]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-07-21T22:20:00.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:00 smithi194 ceph-mon[98288]: mds.f(mds.3): Metadata damage detected 2023-07-21T22:20:00.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:00 smithi194 ceph-mon[98288]: mds.c(mds.2): Metadata damage detected 2023-07-21T22:20:00.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:00 smithi134 ceph-mon[97873]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:20:00.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:00 smithi134 ceph-mon[97873]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-07-21T22:20:00.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:00 smithi134 ceph-mon[97873]: mds.f(mds.3): Metadata damage detected 2023-07-21T22:20:00.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:00 smithi134 ceph-mon[97873]: mds.c(mds.2): Metadata damage detected 2023-07-21T22:20:01.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 106435 3.48 MB/sec execute 38 sec latency 24.706 ms 2023-07-21T22:20:01.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:01 smithi093 ceph-mon[96579]: pgmap v111: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T22:20:01.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:01 smithi194 ceph-mon[98288]: pgmap v111: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T22:20:01.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:01 smithi134 ceph-mon[97873]: pgmap v111: 97 pgs: 97 active+clean; 53 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2023-07-21T22:20:02.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107117 3.44 MB/sec execute 39 sec latency 273.076 ms 2023-07-21T22:20:03.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107117 3.35 MB/sec execute 40 sec latency 1273.107 ms 2023-07-21T22:20:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:03 smithi093 ceph-mon[96579]: pgmap v112: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2023-07-21T22:20:03.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:03 smithi194 ceph-mon[98288]: pgmap v112: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2023-07-21T22:20:03.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:03 smithi134 ceph-mon[97873]: pgmap v112: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2023-07-21T22:20:04.317 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107117 3.27 MB/sec execute 41 sec latency 2273.159 ms 2023-07-21T22:20:05.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107173 3.19 MB/sec execute 42 sec latency 2939.544 ms 2023-07-21T22:20:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:05 smithi093 ceph-mon[96579]: pgmap v113: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:05 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:05.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:05 smithi194 ceph-mon[98288]: pgmap v113: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:05.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:05 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:05.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:05 smithi134 ceph-mon[97873]: pgmap v113: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:05.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:05 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:06.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107173 3.12 MB/sec execute 43 sec latency 1295.625 ms 2023-07-21T22:20:07.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107173 3.05 MB/sec execute 44 sec latency 2295.746 ms 2023-07-21T22:20:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:07 smithi093 ceph-mon[96579]: pgmap v114: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:07.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:07 smithi194 ceph-mon[98288]: pgmap v114: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:07.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:07 smithi134 ceph-mon[97873]: pgmap v114: 97 pgs: 97 active+clean; 54 MiB data, 749 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2023-07-21T22:20:08.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107173 2.98 MB/sec execute 45 sec latency 3295.865 ms 2023-07-21T22:20:09.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107173 2.92 MB/sec execute 46 sec latency 4296.001 ms 2023-07-21T22:20:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:09 smithi093 ceph-mon[96579]: pgmap v115: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-07-21T22:20:09.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:09 smithi194 ceph-mon[98288]: pgmap v115: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-07-21T22:20:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:09 smithi134 ceph-mon[97873]: pgmap v115: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2023-07-21T22:20:10.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107241 2.86 MB/sec execute 47 sec latency 4959.085 ms 2023-07-21T22:20:10.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:10 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:10.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:10 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:10.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:10 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:11.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107241 2.80 MB/sec execute 48 sec latency 1289.779 ms 2023-07-21T22:20:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:11 smithi093 ceph-mon[96579]: pgmap v116: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 976 KiB/s wr, 116 op/s 2023-07-21T22:20:11.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:11 smithi194 ceph-mon[98288]: pgmap v116: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 976 KiB/s wr, 116 op/s 2023-07-21T22:20:11.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:11 smithi134 ceph-mon[97873]: pgmap v116: 97 pgs: 97 active+clean; 57 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 976 KiB/s wr, 116 op/s 2023-07-21T22:20:12.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107241 2.74 MB/sec execute 49 sec latency 2289.875 ms 2023-07-21T22:20:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 -- 172.21.15.93:0/1466853969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a0100780 msgr2=0x7f78a0104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 --2- 172.21.15.93:0/1466853969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a0100780 0x7f78a0104740 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f789c0099c0 tx=0x7f789c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 -- 172.21.15.93:0/1466853969 shutdown_connections 2023-07-21T22:20:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 --2- 172.21.15.93:0/1466853969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0104e20 0x7f78a010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 --2- 172.21.15.93:0/1466853969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a00ffde0 0x7f78a01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 --2- 172.21.15.93:0/1466853969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a0100780 0x7f78a0104740 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 -- 172.21.15.93:0/1466853969 >> 172.21.15.93:0/1466853969 conn(0x7f78a00fb6f0 msgr2=0x7f78a00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 -- 172.21.15.93:0/1466853969 shutdown_connections 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.498+0000 7f78a790b700 1 -- 172.21.15.93:0/1466853969 wait complete. 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 Processor -- start 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 -- start start 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0100780 0x7f78a019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a0104e20 0x7f78a0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a01960b0 con 0x7f78a00ffde0 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a0196220 con 0x7f78a0104e20 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a790b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a01963e0 con 0x7f78a0100780 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a56a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a56a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46414/0 (socket says 172.21.15.93:46414) 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a56a7700 1 -- 172.21.15.93:0/1176068862 learned_addr learned my addr 172.21.15.93:0/1176068862 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0100780 0x7f78a019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.499+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a0104e20 0x7f78a0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 -- 172.21.15.93:0/1176068862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0100780 msgr2=0x7f78a019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0100780 0x7f78a019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 -- 172.21.15.93:0/1176068862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a0104e20 msgr2=0x7f78a0195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a0104e20 0x7f78a0195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 -- 172.21.15.93:0/1176068862 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0095f0 con 0x7f78a00ffde0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a56a7700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f789000e9a0 tx=0x7f789000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78967fc700 1 -- 172.21.15.93:0/1176068862 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890009790 con 0x7f78a00ffde0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78967fc700 1 -- 172.21.15.93:0/1176068862 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7890010430 con 0x7f78a00ffde0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a01966c0 con 0x7f78a00ffde0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78967fc700 1 -- 172.21.15.93:0/1176068862 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890018680 con 0x7f78a00ffde0 2023-07-21T22:20:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.500+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a01a2bd0 con 0x7f78a00ffde0 2023-07-21T22:20:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.503+0000 7f78967fc700 1 -- 172.21.15.93:0/1176068862 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f78900105a0 con 0x7f78a00ffde0 2023-07-21T22:20:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.503+0000 7f78967fc700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 0x7f788c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.503+0000 7f78967fc700 1 -- 172.21.15.93:0/1176068862 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7890014070 con 0x7f78a00ffde0 2023-07-21T22:20:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.503+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 0x7f788c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.504+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 0x7f788c0a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098e0 tx=0x7f789c000bc0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 Processor -- start 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 -- start start 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617372c6210 0x5617372bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617371c5c50 con 0x56173711c6b0 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372189e0 con 0x5617372c6660 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561737211e70 con 0x5617372c6210 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78a56a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78a56a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46422/0 (socket says 172.21.15.93:46422) 2023-07-21T22:20:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78a56a7700 1 -- 172.21.15.93:0/880598803 learned_addr learned my addr 172.21.15.93:0/880598803 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617372c6210 0x5617372bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.505+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 -- 172.21.15.93:0/880598803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617372c6210 msgr2=0x5617372bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617372c6210 0x5617372bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 -- 172.21.15.93:0/880598803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 msgr2=0x5617372c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 -- 172.21.15.93:0/880598803 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173703fd80 con 0x56173711c6b0 2023-07-21T22:20:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f78a56a7700 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f78900a1ed0 tx=0x7f7890099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38415fb5b718710 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f788b7fe700 1 -- 172.21.15.93:0/880598803 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890010880 con 0x56173711c6b0 2023-07-21T22:20:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f788b7fe700 1 -- 172.21.15.93:0/880598803 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78900109e0 con 0x56173711c6b0 2023-07-21T22:20:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.506+0000 7f788b7fe700 1 -- 172.21.15.93:0/880598803 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900a1f70 con 0x56173711c6b0 2023-07-21T22:20:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- 172.21.15.93:0/880598803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 msgr2=0x5617372c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f78900a1ed0 tx=0x7f7890099fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- 172.21.15.93:0/880598803 shutdown_connections 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56173711c6b0 0x5617372c5cd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/880598803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5617372c6210 0x5617372bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- 172.21.15.93:0/880598803 >> 172.21.15.93:0/880598803 conn(0x5617372b5ae0 msgr2=0x5617372bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- 172.21.15.93:0/880598803 shutdown_connections 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- 172.21.15.93:0/880598803 wait complete. 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 -- start start 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.507+0000 7f78b3a90b80 1 Processor -- start 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617371b1a60 con 0x5617372c6210 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617371b1760 con 0x5617372c6660 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617370958c0 con 0x56173711c6b0 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78b3a90b80 10 client.0 _ll_register_callbacks cb 0x5617371af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78a4ea6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78a4ea6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46430/0 (socket says 172.21.15.93:46430) 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78a4ea6700 1 -- 172.21.15.93:0/4243724775 learned_addr learned my addr 172.21.15.93:0/4243724775 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:12.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78a56a7700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.508+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 -- 172.21.15.93:0/4243724775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 msgr2=0x5617372c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 -- 172.21.15.93:0/4243724775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 msgr2=0x5617372c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 -- 172.21.15.93:0/4243724775 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a01a2bd0 con 0x56173711c6b0 2023-07-21T22:20:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78a56a7700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372cc5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78900a1ed0 tx=0x7f7890099750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc1b14477b40df2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f7888ff9700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:20:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900b1070 con 0x56173711c6b0 2023-07-21T22:20:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56173716b490 con 0x56173711c6b0 2023-07-21T22:20:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f7890098350 con 0x56173711c6b0 2023-07-21T22:20:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7890010880 con 0x56173711c6b0 2023-07-21T22:20:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78b3a90b80 10 client.25792 fetch_fsmap learned FSMap version 37 2023-07-21T22:20:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.509+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561737028c00 con 0x56173711c6b0 2023-07-21T22:20:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78900c8080 con 0x56173711c6b0 2023-07-21T22:20:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7890094c50 con 0x56173711c6b0 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 10 client.25792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 20 client.25792 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5617372d96d0 0x5617372dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 4 client.25792 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56173720ae50 con 0x5617372d96d0 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.510+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5617372d96d0 0x5617372dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.511+0000 7f7888ff9700 10 client.25792 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.511+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5617372d96d0 0x5617372dbab0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8986a728a87863d1 server_cookie=f47f20783a60d943 in_seq=0 out_seq=0 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.511+0000 7f7888ff9700 1 -- 172.21.15.93:0/4243724775 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56173720ae50 con 0x5617372d96d0 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.511+0000 7f7888ff9700 10 client.25792 handle_command_reply: tid=0 2023-07-21T22:20:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.511+0000 7f78b3a90b80 1 client.25792 shutdown 2023-07-21T22:20:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 msgr2=0x5617372cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372cc5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f78900a1ed0 tx=0x7f7890099750 comp rx=0 tx=0).stop 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 shutdown_connections 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5617372d96d0 0x5617372dbab0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372cc5d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/4243724775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f7888ff9700 0 client.25792 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 >> 172.21.15.93:0/4243724775 conn(0x5617372b5ae0 msgr2=0x5617372ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 shutdown_connections 2023-07-21T22:20:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 1 -- 172.21.15.93:0/4243724775 wait complete. 2023-07-21T22:20:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.512+0000 7f78b3a90b80 20 client.25792 trim_cache size 0 max 16384 2023-07-21T22:20:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 Processor -- start 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 -- start start 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6210 0x5617372bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372c1190 con 0x5617372c6660 2023-07-21T22:20:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372c0f20 con 0x5617372c6210 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372bbe50 con 0x56173711c6b0 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78a5ea8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78a5ea8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46438/0 (socket says 172.21.15.93:46438) 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78a5ea8700 1 -- 172.21.15.93:0/3980696006 learned_addr learned my addr 172.21.15.93:0/3980696006 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78a56a7700 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.528+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6210 0x5617372bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 -- 172.21.15.93:0/3980696006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 msgr2=0x5617372d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 -- 172.21.15.93:0/3980696006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6210 msgr2=0x5617372bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6210 0x5617372bab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 -- 172.21.15.93:0/3980696006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617370661f0 con 0x5617372c6660 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7898007630 tx=0x7f789800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1198f1925a1a64 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f788bfff700 1 -- 172.21.15.93:0/3980696006 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800e040 con 0x5617372c6660 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f788bfff700 1 -- 172.21.15.93:0/3980696006 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789800f040 con 0x5617372c6660 2023-07-21T22:20:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.529+0000 7f788bfff700 1 -- 172.21.15.93:0/3980696006 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898017650 con 0x5617372c6660 2023-07-21T22:20:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- 172.21.15.93:0/3980696006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 msgr2=0x5617372c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7898007630 tx=0x7f789800b040 comp rx=0 tx=0).stop 2023-07-21T22:20:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- 172.21.15.93:0/3980696006 shutdown_connections 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6660 0x5617372c7330 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/3980696006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6210 0x5617372bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- 172.21.15.93:0/3980696006 >> 172.21.15.93:0/3980696006 conn(0x5617372b5ae0 msgr2=0x5617372c4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- 172.21.15.93:0/3980696006 shutdown_connections 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- 172.21.15.93:0/3980696006 wait complete. 2023-07-21T22:20:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 -- start start 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.530+0000 7f78b3a90b80 1 Processor -- start 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372cfb70 con 0x5617372c6210 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617372cfce0 con 0x5617372c6660 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173732d570 con 0x56173711c6b0 2023-07-21T22:20:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78b3a90b80 10 client.0 _ll_register_callbacks cb 0x5617371af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46444/0 (socket says 172.21.15.93:46444) 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 -- 172.21.15.93:0/229454458 learned_addr learned my addr 172.21.15.93:0/229454458 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a5ea8700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a56a7700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 -- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 msgr2=0x5617372d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 -- 172.21.15.93:0/229454458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 msgr2=0x5617372cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 -- 172.21.15.93:0/229454458 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78980092c0 con 0x5617372c6210 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a56a7700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f78a4ea6700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f789c00b580 tx=0x7f789c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2e44e468bea24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.531+0000 7f7888ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02c070 con 0x5617372c6210 2023-07-21T22:20:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f789c033bd0 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78900109e0 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789c033d90 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 10 client.16410 fetch_fsmap learned FSMap version 37 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x56173703e250 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c041080 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 37) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f789c00f7d0 con 0x5617372c6210 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T22:20:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 20 client.16410 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56173732ec60 0x561737331070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 4 client.16410 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.532+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617371ccff0 con 0x56173732ec60 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.533+0000 7f78a56a7700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56173732ec60 0x561737331070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.533+0000 7f7888ff9700 10 client.16410 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.533+0000 7f78a56a7700 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56173732ec60 0x561737331070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6e5ab24e515a0e server_cookie=9c6763129d97c878 in_seq=0 out_seq=0 2023-07-21T22:20:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f7888ff9700 1 -- 172.21.15.93:0/229454458 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617371ccff0 con 0x56173732ec60 2023-07-21T22:20:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f7888ff9700 10 client.16410 handle_command_reply: tid=0 2023-07-21T22:20:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 client.16410 shutdown 2023-07-21T22:20:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 msgr2=0x5617372cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f789c00b580 tx=0x7f789c01f000 comp rx=0 tx=0).stop 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 shutdown_connections 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56173732ec60 0x561737331070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617372c6660 0x5617372cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56173711c6b0 0x5617372d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f7888ff9700 0 client.16410 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 --2- 172.21.15.93:0/229454458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5617372c6210 0x5617372cef60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 >> 172.21.15.93:0/229454458 conn(0x5617372b5ae0 msgr2=0x5617372c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 shutdown_connections 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.563+0000 7f78b3a90b80 1 -- 172.21.15.93:0/229454458 wait complete. 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78b3a90b80 20 client.16410 trim_cache size 0 max 16384 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:20:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:20:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:20:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:20:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 msgr2=0x7f788c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 0x7f788c0a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f789c0098e0 tx=0x7f789c000bc0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 msgr2=0x7f78a019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f789000e9a0 tx=0x7f789000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 shutdown_connections 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a0104e20 0x7f78a0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f788c0a12c0 0x7f788c0a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a00ffde0 0x7f78a019bbe0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 --2- 172.21.15.93:0/1176068862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78a0100780 0x7f78a019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 >> 172.21.15.93:0/1176068862 conn(0x7f78a00fb6f0 msgr2=0x7f78a0103760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 shutdown_connections 2023-07-21T22:20:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:12.564+0000 7f78a790b700 1 -- 172.21.15.93:0/1176068862 wait complete. 2023-07-21T22:20:13.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107241 2.69 MB/sec execute 50 sec latency 3289.960 ms 2023-07-21T22:20:13.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:13 smithi093 ceph-mon[96579]: pgmap v117: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:20:13.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:13 smithi194 ceph-mon[98288]: pgmap v117: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:20:13.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:13 smithi134 ceph-mon[97873]: pgmap v117: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2023-07-21T22:20:14.318 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107241 2.63 MB/sec execute 51 sec latency 4290.049 ms 2023-07-21T22:20:15.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 107943 2.67 MB/sec execute 52 sec latency 4955.559 ms 2023-07-21T22:20:15.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:15 smithi093 ceph-mon[96579]: pgmap v118: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:15.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:15 smithi194 ceph-mon[98288]: pgmap v118: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:15.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:15 smithi134 ceph-mon[97873]: pgmap v118: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:16.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110300 2.88 MB/sec execute 53 sec latency 10.161 ms 2023-07-21T22:20:17.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110667 2.84 MB/sec execute 54 sec latency 596.511 ms 2023-07-21T22:20:17.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:17 smithi194 ceph-mon[98288]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:17.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:17 smithi134 ceph-mon[97873]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:17.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:17 smithi093 ceph-mon[96579]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 665 KiB/s wr, 43 op/s 2023-07-21T22:20:18.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110667 2.78 MB/sec execute 55 sec latency 1596.610 ms 2023-07-21T22:20:19.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110667 2.73 MB/sec execute 56 sec latency 2596.675 ms 2023-07-21T22:20:19.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:19 smithi194 ceph-mon[98288]: pgmap v120: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1021 KiB/s wr, 94 op/s 2023-07-21T22:20:19.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:19 smithi134 ceph-mon[97873]: pgmap v120: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1021 KiB/s wr, 94 op/s 2023-07-21T22:20:19.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:19 smithi093 ceph-mon[96579]: pgmap v120: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1021 KiB/s wr, 94 op/s 2023-07-21T22:20:20.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110722 2.69 MB/sec execute 57 sec latency 3287.291 ms 2023-07-21T22:20:21.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110722 2.64 MB/sec execute 58 sec latency 1281.105 ms 2023-07-21T22:20:21.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:21 smithi194 ceph-mon[98288]: pgmap v121: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 645 KiB/s wr, 74 op/s 2023-07-21T22:20:21.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:21 smithi134 ceph-mon[97873]: pgmap v121: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 645 KiB/s wr, 74 op/s 2023-07-21T22:20:21.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:21 smithi093 ceph-mon[96579]: pgmap v121: 97 pgs: 97 active+clean; 59 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 645 KiB/s wr, 74 op/s 2023-07-21T22:20:22.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110722 2.60 MB/sec execute 59 sec latency 2281.179 ms 2023-07-21T22:20:23.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110723 2.55 MB/sec execute 60 sec latency 2798.697 ms 2023-07-21T22:20:23.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:23 smithi194 ceph-mon[98288]: pgmap v122: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 936 KiB/s wr, 97 op/s 2023-07-21T22:20:23.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:23 smithi134 ceph-mon[97873]: pgmap v122: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 936 KiB/s wr, 97 op/s 2023-07-21T22:20:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:23 smithi093 ceph-mon[96579]: pgmap v122: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 936 KiB/s wr, 97 op/s 2023-07-21T22:20:24.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110723 2.51 MB/sec execute 61 sec latency 1482.685 ms 2023-07-21T22:20:25.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110723 2.47 MB/sec execute 62 sec latency 2482.744 ms 2023-07-21T22:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.680+0000 7f4e18db4700 1 -- 172.21.15.93:0/3753333137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e14104a30 msgr2=0x7f4e14104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.680+0000 7f4e18db4700 1 --2- 172.21.15.93:0/3753333137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e14104a30 0x7f4e14104e00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009990 tx=0x7f4e04009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 -- 172.21.15.93:0/3753333137 shutdown_connections 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 --2- 172.21.15.93:0/3753333137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e1410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 --2- 172.21.15.93:0/3753333137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e14104a30 0x7f4e14104e00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 --2- 172.21.15.93:0/3753333137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e140fea30 0x7f4e140feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 -- 172.21.15.93:0/3753333137 >> 172.21.15.93:0/3753333137 conn(0x7f4e14074760 msgr2=0x7f4e14074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 -- 172.21.15.93:0/3753333137 shutdown_connections 2023-07-21T22:20:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 -- 172.21.15.93:0/3753333137 wait complete. 2023-07-21T22:20:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 Processor -- start 2023-07-21T22:20:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.681+0000 7f4e18db4700 1 -- start start 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e141970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 0x7f4e141a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a2040 con 0x7f4e140fea30 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a21b0 con 0x7f4e14104a30 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141a2370 con 0x7f4e140ff3e0 2023-07-21T22:20:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46456/0 (socket says 172.21.15.93:46456) 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 -- 172.21.15.93:0/4292696409 learned_addr learned my addr 172.21.15.93:0/4292696409 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e141970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 0x7f4e141a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 -- 172.21.15.93:0/4292696409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 msgr2=0x7f4e141970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e141970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 -- 172.21.15.93:0/4292696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 msgr2=0x7f4e141a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 0x7f4e141a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 -- 172.21.15.93:0/4292696409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040095f0 con 0x7f4e140fea30 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e141970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 0x7f4e141a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e1259c700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004730 tx=0x7f4e04005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e037fe700 1 -- 172.21.15.93:0/4292696409 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0401d070 con 0x7f4e140fea30 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e037fe700 1 -- 172.21.15.93:0/4292696409 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e04004cd0 con 0x7f4e140fea30 2023-07-21T22:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e141a25f0 con 0x7f4e140fea30 2023-07-21T22:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e037fe700 1 -- 172.21.15.93:0/4292696409 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04005540 con 0x7f4e140fea30 2023-07-21T22:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.682+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e141a29b0 con 0x7f4e140fea30 2023-07-21T22:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.684+0000 7f4e037fe700 1 -- 172.21.15.93:0/4292696409 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e0400fbc0 con 0x7f4e140fea30 2023-07-21T22:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.685+0000 7f4e037fe700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 0x7f4dfc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.685+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 0x7f4dfc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.685+0000 7f4e037fe700 1 -- 172.21.15.93:0/4292696409 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4e040cdfa0 con 0x7f4e140fea30 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 0x7f4dfc0a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e14068420 tx=0x7f4e08009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 Processor -- start 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 -- start start 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a2f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a29350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b392ac50 con 0x55d3b3a2fa70 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b397d9e0 con 0x55d3b38816b0 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3976e70 con 0x55d3b3a2f620 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.686+0000 7f4e12d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46468/0 (socket says 172.21.15.93:46468) 2023-07-21T22:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 -- 172.21.15.93:0/1384918157 learned_addr learned my addr 172.21.15.93:0/1384918157 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e1259c700 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a2f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 -- 172.21.15.93:0/1384918157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 msgr2=0x55d3b3a29350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a29350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 -- 172.21.15.93:0/1384918157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 msgr2=0x55d3b3a2f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a2f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 -- 172.21.15.93:0/1384918157 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b37a4d80 con 0x55d3b3a2fa70 2023-07-21T22:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009d50 tx=0x7f4e0c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a3524b5452c42c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e00ff9700 1 -- 172.21.15.93:0/1384918157 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c009ea0 con 0x55d3b3a2fa70 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e00ff9700 1 -- 172.21.15.93:0/1384918157 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0c014ac0 con 0x55d3b3a2fa70 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.687+0000 7f4e00ff9700 1 -- 172.21.15.93:0/1384918157 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c013aa0 con 0x55d3b3a2fa70 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1384918157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 msgr2=0x55d3b3a298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009d50 tx=0x7f4e0c0077e0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1384918157 shutdown_connections 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a298b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a2f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1384918157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a29350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1384918157 >> 172.21.15.93:0/1384918157 conn(0x55d3b3a1aae0 msgr2=0x55d3b3a200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1384918157 shutdown_connections 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1384918157 wait complete. 2023-07-21T22:20:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 -- start start 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.688+0000 7f4e24f39b80 1 Processor -- start 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a33230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3916a60 con 0x55d3b38816b0 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3916760 con 0x55d3b3a2fa70 2023-07-21T22:20:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b37fa8c0 con 0x55d3b3a2f620 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 10 client.0 _ll_register_callbacks cb 0x55d3b3914700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46470/0 (socket says 172.21.15.93:46470) 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 -- 172.21.15.93:0/1551612701 learned_addr learned my addr 172.21.15.93:0/1551612701 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a33230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 -- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 msgr2=0x55d3b3a33230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a33230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 -- 172.21.15.93:0/1551612701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 msgr2=0x55d3b3a33770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 -- 172.21.15.93:0/1551612701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c009980 con 0x55d3b38816b0 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e1259c700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4e04000c00 tx=0x7f4e0409f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2152ff4d1d8d99 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0409a810 con 0x55d3b38816b0 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4e040a3050 con 0x55d3b38816b0 2023-07-21T22:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.689+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3b38d0490 con 0x55d3b38816b0 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e040a3210 con 0x55d3b38816b0 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 10 client.16422 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d3b378dc00 con 0x55d3b38816b0 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3b38d0490 con 0x55d3b38816b0 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e0409a110 con 0x55d3b38816b0 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 20 client.16422 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a42bf0 0x55d3b3a44fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 4 client.16422 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3b396fe50 con 0x55d3b3a42bf0 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.690+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a42bf0 0x55d3b3a44fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4df9ffb700 10 client.16422 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a42bf0 0x55d3b3a44fd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe834ddc430022c server_cookie=d8f7935967fad36a in_seq=0 out_seq=0 2023-07-21T22:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4df9ffb700 1 -- 172.21.15.93:0/1551612701 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3b396fe50 con 0x55d3b3a42bf0 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4df9ffb700 10 client.16422 handle_command_reply: tid=0 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e24f39b80 1 client.16422 shutdown 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 msgr2=0x55d3b3a32cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4e04000c00 tx=0x7f4e0409f280 comp rx=0 tx=0).stop 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 shutdown_connections 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.691+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a42bf0 0x55d3b3a44fd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4df9ffb700 0 client.16422 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b38816b0 0x55d3b3a32cf0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/1551612701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2f620 0x55d3b3a33230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 >> 172.21.15.93:0/1551612701 conn(0x55d3b3a1aae0 msgr2=0x55d3b3a23f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 shutdown_connections 2023-07-21T22:20:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 1 -- 172.21.15.93:0/1551612701 wait complete. 2023-07-21T22:20:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.692+0000 7f4e24f39b80 20 client.16422 trim_cache size 0 max 16384 2023-07-21T22:20:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.706+0000 7f4e24f39b80 1 Processor -- start 2023-07-21T22:20:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.706+0000 7f4e24f39b80 1 -- start start 2023-07-21T22:20:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.706+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a39fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a29d60 con 0x55d3b3a2f620 2023-07-21T22:20:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a3a4f0 con 0x55d3b38816b0 2023-07-21T22:20:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a2e9b0 con 0x55d3b3a2fa70 2023-07-21T22:20:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46480/0 (socket says 172.21.15.93:46480) 2023-07-21T22:20:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 -- 172.21.15.93:0/4161776037 learned_addr learned my addr 172.21.15.93:0/4161776037 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e1259c700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a39fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 -- 172.21.15.93:0/4161776037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2fa70 msgr2=0x55d3b3a335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a335a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 -- 172.21.15.93:0/4161776037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 msgr2=0x55d3b3a39fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a39fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 -- 172.21.15.93:0/4161776037 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3b3a49c00 con 0x55d3b3a2f620 2023-07-21T22:20:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e1259c700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a39fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d3b3a343a0 tx=0x7f4e08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edc9ba74f6d1cd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e017fa700 1 -- 172.21.15.93:0/4161776037 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08026040 con 0x55d3b3a2f620 2023-07-21T22:20:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e017fa700 1 -- 172.21.15.93:0/4161776037 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e08028570 con 0x55d3b3a2f620 2023-07-21T22:20:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.707+0000 7f4e017fa700 1 -- 172.21.15.93:0/4161776037 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08003ea0 con 0x55d3b3a2f620 2023-07-21T22:20:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- 172.21.15.93:0/4161776037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 msgr2=0x55d3b3a2e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d3b3a343a0 tx=0x7f4e08016000 comp rx=0 tx=0).stop 2023-07-21T22:20:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- 172.21.15.93:0/4161776037 shutdown_connections 2023-07-21T22:20:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b38816b0 0x55d3b3a39fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/4161776037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a2e470 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- 172.21.15.93:0/4161776037 >> 172.21.15.93:0/4161776037 conn(0x55d3b3a1aae0 msgr2=0x55d3b3a2dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- 172.21.15.93:0/4161776037 shutdown_connections 2023-07-21T22:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- 172.21.15.93:0/4161776037 wait complete. 2023-07-21T22:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 -- start start 2023-07-21T22:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.708+0000 7f4e24f39b80 1 Processor -- start 2023-07-21T22:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b38816b0 0x55d3b3a39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a364d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a36a10 con 0x55d3b3a2f620 2023-07-21T22:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a36b80 con 0x55d3b3a2fa70 2023-07-21T22:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3b3a36d40 con 0x55d3b38816b0 2023-07-21T22:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 10 client.0 _ll_register_callbacks cb 0x7f4e0409b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46486/0 (socket says 172.21.15.93:46486) 2023-07-21T22:20:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 -- 172.21.15.93:0/14761147 learned_addr learned my addr 172.21.15.93:0/14761147 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e1259c700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b38816b0 0x55d3b3a39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 -- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b38816b0 msgr2=0x55d3b3a39ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b38816b0 0x55d3b3a39ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 -- 172.21.15.93:0/14761147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 msgr2=0x55d3b3a364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 -- 172.21.15.93:0/14761147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e08010950 con 0x55d3b3a2f620 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e12d9d700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e11d9b700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015040 tx=0x7f4e08028480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda649eee9e97bcf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08003e20 con 0x55d3b3a2f620 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f4e08005050 con 0x55d3b3a2f620 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.709+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e0c014ac0 con 0x55d3b3a2f620 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e08005250 con 0x55d3b3a2f620 2023-07-21T22:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 10 client.16428 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d3b37cb1f0 con 0x55d3b3a2f620 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0c014ac0 con 0x55d3b3a2f620 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4e08028700 con 0x55d3b3a2f620 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 20 client.16428 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a97d60 0x55d3b3a9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 4 client.16428 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d3b3931ff0 con 0x55d3b3a97d60 2023-07-21T22:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.710+0000 7f4e1259c700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a97d60 0x55d3b3a9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.711+0000 7f4df9ffb700 10 client.16428 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.711+0000 7f4e1259c700 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a97d60 0x55d3b3a9a170 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a85c2137d14057 server_cookie=6d5dee90ae3c9fb2 in_seq=0 out_seq=0 2023-07-21T22:20:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4df9ffb700 1 -- 172.21.15.93:0/14761147 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d3b3931ff0 con 0x55d3b3a97d60 2023-07-21T22:20:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4df9ffb700 10 client.16428 handle_command_reply: tid=0 2023-07-21T22:20:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 client.16428 shutdown 2023-07-21T22:20:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 msgr2=0x55d3b3a35f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015040 tx=0x7f4e08028480 comp rx=0 tx=0).stop 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 shutdown_connections 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d3b3a97d60 0x55d3b3a9a170 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d3b3a2fa70 0x55d3b3a364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d3b38816b0 0x55d3b3a39ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 --2- 172.21.15.93:0/14761147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d3b3a2f620 0x55d3b3a35f90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4df9ffb700 0 client.16428 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 >> 172.21.15.93:0/14761147 conn(0x55d3b3a1aae0 msgr2=0x55d3b3a33280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 shutdown_connections 2023-07-21T22:20:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 1 -- 172.21.15.93:0/14761147 wait complete. 2023-07-21T22:20:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e24f39b80 20 client.16428 trim_cache size 0 max 16384 2023-07-21T22:20:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:20:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:20:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:20:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 msgr2=0x7f4dfc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 0x7f4dfc0a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e14068420 tx=0x7f4e08009260 comp rx=0 tx=0).stop 2023-07-21T22:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 msgr2=0x7f4e14196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.742+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4e04004730 tx=0x7f4e04005ef0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 shutdown_connections 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e14104a30 0x7f4e141a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4dfc0a12c0 0x7f4dfc0a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e140fea30 0x7f4e14196ba0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 --2- 172.21.15.93:0/4292696409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e140ff3e0 0x7f4e141970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 >> 172.21.15.93:0/4292696409 conn(0x7f4e14074760 msgr2=0x7f4e140fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 shutdown_connections 2023-07-21T22:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:25.743+0000 7f4e18db4700 1 -- 172.21.15.93:0/4292696409 wait complete. 2023-07-21T22:20:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:25 smithi194 ceph-mon[98288]: pgmap v123: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:25.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:25 smithi134 ceph-mon[97873]: pgmap v123: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:25.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:25 smithi093 ceph-mon[96579]: pgmap v123: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:26.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110723 2.43 MB/sec execute 63 sec latency 3482.818 ms 2023-07-21T22:20:26.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:26 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:26.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:26 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:26.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:26 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:20:27.319 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110723 2.39 MB/sec execute 64 sec latency 4482.862 ms 2023-07-21T22:20:27.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:27 smithi194 ceph-mon[98288]: pgmap v124: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:27.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:27 smithi134 ceph-mon[97873]: pgmap v124: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:27.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:27 smithi093 ceph-mon[96579]: pgmap v124: 97 pgs: 97 active+clean; 59 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 646 KiB/s wr, 74 op/s 2023-07-21T22:20:28.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110790 2.36 MB/sec execute 65 sec latency 4994.855 ms 2023-07-21T22:20:28.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:28 smithi194 ceph-mon[98288]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 920 KiB/s wr, 101 op/s 2023-07-21T22:20:28.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:28 smithi134 ceph-mon[97873]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 920 KiB/s wr, 101 op/s 2023-07-21T22:20:28.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:28 smithi093 ceph-mon[96579]: pgmap v125: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 920 KiB/s wr, 101 op/s 2023-07-21T22:20:29.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110790 2.32 MB/sec execute 66 sec latency 1450.708 ms 2023-07-21T22:20:30.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110790 2.29 MB/sec execute 67 sec latency 2450.833 ms 2023-07-21T22:20:31.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110790 2.26 MB/sec execute 68 sec latency 3450.929 ms 2023-07-21T22:20:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:31 smithi093 ceph-mon[96579]: pgmap v126: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 563 KiB/s wr, 50 op/s 2023-07-21T22:20:31.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:31 smithi194 ceph-mon[98288]: pgmap v126: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 563 KiB/s wr, 50 op/s 2023-07-21T22:20:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:31 smithi134 ceph-mon[97873]: pgmap v126: 97 pgs: 97 active+clean; 56 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 563 KiB/s wr, 50 op/s 2023-07-21T22:20:32.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 110790 2.22 MB/sec execute 69 sec latency 4451.031 ms 2023-07-21T22:20:33.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 111695 2.26 MB/sec execute 70 sec latency 4969.729 ms 2023-07-21T22:20:33.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:33 smithi093 ceph-mon[96579]: pgmap v127: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 826 KiB/s wr, 66 op/s 2023-07-21T22:20:33.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:33 smithi194 ceph-mon[98288]: pgmap v127: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 826 KiB/s wr, 66 op/s 2023-07-21T22:20:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:33 smithi134 ceph-mon[97873]: pgmap v127: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 826 KiB/s wr, 66 op/s 2023-07-21T22:20:34.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 113781 2.42 MB/sec execute 71 sec latency 18.593 ms 2023-07-21T22:20:34.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:34 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:34 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:34 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:35.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114210 2.39 MB/sec execute 72 sec latency 396.503 ms 2023-07-21T22:20:35.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:35 smithi093 ceph-mon[96579]: pgmap v128: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:35.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:35.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:35.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:20:35.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:35 smithi194 ceph-mon[98288]: pgmap v128: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:20:35.629 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:35 smithi134 ceph-mon[97873]: pgmap v128: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:20:36.320 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114210 2.36 MB/sec execute 73 sec latency 1396.620 ms 2023-07-21T22:20:37.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114210 2.33 MB/sec execute 74 sec latency 2396.739 ms 2023-07-21T22:20:37.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:37 smithi093 ceph-mon[96579]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:37.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:37 smithi194 ceph-mon[98288]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:37.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:37 smithi134 ceph-mon[97873]: pgmap v129: 97 pgs: 97 active+clean; 55 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 536 KiB/s wr, 43 op/s 2023-07-21T22:20:38.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114266 2.30 MB/sec execute 75 sec latency 2911.341 ms 2023-07-21T22:20:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 -- 172.21.15.93:0/3620714249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4300ffab0 msgr2=0x7fd43010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 --2- 172.21.15.93:0/3620714249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4300ffab0 0x7fd43010b840 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd42800b0d0 tx=0x7fd42800b3e0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 -- 172.21.15.93:0/3620714249 shutdown_connections 2023-07-21T22:20:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 --2- 172.21.15.93:0/3620714249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4300ffab0 0x7fd43010b840 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 --2- 172.21.15.93:0/3620714249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4301039b0 0x7fd430103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 --2- 172.21.15.93:0/3620714249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 0x7fd4300ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 -- 172.21.15.93:0/3620714249 >> 172.21.15.93:0/3620714249 conn(0x7fd430075f90 msgr2=0x7fd430076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 -- 172.21.15.93:0/3620714249 shutdown_connections 2023-07-21T22:20:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.859+0000 7fd437197700 1 -- 172.21.15.93:0/3620714249 wait complete. 2023-07-21T22:20:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 Processor -- start 2023-07-21T22:20:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 -- start start 2023-07-21T22:20:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 0x7fd43019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4300ffab0 0x7fd43019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430195040 con 0x7fd4301039b0 2023-07-21T22:20:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4301951b0 con 0x7fd4300ffab0 2023-07-21T22:20:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd437197700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430195370 con 0x7fd4300ff100 2023-07-21T22:20:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46500/0 (socket says 172.21.15.93:46500) 2023-07-21T22:20:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 -- 172.21.15.93:0/768182677 learned_addr learned my addr 172.21.15.93:0/768182677 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd42ffff700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4300ffab0 0x7fd43019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd434f33700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 0x7fd43019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 -- 172.21.15.93:0/768182677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 msgr2=0x7fd43019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 0x7fd43019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 -- 172.21.15.93:0/768182677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4300ffab0 msgr2=0x7fd43019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4300ffab0 0x7fd43019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.860+0000 7fd435734700 1 -- 172.21.15.93:0/768182677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009d00 con 0x7fd4301039b0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd435734700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d010 tx=0x7fd428007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd42dffb700 1 -- 172.21.15.93:0/768182677 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800e040 con 0x7fd4301039b0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4301955f0 con 0x7fd4301039b0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd42dffb700 1 -- 172.21.15.93:0/768182677 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd428003e10 con 0x7fd4301039b0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd42dffb700 1 -- 172.21.15.93:0/768182677 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42801ba50 con 0x7fd4301039b0 2023-07-21T22:20:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.861+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4301a1b60 con 0x7fd4301039b0 2023-07-21T22:20:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.863+0000 7fd42dffb700 1 -- 172.21.15.93:0/768182677 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd428012470 con 0x7fd4301039b0 2023-07-21T22:20:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.864+0000 7fd42dffb700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 0x7fd4200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.864+0000 7fd434f33700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 0x7fd4200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.864+0000 7fd42dffb700 1 -- 172.21.15.93:0/768182677 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd4280ce3f0 con 0x7fd4301039b0 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd434f33700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 0x7fd4200a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd424005950 tx=0x7fd4240058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 Processor -- start 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 -- start start 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7870 0x561e2f2e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f1e4c50 con 0x561e2f13b6b0 2023-07-21T22:20:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2379e0 con 0x561e2f2e7870 2023-07-21T22:20:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd44331cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f230e70 con 0x561e2f2e7420 2023-07-21T22:20:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.865+0000 7fd434f33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46504/0 (socket says 172.21.15.93:46504) 2023-07-21T22:20:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 -- 172.21.15.93:0/2186489541 learned_addr learned my addr 172.21.15.93:0/2186489541 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd42ffff700 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd435734700 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7870 0x561e2f2e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 -- 172.21.15.93:0/2186489541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7420 msgr2=0x561e2f2e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 -- 172.21.15.93:0/2186489541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7870 msgr2=0x561e2f2e16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7870 0x561e2f2e16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 -- 172.21.15.93:0/2186489541 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2f05ed80 con 0x561e2f13b6b0 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd434f33700 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd42400fcd0 tx=0x7fd424012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d6aaddd38018f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd41affd700 1 -- 172.21.15.93:0/2186489541 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424011c40 con 0x561e2f13b6b0 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd41affd700 1 -- 172.21.15.93:0/2186489541 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424011da0 con 0x561e2f13b6b0 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.866+0000 7fd41affd700 1 -- 172.21.15.93:0/2186489541 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42401d610 con 0x561e2f13b6b0 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- 172.21.15.93:0/2186489541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 msgr2=0x561e2f2e6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd42400fcd0 tx=0x7fd424012000 comp rx=0 tx=0).stop 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- 172.21.15.93:0/2186489541 shutdown_connections 2023-07-21T22:20:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7870 0x561e2f2e16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2e6ee0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 --2- 172.21.15.93:0/2186489541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- 172.21.15.93:0/2186489541 >> 172.21.15.93:0/2186489541 conn(0x561e2f2d4ae0 msgr2=0x561e2f2da0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- 172.21.15.93:0/2186489541 shutdown_connections 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- 172.21.15.93:0/2186489541 wait complete. 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 -- start start 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.867+0000 7fd44331cb80 1 Processor -- start 2023-07-21T22:20:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f13b6b0 0x561e2f2ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f1d0a60 con 0x561e2f2e7420 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f1d0760 con 0x561e2f13b6b0 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f0b48c0 con 0x561e2f2e7870 2023-07-21T22:20:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 10 client.0 _ll_register_callbacks cb 0x561e2f1ce700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46512/0 (socket says 172.21.15.93:46512) 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 -- 172.21.15.93:0/3799533126 learned_addr learned my addr 172.21.15.93:0/3799533126 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd435734700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd434f33700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f13b6b0 0x561e2f2ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 -- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 msgr2=0x561e2f2eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 -- 172.21.15.93:0/3799533126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f13b6b0 msgr2=0x561e2f2ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f13b6b0 0x561e2f2ed740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 -- 172.21.15.93:0/3799533126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42400f980 con 0x561e2f2e7420 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd435734700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd42ffff700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd41c007fa0 tx=0x7fd41c0127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f9b72340e9ec49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd40ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c010870 con 0x561e2f2e7420 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd41c0109d0 con 0x561e2f2e7420 2023-07-21T22:20:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.868+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e2f18a490 con 0x561e2f2e7420 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41c010b90 con 0x561e2f2e7420 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 10 client.16440 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561e2f047c00 con 0x561e2f2e7420 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e2f18a490 con 0x561e2f2e7420 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd41c0233e0 con 0x561e2f2e7420 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 10 client.16440 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 20 client.16440 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f2fad60 0x561e2f2fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 4 client.16440 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e2f229e50 con 0x561e2f2fad60 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.869+0000 7fd434f33700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f2fad60 0x561e2f2fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.870+0000 7fd40ffff700 10 client.16440 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.870+0000 7fd434f33700 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f2fad60 0x561e2f2fd170 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4eedb8c0767e7d8 server_cookie=cea2617dd2d70cde in_seq=0 out_seq=0 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.870+0000 7fd40ffff700 1 -- 172.21.15.93:0/3799533126 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e2f229e50 con 0x561e2f2fad60 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.870+0000 7fd40ffff700 10 client.16440 handle_command_reply: tid=0 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.870+0000 7fd44331cb80 1 client.16440 shutdown 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 msgr2=0x561e2f2eaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd41c007fa0 tx=0x7fd41c0127e0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 shutdown_connections 2023-07-21T22:20:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f2fad60 0x561e2f2fd170 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f13b6b0 0x561e2f2ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 --2- 172.21.15.93:0/3799533126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7420 0x561e2f2eaa30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 >> 172.21.15.93:0/3799533126 conn(0x561e2f2d4ae0 msgr2=0x561e2f2d93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd40ffff700 0 client.16440 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 shutdown_connections 2023-07-21T22:20:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 1 -- 172.21.15.93:0/3799533126 wait complete. 2023-07-21T22:20:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.871+0000 7fd44331cb80 20 client.16440 trim_cache size 0 max 16384 2023-07-21T22:20:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 Processor -- start 2023-07-21T22:20:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 -- start start 2023-07-21T22:20:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e2130 con 0x561e2f13b6b0 2023-07-21T22:20:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e6b10 con 0x561e2f2e7420 2023-07-21T22:20:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd44331cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e8fb0 con 0x561e2f2e7870 2023-07-21T22:20:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46518/0 (socket says 172.21.15.93:46518) 2023-07-21T22:20:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 -- 172.21.15.93:0/1133553428 learned_addr learned my addr 172.21.15.93:0/1133553428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd435734700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 -- 172.21.15.93:0/1133553428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 msgr2=0x561e2f2e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd42ffff700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 -- 172.21.15.93:0/1133553428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 msgr2=0x561e2f2e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 -- 172.21.15.93:0/1133553428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2f0851f0 con 0x561e2f13b6b0 2023-07-21T22:20:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd435734700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd42ffff700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.886+0000 7fd434f33700 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd42401eef0 tx=0x7fd424020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2439d2e2121aac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd41b7fe700 1 -- 172.21.15.93:0/1133553428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42402e040 con 0x561e2f13b6b0 2023-07-21T22:20:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd41b7fe700 1 -- 172.21.15.93:0/1133553428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424011c00 con 0x561e2f13b6b0 2023-07-21T22:20:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd41b7fe700 1 -- 172.21.15.93:0/1133553428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424004710 con 0x561e2f13b6b0 2023-07-21T22:20:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- 172.21.15.93:0/1133553428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 msgr2=0x561e2f2f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd42401eef0 tx=0x7fd424020040 comp rx=0 tx=0).stop 2023-07-21T22:20:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- 172.21.15.93:0/1133553428 shutdown_connections 2023-07-21T22:20:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f2e7870 0x561e2f2e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f13b6b0 0x561e2f2f1d40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 --2- 172.21.15.93:0/1133553428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- 172.21.15.93:0/1133553428 >> 172.21.15.93:0/1133553428 conn(0x561e2f2d4ae0 msgr2=0x561e2f2e4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- 172.21.15.93:0/1133553428 shutdown_connections 2023-07-21T22:20:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- 172.21.15.93:0/1133553428 wait complete. 2023-07-21T22:20:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.887+0000 7fd44331cb80 1 -- start start 2023-07-21T22:20:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 Processor -- start 2023-07-21T22:20:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 0x561e2f2e2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e3cb0 con 0x561e2f2e7870 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e3e20 con 0x561e2f2e7420 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f2e3fe0 con 0x561e2f13b6b0 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd44331cb80 10 client.0 _ll_register_callbacks cb 0x7fd41c007a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46528/0 (socket says 172.21.15.93:46528) 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 -- 172.21.15.93:0/883040540 learned_addr learned my addr 172.21.15.93:0/883040540 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd42ffff700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd434f33700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 0x561e2f2e2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 -- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 msgr2=0x561e2f2e2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 0x561e2f2e2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 -- 172.21.15.93:0/883040540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 msgr2=0x561e2f2e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd435734700 1 -- 172.21.15.93:0/883040540 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e2f05ed80 con 0x561e2f2e7870 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.888+0000 7fd434f33700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 0x561e2f2e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd435734700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd4280cbbc0 tx=0x7fd4280099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7a363fa51781e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42809aed0 con 0x561e2f2e7870 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd42809b570 con 0x561e2f2e7870 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd424011da0 con 0x561e2f2e7870 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd42809f5a0 con 0x561e2f2e7870 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 10 client.16446 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561e2f05d250 con 0x561e2f2e7870 2023-07-21T22:20:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424011da0 con 0x561e2f2e7870 2023-07-21T22:20:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4280cd030 con 0x561e2f2e7870 2023-07-21T22:20:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 10 client.16446 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 20 client.16446 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f34f730 0x561e2f351b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 4 client.16446 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:20:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.889+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e2f1ebff0 con 0x561e2f34f730 2023-07-21T22:20:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.890+0000 7fd434f33700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f34f730 0x561e2f351b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.890+0000 7fd40ffff700 10 client.16446 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.890+0000 7fd434f33700 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f34f730 0x561e2f351b10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56206f7bfe6e0cc9 server_cookie=b1eff858bb761870 in_seq=0 out_seq=0 2023-07-21T22:20:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd40ffff700 1 -- 172.21.15.93:0/883040540 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e2f1ebff0 con 0x561e2f34f730 2023-07-21T22:20:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd40ffff700 10 client.16446 handle_command_reply: tid=0 2023-07-21T22:20:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 client.16446 shutdown 2023-07-21T22:20:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 msgr2=0x561e2f2e35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd4280cbbc0 tx=0x7fd4280099e0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 shutdown_connections 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561e2f34f730 0x561e2f351b10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561e2f2e7870 0x561e2f2e35a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd40ffff700 0 client.16446 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561e2f13b6b0 0x561e2f2e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.920+0000 7fd44331cb80 1 --2- 172.21.15.93:0/883040540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561e2f2e7420 0x561e2f2e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 >> 172.21.15.93:0/883040540 conn(0x561e2f2d4ae0 msgr2=0x561e2f2e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 shutdown_connections 2023-07-21T22:20:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd44331cb80 1 -- 172.21.15.93:0/883040540 wait complete. 2023-07-21T22:20:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd44331cb80 20 client.16446 trim_cache size 0 max 16384 2023-07-21T22:20:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:20:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:20:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:20:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:20:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 msgr2=0x7fd4200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 0x7fd4200a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd424005950 tx=0x7fd4240058e0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 msgr2=0x7fd4301949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d010 tx=0x7fd428007870 comp rx=0 tx=0).stop 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 shutdown_connections 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd4200a12c0 0x7fd4200a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4301039b0 0x7fd4301949d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4300ff100 0x7fd43019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 --2- 172.21.15.93:0/768182677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4300ffab0 0x7fd43019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 >> 172.21.15.93:0/768182677 conn(0x7fd430075f90 msgr2=0x7fd4300fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 shutdown_connections 2023-07-21T22:20:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:38.921+0000 7fd437197700 1 -- 172.21.15.93:0/768182677 wait complete. 2023-07-21T22:20:39.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114266 2.27 MB/sec execute 76 sec latency 1444.017 ms 2023-07-21T22:20:39.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:39 smithi093 ceph-mon[96579]: pgmap v130: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 889 KiB/s wr, 94 op/s 2023-07-21T22:20:39.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:39 smithi194 ceph-mon[98288]: pgmap v130: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 889 KiB/s wr, 94 op/s 2023-07-21T22:20:39.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:39 smithi134 ceph-mon[97873]: pgmap v130: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 889 KiB/s wr, 94 op/s 2023-07-21T22:20:40.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114266 2.24 MB/sec execute 77 sec latency 2444.100 ms 2023-07-21T22:20:41.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114266 2.21 MB/sec execute 78 sec latency 3444.199 ms 2023-07-21T22:20:41.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:41 smithi093 ceph-mon[96579]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 615 KiB/s wr, 67 op/s 2023-07-21T22:20:41.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:41 smithi194 ceph-mon[98288]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 615 KiB/s wr, 67 op/s 2023-07-21T22:20:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:41 smithi134 ceph-mon[97873]: pgmap v131: 97 pgs: 97 active+clean; 56 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 615 KiB/s wr, 67 op/s 2023-07-21T22:20:42.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 114266 2.18 MB/sec execute 79 sec latency 4444.303 ms 2023-07-21T22:20:43.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 115459 2.22 MB/sec execute 80 sec latency 4954.776 ms 2023-07-21T22:20:43.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:43 smithi093 ceph-mon[96579]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 670 KiB/s wr, 85 op/s 2023-07-21T22:20:43.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:43 smithi194 ceph-mon[98288]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 670 KiB/s wr, 85 op/s 2023-07-21T22:20:43.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:43 smithi134 ceph-mon[97873]: pgmap v132: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 670 KiB/s wr, 85 op/s 2023-07-21T22:20:44.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117498 2.36 MB/sec execute 81 sec latency 10.213 ms 2023-07-21T22:20:45.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117756 2.33 MB/sec execute 82 sec latency 803.928 ms 2023-07-21T22:20:45.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:45 smithi093 ceph-mon[96579]: pgmap v133: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:45 smithi194 ceph-mon[98288]: pgmap v133: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:45.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:45 smithi134 ceph-mon[97873]: pgmap v133: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:46.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117756 2.31 MB/sec execute 83 sec latency 1804.024 ms 2023-07-21T22:20:47.321 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117756 2.28 MB/sec execute 84 sec latency 2804.098 ms 2023-07-21T22:20:47.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:47 smithi093 ceph-mon[96579]: pgmap v134: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:47.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:47 smithi194 ceph-mon[98288]: pgmap v134: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:47 smithi134 ceph-mon[97873]: pgmap v134: 97 pgs: 97 active+clean; 56 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 407 KiB/s wr, 69 op/s 2023-07-21T22:20:48.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117812 2.25 MB/sec execute 85 sec latency 3316.698 ms 2023-07-21T22:20:49.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117812 2.23 MB/sec execute 86 sec latency 1446.835 ms 2023-07-21T22:20:49.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:49 smithi093 ceph-mon[96579]: pgmap v135: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 622 KiB/s wr, 106 op/s 2023-07-21T22:20:49.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:49 smithi194 ceph-mon[98288]: pgmap v135: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 622 KiB/s wr, 106 op/s 2023-07-21T22:20:49.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:49 smithi134 ceph-mon[97873]: pgmap v135: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 622 KiB/s wr, 106 op/s 2023-07-21T22:20:50.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117812 2.20 MB/sec execute 87 sec latency 2446.890 ms 2023-07-21T22:20:51.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117812 2.18 MB/sec execute 88 sec latency 3446.975 ms 2023-07-21T22:20:51.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:51 smithi194 ceph-mon[98288]: pgmap v136: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 269 KiB/s wr, 55 op/s 2023-07-21T22:20:51.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:51 smithi134 ceph-mon[97873]: pgmap v136: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 269 KiB/s wr, 55 op/s 2023-07-21T22:20:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:51 smithi093 ceph-mon[96579]: pgmap v136: 97 pgs: 97 active+clean; 58 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 269 KiB/s wr, 55 op/s 2023-07-21T22:20:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 -- 172.21.15.93:0/2832777061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 msgr2=0x7fe8000fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 --2- 172.21.15.93:0/2832777061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe8000fe7c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00099c0 tx=0x7fe7f0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 -- 172.21.15.93:0/2832777061 shutdown_connections 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 --2- 172.21.15.93:0/2832777061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 0x7fe80010b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 --2- 172.21.15.93:0/2832777061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe8000fe7c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 --2- 172.21.15.93:0/2832777061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe800102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 -- 172.21.15.93:0/2832777061 >> 172.21.15.93:0/2832777061 conn(0x7fe800075f90 msgr2=0x7fe800076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 -- 172.21.15.93:0/2832777061 shutdown_connections 2023-07-21T22:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.038+0000 7fe8068fa700 1 -- 172.21.15.93:0/2832777061 wait complete. 2023-07-21T22:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 Processor -- start 2023-07-21T22:20:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 -- start start 2023-07-21T22:20:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe8001972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 0x7fe8001a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001a2140 con 0x7fe8000fe3f0 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001a2280 con 0x7fe8000fed90 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe8068fa700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001a2440 con 0x7fe800103430 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46538/0 (socket says 172.21.15.93:46538) 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 -- 172.21.15.93:0/285817972 learned_addr learned my addr 172.21.15.93:0/285817972 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe8001972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe804e97700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 0x7fe8001a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 -- 172.21.15.93:0/285817972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 msgr2=0x7fe8001a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 0x7fe8001a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 -- 172.21.15.93:0/285817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 msgr2=0x7fe8001972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe8001972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 -- 172.21.15.93:0/285817972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00095f0 con 0x7fe8000fe3f0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe8001972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.039+0000 7fe7fffff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009990 tx=0x7fe7f0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.040+0000 7fe7fdffb700 1 -- 172.21.15.93:0/285817972 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f001d070 con 0x7fe8000fe3f0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.040+0000 7fe7fdffb700 1 -- 172.21.15.93:0/285817972 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f0004010 con 0x7fe8000fe3f0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.040+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8001a26c0 con 0x7fe8000fe3f0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.040+0000 7fe7fdffb700 1 -- 172.21.15.93:0/285817972 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0022d40 con 0x7fe8000fe3f0 2023-07-21T22:20:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.040+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8001a2ae0 con 0x7fe8000fe3f0 2023-07-21T22:20:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.042+0000 7fe7fdffb700 1 -- 172.21.15.93:0/285817972 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe7f0022470 con 0x7fe8000fe3f0 2023-07-21T22:20:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.042+0000 7fe7fdffb700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 0x7fe7ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.042+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 0x7fe7ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe7fdffb700 1 -- 172.21.15.93:0/285817972 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe7f00ce050 con 0x7fe8000fe3f0 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 0x7fe7ec0a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005950 tx=0x7fe7e80058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 Processor -- start 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 -- start start 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.043+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d598dfc50 con 0x558d598366b0 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d599329e0 con 0x558d599e0660 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5992be70 con 0x558d599e0210 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46546/0 (socket says 172.21.15.93:46546) 2023-07-21T22:20:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 -- 172.21.15.93:0/3458758604 learned_addr learned my addr 172.21.15.93:0/3458758604 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 -- 172.21.15.93:0/3458758604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 msgr2=0x558d599d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe804e97700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 -- 172.21.15.93:0/3458758604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 msgr2=0x558d599da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 -- 172.21.15.93:0/3458758604 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d59759d80 con 0x558d598366b0 2023-07-21T22:20:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe804e97700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7fffff700 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe7f009b160 tx=0x7fe7f0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a30413f4371e62 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7f67fc700 1 -- 172.21.15.93:0/3458758604 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f009b1d0 con 0x558d598366b0 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7f67fc700 1 -- 172.21.15.93:0/3458758604 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f009f200 con 0x558d598366b0 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.044+0000 7fe7f67fc700 1 -- 172.21.15.93:0/3458758604 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f009a920 con 0x558d598366b0 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- 172.21.15.93:0/3458758604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 msgr2=0x558d599dfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe7f009b160 tx=0x7fe7f0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- 172.21.15.93:0/3458758604 shutdown_connections 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dfcd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/3458758604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- 172.21.15.93:0/3458758604 >> 172.21.15.93:0/3458758604 conn(0x558d599cfae0 msgr2=0x558d599d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- 172.21.15.93:0/3458758604 shutdown_connections 2023-07-21T22:20:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- 172.21.15.93:0/3458758604 wait complete. 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.045+0000 7fe812a7fb80 1 -- start start 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 Processor -- start 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 0x558d599e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0660 0x558d599e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d598cba60 con 0x558d599e0210 2023-07-21T22:20:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d598cb760 con 0x558d598366b0 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d597af8c0 con 0x558d599e0660 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe812a7fb80 10 client.0 _ll_register_callbacks cb 0x558d598c9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46552/0 (socket says 172.21.15.93:46552) 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 -- 172.21.15.93:0/2273336504 learned_addr learned my addr 172.21.15.93:0/2273336504 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 0x558d599e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe804e97700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0660 0x558d599e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 -- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0660 msgr2=0x558d599e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0660 0x558d599e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 -- 172.21.15.93:0/2273336504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 msgr2=0x558d599e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 0x558d599e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 -- 172.21.15.93:0/2273336504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8001a2ae0 con 0x558d599e0210 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 0x558d599e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800fe10 tx=0x7fe7e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b807f1dec313d16c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.046+0000 7fe7df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8017040 con 0x558d599e0210 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe7e8003a90 con 0x558d599e0210 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d59885490 con 0x558d599e0210 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e8004580 con 0x558d599e0210 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 10 client.16458 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558d59742c00 con 0x558d599e0210 2023-07-21T22:20:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d59885490 con 0x558d599e0210 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7e801f430 con 0x558d599e0210 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 20 client.16458 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d599f3720 0x558d599f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 4 client.16458 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.047+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d59924e50 con 0x558d599f3720 2023-07-21T22:20:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.048+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d599f3720 0x558d599f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.048+0000 7fe7df7fe700 10 client.16458 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.048+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d599f3720 0x558d599f5b30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd4edd855434cce server_cookie=80365274dbe7c23b in_seq=0 out_seq=0 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2273336504 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558d59924e50 con 0x558d599f3720 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe7df7fe700 10 client.16458 handle_command_reply: tid=0 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 client.16458 shutdown 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 msgr2=0x558d599e9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800fe10 tx=0x7fe7e8004750 comp rx=0 tx=0).stop 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 shutdown_connections 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d599f3720 0x558d599f5b30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0660 0x558d599e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d598366b0 0x558d599e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe7df7fe700 0 client.16458 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2273336504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0210 0x558d599e9790 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 >> 172.21.15.93:0/2273336504 conn(0x558d599cfae0 msgr2=0x558d599d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 shutdown_connections 2023-07-21T22:20:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2273336504 wait complete. 2023-07-21T22:20:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.049+0000 7fe812a7fb80 20 client.16458 trim_cache size 0 max 16384 2023-07-21T22:20:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.064+0000 7fe812a7fb80 1 Processor -- start 2023-07-21T22:20:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.064+0000 7fe812a7fb80 1 -- start start 2023-07-21T22:20:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d599d1c00 con 0x558d598366b0 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d599e4620 con 0x558d599e0660 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d599f0d80 con 0x558d599e0210 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46560/0 (socket says 172.21.15.93:46560) 2023-07-21T22:20:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 -- 172.21.15.93:0/490851649 learned_addr learned my addr 172.21.15.93:0/490851649 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 -- 172.21.15.93:0/490851649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 msgr2=0x558d599d4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 -- 172.21.15.93:0/490851649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 msgr2=0x558d599d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 -- 172.21.15.93:0/490851649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d597801f0 con 0x558d598366b0 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7fffff700 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0099520 tx=0x7fe7f009ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87ec9d635f4b82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7f6ffd700 1 -- 172.21.15.93:0/490851649 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f009a6c0 con 0x558d598366b0 2023-07-21T22:20:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7f6ffd700 1 -- 172.21.15.93:0/490851649 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f009a820 con 0x558d598366b0 2023-07-21T22:20:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.065+0000 7fe7f6ffd700 1 -- 172.21.15.93:0/490851649 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00a3360 con 0x558d598366b0 2023-07-21T22:20:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- 172.21.15.93:0/490851649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 msgr2=0x558d599dd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0099520 tx=0x7fe7f009ac60 comp rx=0 tx=0).stop 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- 172.21.15.93:0/490851649 shutdown_connections 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0660 0x558d599d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d598366b0 0x558d599dd990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/490851649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d599e0210 0x558d599d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- 172.21.15.93:0/490851649 >> 172.21.15.93:0/490851649 conn(0x558d599cfae0 msgr2=0x558d599e3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- 172.21.15.93:0/490851649 shutdown_connections 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- 172.21.15.93:0/490851649 wait complete. 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.066+0000 7fe812a7fb80 1 -- start start 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 Processor -- start 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d598366b0 0x558d599e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0210 0x558d599e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d599e7df0 con 0x558d599e0660 2023-07-21T22:20:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59a46b20 con 0x558d599e0210 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d59a46c60 con 0x558d598366b0 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe812a7fb80 10 client.0 _ll_register_callbacks cb 0x7fe7e800be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46568/0 (socket says 172.21.15.93:46568) 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 -- 172.21.15.93:0/2043122996 learned_addr learned my addr 172.21.15.93:0/2043122996 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d598366b0 0x558d599e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 -- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d598366b0 msgr2=0x558d599e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe7f7fff700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0210 0x558d599e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d598366b0 0x558d599e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 -- 172.21.15.93:0/2043122996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0210 msgr2=0x558d599e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0210 0x558d599e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.067+0000 7fe804e97700 1 -- 172.21.15.93:0/2043122996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d59759d80 con 0x558d599e0660 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe804e97700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007780 tx=0x7fe7f8007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808e772d902d0b10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:20:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004bb0 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fe7f8004d10 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7f009f200 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f8016660 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 10 client.16464 fetch_fsmap learned FSMap version 38 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558d59758250 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f009f200 con 0x558d599e0660 2023-07-21T22:20:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe7f8014070 con 0x558d599e0660 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 10 client.16464 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 20 client.16464 populate_metadata read hostname 'smithi093' 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d59a48360 0x558d59a4a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 4 client.16464 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.068+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d598e6ff0 con 0x558d59a48360 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.069+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d59a48360 0x558d59a4a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.069+0000 7fe7df7fe700 10 client.16464 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.069+0000 7fe7fffff700 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d59a48360 0x558d59a4a770 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46da3748a55f63bb server_cookie=21d405e94653aa3 in_seq=0 out_seq=0 2023-07-21T22:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe7df7fe700 1 -- 172.21.15.93:0/2043122996 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d598e6ff0 con 0x558d59a48360 2023-07-21T22:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe7df7fe700 10 client.16464 handle_command_reply: tid=0 2023-07-21T22:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 client.16464 shutdown 2023-07-21T22:20:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 msgr2=0x558d59a465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8007780 tx=0x7fe7f8007b40 comp rx=0 tx=0).stop 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 shutdown_connections 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558d59a48360 0x558d59a4a770 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558d599e0660 0x558d59a465e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe7df7fe700 0 client.16464 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558d598366b0 0x558d599e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 --2- 172.21.15.93:0/2043122996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d599e0210 0x558d599e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 >> 172.21.15.93:0/2043122996 conn(0x558d599cfae0 msgr2=0x558d599dd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 shutdown_connections 2023-07-21T22:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 1 -- 172.21.15.93:0/2043122996 wait complete. 2023-07-21T22:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.100+0000 7fe812a7fb80 20 client.16464 trim_cache size 0 max 16384 2023-07-21T22:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:20:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:20:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:20:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:20:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 msgr2=0x7fe7ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 0x7fe7ec0a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005950 tx=0x7fe7e80058e0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 msgr2=0x7fe800196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009990 tx=0x7fe7f0009580 comp rx=0 tx=0).stop 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 shutdown_connections 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe800103430 0x7fe8001a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe7ec0a12c0 0x7fe7ec0a3770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8000fe3f0 0x7fe800196d60 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 --2- 172.21.15.93:0/285817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8000fed90 0x7fe8001972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 >> 172.21.15.93:0/285817972 conn(0x7fe800075f90 msgr2=0x7fe8000fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 shutdown_connections 2023-07-21T22:20:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:20:52.101+0000 7fe8068fa700 1 -- 172.21.15.93:0/285817972 wait complete. 2023-07-21T22:20:52.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 117812 2.15 MB/sec execute 89 sec latency 4447.069 ms 2023-07-21T22:20:53.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 118984 2.19 MB/sec execute 90 sec latency 4955.291 ms 2023-07-21T22:20:53.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:53 smithi194 ceph-mon[98288]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 644 KiB/s wr, 78 op/s 2023-07-21T22:20:53.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:53 smithi134 ceph-mon[97873]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 644 KiB/s wr, 78 op/s 2023-07-21T22:20:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:53 smithi093 ceph-mon[96579]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 644 KiB/s wr, 78 op/s 2023-07-21T22:20:54.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121043 2.31 MB/sec execute 91 sec latency 9.886 ms 2023-07-21T22:20:55.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121300 2.29 MB/sec execute 92 sec latency 769.221 ms 2023-07-21T22:20:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:55 smithi194 ceph-mon[98288]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:55.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:55 smithi134 ceph-mon[97873]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:55 smithi093 ceph-mon[96579]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:56.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121300 2.26 MB/sec execute 93 sec latency 1769.325 ms 2023-07-21T22:20:57.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121300 2.24 MB/sec execute 94 sec latency 2769.392 ms 2023-07-21T22:20:57.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:57 smithi194 ceph-mon[98288]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:57 smithi134 ceph-mon[97873]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:57 smithi093 ceph-mon[96579]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 590 KiB/s wr, 60 op/s 2023-07-21T22:20:58.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121356 2.22 MB/sec execute 95 sec latency 3281.330 ms 2023-07-21T22:20:59.322 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121356 2.19 MB/sec execute 96 sec latency 1449.777 ms 2023-07-21T22:20:59.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:20:59 smithi194 ceph-mon[98288]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 835 KiB/s wr, 113 op/s 2023-07-21T22:20:59.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:20:59 smithi134 ceph-mon[97873]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 835 KiB/s wr, 113 op/s 2023-07-21T22:20:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:20:59 smithi093 ceph-mon[96579]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 835 KiB/s wr, 113 op/s 2023-07-21T22:21:00.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121356 2.17 MB/sec execute 97 sec latency 2449.882 ms 2023-07-21T22:21:01.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 121668 2.17 MB/sec execute 98 sec latency 2969.564 ms 2023-07-21T22:21:01.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:01 smithi194 ceph-mon[98288]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 621 KiB/s wr, 75 op/s 2023-07-21T22:21:01.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:01 smithi134 ceph-mon[97873]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 621 KiB/s wr, 75 op/s 2023-07-21T22:21:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:01 smithi093 ceph-mon[96579]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 621 KiB/s wr, 75 op/s 2023-07-21T22:21:02.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124310 2.30 MB/sec execute 99 sec latency 22.530 ms 2023-07-21T22:21:03.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124905 2.30 MB/sec execute 100 sec latency 400.370 ms 2023-07-21T22:21:03.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:03 smithi194 ceph-mon[98288]: pgmap v142: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 912 KiB/s wr, 138 op/s 2023-07-21T22:21:03.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:03 smithi134 ceph-mon[97873]: pgmap v142: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 912 KiB/s wr, 138 op/s 2023-07-21T22:21:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:03 smithi093 ceph-mon[96579]: pgmap v142: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 912 KiB/s wr, 138 op/s 2023-07-21T22:21:04.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124905 2.27 MB/sec execute 101 sec latency 1400.463 ms 2023-07-21T22:21:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 -- 172.21.15.93:0/667282416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00fea30 msgr2=0x7fa0f00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/667282416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00fea30 0x7fa0f00feea0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 -- 172.21.15.93:0/667282416 shutdown_connections 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/667282416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 0x7fa0f010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/667282416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f0104a30 0x7fa0f0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/667282416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00fea30 0x7fa0f00feea0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 -- 172.21.15.93:0/667282416 >> 172.21.15.93:0/667282416 conn(0x7fa0f0074760 msgr2=0x7fa0f0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 -- 172.21.15.93:0/667282416 shutdown_connections 2023-07-21T22:21:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.223+0000 7fa0f83ef700 1 -- 172.21.15.93:0/667282416 wait complete. 2023-07-21T22:21:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 Processor -- start 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 -- start start 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f00fea30 0x7fa0f0196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 0x7fa0f0197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01a2070 con 0x7fa0f0104a30 2023-07-21T22:21:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01a21e0 con 0x7fa0f00ff3e0 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f83ef700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01a23a0 con 0x7fa0f00fea30 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46584/0 (socket says 172.21.15.93:46584) 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 -- 172.21.15.93:0/2965045652 learned_addr learned my addr 172.21.15.93:0/2965045652 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f618b700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f00fea30 0x7fa0f0196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f598a700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 0x7fa0f0197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 -- 172.21.15.93:0/2965045652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f00fea30 msgr2=0x7fa0f0196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f00fea30 0x7fa0f0196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 -- 172.21.15.93:0/2965045652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 msgr2=0x7fa0f0197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 0x7fa0f0197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.224+0000 7fa0f698c700 1 -- 172.21.15.93:0/2965045652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0095f0 con 0x7fa0f0104a30 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0f698c700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8007ee0 tx=0x7fa0e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0e77fe700 1 -- 172.21.15.93:0/2965045652 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800dd20 con 0x7fa0f0104a30 2023-07-21T22:21:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0e77fe700 1 -- 172.21.15.93:0/2965045652 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e800f040 con 0x7fa0f0104a30 2023-07-21T22:21:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f01a2680 con 0x7fa0f0104a30 2023-07-21T22:21:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0e77fe700 1 -- 172.21.15.93:0/2965045652 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80048a0 con 0x7fa0f0104a30 2023-07-21T22:21:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.225+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f01a2b40 con 0x7fa0f0104a30 2023-07-21T22:21:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.227+0000 7fa0e77fe700 1 -- 172.21.15.93:0/2965045652 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa0e8003a00 con 0x7fa0f0104a30 2023-07-21T22:21:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.228+0000 7fa0e77fe700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 0x7fa0dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.228+0000 7fa0e77fe700 1 -- 172.21.15.93:0/2965045652 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0e80cc590 con 0x7fa0f0104a30 2023-07-21T22:21:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.228+0000 7fa0f618b700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 0x7fa0dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.228+0000 7fa0f618b700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 0x7fa0dc0a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ffba0 tx=0x7fa0e000b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 Processor -- start 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 -- start start 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab7a1c50 con 0x5647ab8a2210 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab7f49e0 con 0x5647ab8a2660 2023-07-21T22:21:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa104574b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab7ede70 con 0x5647ab6f86b0 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa0f598a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa0f598a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46590/0 (socket says 172.21.15.93:46590) 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa0f598a700 1 -- 172.21.15.93:0/3000776543 learned_addr learned my addr 172.21.15.93:0/3000776543 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa0f698c700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.229+0000 7fa0f618b700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 -- 172.21.15.93:0/3000776543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 msgr2=0x5647ab8a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 -- 172.21.15.93:0/3000776543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 msgr2=0x5647ab89c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 -- 172.21.15.93:0/3000776543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ab61bd80 con 0x5647ab8a2210 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f618b700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0f598a700 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec015000 tx=0x7fa0ec005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c67d347d73bf51 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0e4dac700 1 -- 172.21.15.93:0/3000776543 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec01d050 con 0x5647ab8a2210 2023-07-21T22:21:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0e4dac700 1 -- 172.21.15.93:0/3000776543 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec01e040 con 0x5647ab8a2210 2023-07-21T22:21:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.230+0000 7fa0e4dac700 1 -- 172.21.15.93:0/3000776543 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0052d0 con 0x5647ab8a2210 2023-07-21T22:21:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- 172.21.15.93:0/3000776543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 msgr2=0x5647ab89bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec015000 tx=0x7fa0ec005e80 comp rx=0 tx=0).stop 2023-07-21T22:21:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- 172.21.15.93:0/3000776543 shutdown_connections 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- 172.21.15.93:0/3000776543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- 172.21.15.93:0/3000776543 >> 172.21.15.93:0/3000776543 conn(0x5647ab891ae0 msgr2=0x5647ab8970e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- 172.21.15.93:0/3000776543 shutdown_connections 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- 172.21.15.93:0/3000776543 wait complete. 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- start start 2023-07-21T22:21:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 Processor -- start 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab78da60 con 0x5647ab8a2660 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab78d760 con 0x5647ab8a2210 2023-07-21T22:21:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.231+0000 7fa104574b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab6718c0 con 0x5647ab6f86b0 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa104574b80 10 client.0 _ll_register_callbacks cb 0x5647ab78b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46598/0 (socket says 172.21.15.93:46598) 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 -- 172.21.15.93:0/1259266572 learned_addr learned my addr 172.21.15.93:0/1259266572 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f598a700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 -- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 msgr2=0x5647ab8ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 -- 172.21.15.93:0/1259266572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 msgr2=0x5647ab8abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 -- 172.21.15.93:0/1259266572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f010c160 con 0x5647ab8a2660 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0f698c700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8095b80 tx=0x7fa0e8098fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74c096964ce7e34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80a1ad0 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0e8009410 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647ab747490 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0e8095d90 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa104574b80 10 client.16476 fetch_fsmap learned FSMap version 38 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.232+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5647ab604c00 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647ab747490 con 0x5647ab8a2660 2023-07-21T22:21:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0e8098be0 con 0x5647ab8a2660 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 20 client.16476 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab8b59f0 0x5647ab8b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 4 client.16476 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647ab7e6e50 con 0x5647ab8b59f0 2023-07-21T22:21:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab8b59f0 0x5647ab8b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab8b59f0 0x5647ab8b7e00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ab0f51364b9890 server_cookie=978d5d7625b30ec1 in_seq=0 out_seq=0 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.233+0000 7fa0d9ffb700 10 client.16476 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1259266572 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5647ab7e6e50 con 0x5647ab8b59f0 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa0d9ffb700 10 client.16476 handle_command_reply: tid=0 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 client.16476 shutdown 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 msgr2=0x5647ab8a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8095b80 tx=0x7fa0e8098fe0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 shutdown_connections 2023-07-21T22:21:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab8b59f0 0x5647ab8b7e00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2660 0x5647ab8a5850 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa0d9ffb700 0 client.16476 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 --2- 172.21.15.93:0/1259266572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 >> 172.21.15.93:0/1259266572 conn(0x5647ab891ae0 msgr2=0x5647ab8963e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 shutdown_connections 2023-07-21T22:21:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.234+0000 7fa104574b80 1 -- 172.21.15.93:0/1259266572 wait complete. 2023-07-21T22:21:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.235+0000 7fa104574b80 20 client.16476 trim_cache size 0 max 16384 2023-07-21T22:21:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.249+0000 7fa104574b80 1 Processor -- start 2023-07-21T22:21:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.249+0000 7fa104574b80 1 -- start start 2023-07-21T22:21:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab897fa0 con 0x5647ab8a2210 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab88eb30 con 0x5647ab8a2660 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa104574b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab893520 con 0x5647ab6f86b0 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46604/0 (socket says 172.21.15.93:46604) 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 -- 172.21.15.93:0/2860154179 learned_addr learned my addr 172.21.15.93:0/2860154179 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f618b700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f698c700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 -- 172.21.15.93:0/2860154179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 msgr2=0x5647ab8a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 -- 172.21.15.93:0/2860154179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 msgr2=0x5647ab89c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 -- 172.21.15.93:0/2860154179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ab8a5660 con 0x5647ab8a2210 2023-07-21T22:21:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f698c700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0f598a700 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec003e10 tx=0x7fa0ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f4f403cfaf6011 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0e55ad700 1 -- 172.21.15.93:0/2860154179 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec003890 con 0x5647ab8a2210 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0e55ad700 1 -- 172.21.15.93:0/2860154179 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec0039f0 con 0x5647ab8a2210 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.250+0000 7fa0e55ad700 1 -- 172.21.15.93:0/2860154179 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00f6c0 con 0x5647ab8a2210 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- 172.21.15.93:0/2860154179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 msgr2=0x5647ab89bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec003e10 tx=0x7fa0ec005040 comp rx=0 tx=0).stop 2023-07-21T22:21:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- 172.21.15.93:0/2860154179 shutdown_connections 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2660 0x5647ab89c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab6f86b0 0x5647ab8a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 --2- 172.21.15.93:0/2860154179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab8a2210 0x5647ab89bf40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- 172.21.15.93:0/2860154179 >> 172.21.15.93:0/2860154179 conn(0x5647ab891ae0 msgr2=0x5647ab8a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- 172.21.15.93:0/2860154179 shutdown_connections 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- 172.21.15.93:0/2860154179 wait complete. 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 -- start start 2023-07-21T22:21:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.251+0000 7fa104574b80 1 Processor -- start 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 0x5647ab9028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab902e70 con 0x5647ab6f86b0 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab902fe0 con 0x5647ab8a2210 2023-07-21T22:21:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ab9031a0 con 0x5647ab8a2660 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 10 client.0 _ll_register_callbacks cb 0x7fa0e80994e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46608/0 (socket says 172.21.15.93:46608) 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 -- 172.21.15.93:0/1420621608 learned_addr learned my addr 172.21.15.93:0/1420621608 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f698c700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 0x5647ab9028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f598a700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 -- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 msgr2=0x5647ab9028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 0x5647ab9028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 -- 172.21.15.93:0/1420621608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 msgr2=0x5647ab8aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 -- 172.21.15.93:0/1420621608 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647ab61bd80 con 0x5647ab6f86b0 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f698c700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 0x5647ab9028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0f618b700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x5647ab8b86e0 tx=0x7fa0e0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537c32d3afea47de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0026c00 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fa0e0017070 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0ec01e040 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0e0026d60 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.252+0000 7fa104574b80 10 client.16482 fetch_fsmap learned FSMap version 38 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5647ab6421f0 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec01e040 con 0x5647ab6f86b0 2023-07-21T22:21:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0e0011d10 con 0x5647ab6f86b0 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 20 client.16482 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab90a920 0x5647ab90cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 4 client.16482 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647ab7a8ff0 con 0x5647ab90a920 2023-07-21T22:21:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa0f598a700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab90a920 0x5647ab90cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa0d9ffb700 10 client.16482 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.253+0000 7fa0f598a700 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab90a920 0x5647ab90cd00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee79ca2b41b73f1 server_cookie=365bb347a3402439 in_seq=0 out_seq=0 2023-07-21T22:21:05.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124905 2.25 MB/sec execute 102 sec latency 2400.561 ms 2023-07-21T22:21:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa0d9ffb700 1 -- 172.21.15.93:0/1420621608 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647ab7a8ff0 con 0x5647ab90a920 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa0d9ffb700 10 client.16482 handle_command_reply: tid=0 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 client.16482 shutdown 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 msgr2=0x5647ab8ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x5647ab8b86e0 tx=0x7fa0e0013070 comp rx=0 tx=0).stop 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 shutdown_connections 2023-07-21T22:21:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5647ab90a920 0x5647ab90cd00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5647ab8a2660 0x5647ab9028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5647ab6f86b0 0x5647ab8ae4d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 --2- 172.21.15.93:0/1420621608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5647ab8a2210 0x5647ab8aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa0d9ffb700 0 client.16482 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 >> 172.21.15.93:0/1420621608 conn(0x5647ab891ae0 msgr2=0x5647ab8a48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 shutdown_connections 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 1 -- 172.21.15.93:0/1420621608 wait complete. 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.289+0000 7fa104574b80 20 client.16482 trim_cache size 0 max 16384 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:21:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:21:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 msgr2=0x7fa0dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 0x7fa0dc0a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ffba0 tx=0x7fa0e000b410 comp rx=0 tx=0).stop 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 msgr2=0x7fa0f01a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8007ee0 tx=0x7fa0e800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 shutdown_connections 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa0dc0a12c0 0x7fa0dc0a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f0104a30 0x7fa0f01a1aa0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f00fea30 0x7fa0f0196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 --2- 172.21.15.93:0/2965045652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f00ff3e0 0x7fa0f0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 >> 172.21.15.93:0/2965045652 conn(0x7fa0f0074760 msgr2=0x7fa0f0102650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 shutdown_connections 2023-07-21T22:21:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:05.290+0000 7fa0f83ef700 1 -- 172.21.15.93:0/2965045652 wait complete. 2023-07-21T22:21:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:05 smithi194 ceph-mon[98288]: pgmap v143: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:05.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:05 smithi134 ceph-mon[97873]: pgmap v143: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:05 smithi093 ceph-mon[96579]: pgmap v143: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:06.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124905 2.23 MB/sec execute 103 sec latency 3400.653 ms 2023-07-21T22:21:07.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124905 2.21 MB/sec execute 104 sec latency 4400.745 ms 2023-07-21T22:21:07.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:07 smithi194 ceph-mon[98288]: pgmap v144: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:07.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:07 smithi134 ceph-mon[97873]: pgmap v144: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:07 smithi093 ceph-mon[96579]: pgmap v144: 97 pgs: 97 active+clean; 62 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 537 KiB/s wr, 115 op/s 2023-07-21T22:21:08.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124973 2.19 MB/sec execute 105 sec latency 4907.805 ms 2023-07-21T22:21:09.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124973 2.17 MB/sec execute 106 sec latency 1444.824 ms 2023-07-21T22:21:09.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:09 smithi194 ceph-mon[98288]: pgmap v145: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 922 KiB/s wr, 158 op/s 2023-07-21T22:21:09.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:09 smithi134 ceph-mon[97873]: pgmap v145: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 922 KiB/s wr, 158 op/s 2023-07-21T22:21:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:09 smithi093 ceph-mon[96579]: pgmap v145: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 922 KiB/s wr, 158 op/s 2023-07-21T22:21:10.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124973 2.15 MB/sec execute 107 sec latency 2444.902 ms 2023-07-21T22:21:11.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124973 2.13 MB/sec execute 108 sec latency 3444.947 ms 2023-07-21T22:21:11.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:11 smithi194 ceph-mon[98288]: pgmap v146: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 677 KiB/s wr, 105 op/s 2023-07-21T22:21:11.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:11 smithi134 ceph-mon[97873]: pgmap v146: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 677 KiB/s wr, 105 op/s 2023-07-21T22:21:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:11 smithi093 ceph-mon[96579]: pgmap v146: 97 pgs: 97 active+clean; 62 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 677 KiB/s wr, 105 op/s 2023-07-21T22:21:12.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 124973 2.11 MB/sec execute 109 sec latency 4445.011 ms 2023-07-21T22:21:13.323 INFO:tasks.workunit.client.0.smithi093.stdout: 1 125854 2.13 MB/sec execute 110 sec latency 4954.138 ms 2023-07-21T22:21:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:13 smithi194 ceph-mon[98288]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 727 KiB/s wr, 120 op/s 2023-07-21T22:21:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:13 smithi134 ceph-mon[97873]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 727 KiB/s wr, 120 op/s 2023-07-21T22:21:13.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:13 smithi093 ceph-mon[96579]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 727 KiB/s wr, 120 op/s 2023-07-21T22:21:14.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 128031 2.24 MB/sec execute 111 sec latency 15.614 ms 2023-07-21T22:21:15.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 128393 2.22 MB/sec execute 112 sec latency 618.984 ms 2023-07-21T22:21:15.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:15 smithi194 ceph-mon[98288]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:15.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:15 smithi134 ceph-mon[97873]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:15.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:15 smithi093 ceph-mon[96579]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:16.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 128393 2.20 MB/sec execute 113 sec latency 1619.082 ms 2023-07-21T22:21:17.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 128393 2.18 MB/sec execute 114 sec latency 2619.182 ms 2023-07-21T22:21:17.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:17 smithi194 ceph-mon[98288]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:17 smithi134 ceph-mon[97873]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:17.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:17 smithi093 ceph-mon[96579]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 436 KiB/s wr, 58 op/s 2023-07-21T22:21:18.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 129286 2.21 MB/sec execute 115 sec latency 3130.355 ms 2023-07-21T22:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.403+0000 7f24c0d33700 1 -- 172.21.15.93:0/1168156187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ffb50 msgr2=0x7f24bc10b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.403+0000 7f24c0d33700 1 --2- 172.21.15.93:0/1168156187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ffb50 0x7f24bc10b7e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b0d0 tx=0x7f24b400b3e0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 -- 172.21.15.93:0/1168156187 shutdown_connections 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 --2- 172.21.15.93:0/1168156187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ffb50 0x7f24bc10b7e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 --2- 172.21.15.93:0/1168156187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 0x7f24bc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 --2- 172.21.15.93:0/1168156187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc0ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 -- 172.21.15.93:0/1168156187 >> 172.21.15.93:0/1168156187 conn(0x7f24bc075f90 msgr2=0x7f24bc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 -- 172.21.15.93:0/1168156187 shutdown_connections 2023-07-21T22:21:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 -- 172.21.15.93:0/1168156187 wait complete. 2023-07-21T22:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 Processor -- start 2023-07-21T22:21:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.404+0000 7f24c0d33700 1 -- start start 2023-07-21T22:21:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ffb50 0x7f24bc197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 0x7f24bc1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a2150 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a22c0 con 0x7f24bc103970 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1a2480 con 0x7f24bc0ffb50 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46622/0 (socket says 172.21.15.93:46622) 2023-07-21T22:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 -- 172.21.15.93:0/4187780290 learned_addr learned my addr 172.21.15.93:0/4187780290 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ffb50 0x7f24bc197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24bad9d700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 0x7f24bc1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 -- 172.21.15.93:0/4187780290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ffb50 msgr2=0x7f24bc197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ffb50 0x7f24bc197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 -- 172.21.15.93:0/4187780290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 msgr2=0x7f24bc1a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 0x7f24bc1a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 -- 172.21.15.93:0/4187780290 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b4009d00 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ba59c700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00d950 tx=0x7f24ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ab7fe700 1 -- 172.21.15.93:0/4187780290 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00f890 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ab7fe700 1 -- 172.21.15.93:0/4187780290 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ac010460 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc1a2760 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.405+0000 7f24ab7fe700 1 -- 172.21.15.93:0/4187780290 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00e610 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.406+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc1a2c70 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.407+0000 7f24ab7fe700 1 -- 172.21.15.93:0/4187780290 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f24ac0108d0 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.408+0000 7f24ab7fe700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 0x7f24a40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.408+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 0x7f24a40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.408+0000 7f24ab7fe700 1 -- 172.21.15.93:0/4187780290 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f24ac0ccae0 con 0x7f24bc0ff1a0 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.409+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 0x7f24a40a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ff170 tx=0x7f24b000b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.409+0000 7f24cceb8b80 1 Processor -- start 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.409+0000 7f24cceb8b80 1 -- start start 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d040f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16cf46c50 con 0x55f16ce9d6b0 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16cf999e0 con 0x55f16d047660 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16cf92e70 con 0x55f16d047210 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46630/0 (socket says 172.21.15.93:46630) 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 -- 172.21.15.93:0/850957568 learned_addr learned my addr 172.21.15.93:0/850957568 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d040f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24bad9d700 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 -- 172.21.15.93:0/850957568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 msgr2=0x55f16d040f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d040f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 -- 172.21.15.93:0/850957568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 msgr2=0x55f16d0414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 -- 172.21.15.93:0/850957568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f16cdc0d80 con 0x55f16ce9d6b0 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24ba59c700 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0960d0 tx=0x7f24ac099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbc5fe03d845b05 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24a8ff9700 1 -- 172.21.15.93:0/850957568 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0a2020 con 0x55f16ce9d6b0 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24a8ff9700 1 -- 172.21.15.93:0/850957568 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ac0962e0 con 0x55f16ce9d6b0 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.410+0000 7f24a8ff9700 1 -- 172.21.15.93:0/850957568 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0a1b90 con 0x55f16ce9d6b0 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 -- 172.21.15.93:0/850957568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 msgr2=0x55f16d046cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0960d0 tx=0x7f24ac099bd0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 -- 172.21.15.93:0/850957568 shutdown_connections 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d046cd0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/850957568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d040f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 -- 172.21.15.93:0/850957568 >> 172.21.15.93:0/850957568 conn(0x55f16d036ae0 msgr2=0x55f16d03c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 -- 172.21.15.93:0/850957568 shutdown_connections 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.411+0000 7f24cceb8b80 1 -- 172.21.15.93:0/850957568 wait complete. 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 -- start start 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 Processor -- start 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d050ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d04a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16cf32a60 con 0x55f16ce9d6b0 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16cf32760 con 0x55f16d047660 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16ce168c0 con 0x55f16d047210 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24cceb8b80 10 client.0 _ll_register_callbacks cb 0x55f16cf30700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46636/0 (socket says 172.21.15.93:46636) 2023-07-21T22:21:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 -- 172.21.15.93:0/428630894 learned_addr learned my addr 172.21.15.93:0/428630894 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d050ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24bad9d700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d04a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 -- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 msgr2=0x55f16d050ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d050ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 -- 172.21.15.93:0/428630894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 msgr2=0x55f16d04a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d04a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 -- 172.21.15.93:0/428630894 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc1a2c70 con 0x55f16ce9d6b0 2023-07-21T22:21:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d050ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24ba59c700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00e010 tx=0x7f24ac09a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e727a90643214 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.412+0000 7f24a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0b23b0 con 0x55f16ce9d6b0 2023-07-21T22:21:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f24ac0a2020 con 0x55f16ce9d6b0 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f16ceec490 con 0x55f16ce9d6b0 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24ac0990c0 con 0x55f16ce9d6b0 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 10 client.16494 fetch_fsmap learned FSMap version 38 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f16cda9c00 con 0x55f16ce9d6b0 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f16ceec490 con 0x55f16ce9d6b0 2023-07-21T22:21:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24ac098cd0 con 0x55f16ce9d6b0 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 20 client.16494 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d05a580 0x55f16d05c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 4 client.16494 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f16cf8be50 con 0x55f16d05a580 2023-07-21T22:21:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.413+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d05a580 0x55f16d05c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.414+0000 7f24a1ffb700 10 client.16494 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.414+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d05a580 0x55f16d05c990 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809575eaa68d64b9 server_cookie=b215f567b6efaa47 in_seq=0 out_seq=0 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24a1ffb700 1 -- 172.21.15.93:0/428630894 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f16cf8be50 con 0x55f16d05a580 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24a1ffb700 10 client.16494 handle_command_reply: tid=0 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 client.16494 shutdown 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 msgr2=0x55f16d050580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00e010 tx=0x7f24ac09a0e0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 shutdown_connections 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d05a580 0x55f16d05c990 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d04a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16ce9d6b0 0x55f16d050580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24a1ffb700 0 client.16494 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/428630894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16d047210 0x55f16d050ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.415+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 >> 172.21.15.93:0/428630894 conn(0x55f16d036ae0 msgr2=0x55f16d03b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.416+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 shutdown_connections 2023-07-21T22:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.416+0000 7f24cceb8b80 1 -- 172.21.15.93:0/428630894 wait complete. 2023-07-21T22:21:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.416+0000 7f24cceb8b80 20 client.16494 trim_cache size 0 max 16384 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 Processor -- start 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 -- start start 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d046250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 0x55f16d04a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d04ad40 con 0x55f16d047210 2023-07-21T22:21:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d04bdb0 con 0x55f16d047660 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d038520 con 0x55f16ce9d6b0 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24b9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 0x55f16d04a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d046250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24bad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d046250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:33724/0 (socket says 172.21.15.93:33724) 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.431+0000 7f24ba59c700 1 -- 172.21.15.93:0/2068385889 learned_addr learned my addr 172.21.15.93:0/2068385889 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 -- 172.21.15.93:0/2068385889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 msgr2=0x55f16d046250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d046250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 -- 172.21.15.93:0/2068385889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 msgr2=0x55f16d0481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 -- 172.21.15.93:0/2068385889 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f16d061580 con 0x55f16d047210 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24bad9d700 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 0x55f16d04a800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f16d05d230 tx=0x7f24b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ec627e0ebead53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24a97fa700 1 -- 172.21.15.93:0/2068385889 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0024040 con 0x55f16d047210 2023-07-21T22:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24a97fa700 1 -- 172.21.15.93:0/2068385889 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b0015070 con 0x55f16d047210 2023-07-21T22:21:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24a97fa700 1 -- 172.21.15.93:0/2068385889 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001f430 con 0x55f16d047210 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24cceb8b80 1 -- 172.21.15.93:0/2068385889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 msgr2=0x55f16d04a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.432+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 0x55f16d04a800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f16d05d230 tx=0x7f24b0013000 comp rx=0 tx=0).stop 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- 172.21.15.93:0/2068385889 shutdown_connections 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047660 0x55f16d0481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d046250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/2068385889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047210 0x55f16d04a800 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- 172.21.15.93:0/2068385889 >> 172.21.15.93:0/2068385889 conn(0x55f16d036ae0 msgr2=0x55f16d047aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- 172.21.15.93:0/2068385889 shutdown_connections 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- 172.21.15.93:0/2068385889 wait complete. 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- start start 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 Processor -- start 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d0a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047210 0x55f16d0a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d0ad540 con 0x55f16d047660 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d0ad680 con 0x55f16d047210 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16d0ad840 con 0x55f16ce9d6b0 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24cceb8b80 10 client.0 _ll_register_callbacks cb 0x7f24b4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24bad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24ba59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d0a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24bad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46654/0 (socket says 172.21.15.93:46654) 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24bad9d700 1 -- 172.21.15.93:0/1749531892 learned_addr learned my addr 172.21.15.93:0/1749531892 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.433+0000 7f24b9d9b700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047210 0x55f16d0a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 -- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 msgr2=0x55f16d0a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d0a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 -- 172.21.15.93:0/1749531892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047210 msgr2=0x55f16d0a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047210 0x55f16d0a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 -- 172.21.15.93:0/1749531892 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc10c1b0 con 0x55f16d047660 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24bad9d700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b0d0 tx=0x7f24b40095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8adace6c1dbcdb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b400e040 con 0x55f16d047660 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f24b4007970 con 0x55f16d047660 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24ac0962e0 con 0x55f16d047660 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b40050b0 con 0x55f16d047660 2023-07-21T22:21:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 10 client.16500 fetch_fsmap learned FSMap version 38 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f24b0010e30 con 0x55f16d047660 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ac0962e0 con 0x55f16d047660 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24b400b100 con 0x55f16d047660 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 20 client.16500 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d0af080 0x55f16d0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 4 client.16500 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.434+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f16cf4dff0 con 0x55f16d0af080 2023-07-21T22:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.435+0000 7f24ba59c700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d0af080 0x55f16d0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.435+0000 7f24ba59c700 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d0af080 0x55f16d0b1490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a90eb63f658e5de server_cookie=7be9c60df1ee3165 in_seq=0 out_seq=0 2023-07-21T22:21:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.435+0000 7f24a1ffb700 10 client.16500 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24a1ffb700 1 -- 172.21.15.93:0/1749531892 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f16cf4dff0 con 0x55f16d0af080 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24a1ffb700 10 client.16500 handle_command_reply: tid=0 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 client.16500 shutdown 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 msgr2=0x55f16d0a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b0d0 tx=0x7f24b40095b0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 shutdown_connections 2023-07-21T22:21:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f16d0af080 0x55f16d0b1490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f16d047660 0x55f16d0a2e10 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24a1ffb700 0 client.16500 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f16ce9d6b0 0x55f16d0a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 --2- 172.21.15.93:0/1749531892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f16d047210 0x55f16d0a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 >> 172.21.15.93:0/1749531892 conn(0x55f16d036ae0 msgr2=0x55f16d049920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 shutdown_connections 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 1 -- 172.21.15.93:0/1749531892 wait complete. 2023-07-21T22:21:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.477+0000 7f24cceb8b80 20 client.16500 trim_cache size 0 max 16384 2023-07-21T22:21:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:21:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:21:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:21:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:21:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 msgr2=0x7f24a40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 0x7f24a40a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ff170 tx=0x7f24b000b410 comp rx=0 tx=0).stop 2023-07-21T22:21:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 msgr2=0x7f24bc196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00d950 tx=0x7f24ac00dc60 comp rx=0 tx=0).stop 2023-07-21T22:21:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 shutdown_connections 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24bc103970 0x7f24bc1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f24a40a12c0 0x7f24a40a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24bc0ff1a0 0x7f24bc196ce0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 --2- 172.21.15.93:0/4187780290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24bc0ffb50 0x7f24bc197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 >> 172.21.15.93:0/4187780290 conn(0x7f24bc075f90 msgr2=0x7f24bc0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 shutdown_connections 2023-07-21T22:21:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:18.478+0000 7f24c0d33700 1 -- 172.21.15.93:0/4187780290 wait complete. 2023-07-21T22:21:18.880 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:21:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:18.881 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:21:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:18.881 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:21:18 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/2444487904 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587b1afb1e0 con 0x5587b1ad0800 2023-07-21T22:21:18.881 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:21:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:21:19.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131509 2.31 MB/sec execute 116 sec latency 33.354 ms 2023-07-21T22:21:19.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:19 smithi194 ceph-mon[98288]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 912 KiB/s wr, 105 op/s 2023-07-21T22:21:19.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:19 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:21:19.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:19 smithi134 ceph-mon[97873]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 912 KiB/s wr, 105 op/s 2023-07-21T22:21:19.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:19 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:21:19.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:19 smithi093 ceph-mon[96579]: pgmap v150: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 912 KiB/s wr, 105 op/s 2023-07-21T22:21:19.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:19 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:21:20.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131943 2.29 MB/sec execute 117 sec latency 523.510 ms 2023-07-21T22:21:20.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:20 smithi194 ceph-mon[98288]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:21:20.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:20 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/373386307,v1:172.21.15.134:6837/373386307] up:boot 2023-07-21T22:21:20.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:20 smithi194 ceph-mon[98288]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:21:20.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:20 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:20.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:20 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:20.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:20 smithi134 ceph-mon[97873]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:21:20.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:20 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/373386307,v1:172.21.15.134:6837/373386307] up:boot 2023-07-21T22:21:20.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:20 smithi134 ceph-mon[97873]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:21:20.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:20 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:20.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:20 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:20.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:20 smithi093 ceph-mon[96579]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:21:20.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:20 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/373386307,v1:172.21.15.134:6837/373386307] up:boot 2023-07-21T22:21:20.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:20 smithi093 ceph-mon[96579]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:21:20.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:20 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:20.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:20 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:21.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131943 2.27 MB/sec execute 118 sec latency 1523.595 ms 2023-07-21T22:21:21.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:21 smithi194 ceph-mon[98288]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 632 KiB/s wr, 74 op/s 2023-07-21T22:21:21.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:21 smithi134 ceph-mon[97873]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 632 KiB/s wr, 74 op/s 2023-07-21T22:21:21.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:21 smithi093 ceph-mon[96579]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 632 KiB/s wr, 74 op/s 2023-07-21T22:21:22.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131943 2.25 MB/sec execute 119 sec latency 2523.720 ms 2023-07-21T22:21:23.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131999 2.23 MB/sec execute 120 sec latency 3029.956 ms 2023-07-21T22:21:23.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:23 smithi194 ceph-mon[98288]: pgmap v153: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:23.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:23 smithi134 ceph-mon[97873]: pgmap v153: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:23.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:23 smithi093 ceph-mon[96579]: pgmap v153: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:24.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131999 2.21 MB/sec execute 121 sec latency 1449.944 ms 2023-07-21T22:21:25.324 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131999 2.20 MB/sec execute 122 sec latency 2449.977 ms 2023-07-21T22:21:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:25 smithi194 ceph-mon[98288]: pgmap v154: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:25 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:25.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:25 smithi134 ceph-mon[97873]: pgmap v154: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:25.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:25 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:25 smithi093 ceph-mon[96579]: pgmap v154: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:25 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:21:26.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131999 2.18 MB/sec execute 123 sec latency 3450.065 ms 2023-07-21T22:21:27.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 131999 2.16 MB/sec execute 124 sec latency 4450.177 ms 2023-07-21T22:21:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:27 smithi194 ceph-mon[98288]: pgmap v155: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:27.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:27 smithi134 ceph-mon[97873]: pgmap v155: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:27.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:27 smithi093 ceph-mon[96579]: pgmap v155: 97 pgs: 97 active+clean; 61 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 957 KiB/s wr, 133 op/s 2023-07-21T22:21:28.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 132799 2.18 MB/sec execute 125 sec latency 4958.413 ms 2023-07-21T22:21:29.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 134952 2.27 MB/sec execute 126 sec latency 111.429 ms 2023-07-21T22:21:29.587 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:29 smithi194 ceph-mon[98288]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-07-21T22:21:29.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:29 smithi134 ceph-mon[97873]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-07-21T22:21:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:29 smithi093 ceph-mon[96579]: pgmap v156: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 849 KiB/s wr, 103 op/s 2023-07-21T22:21:30.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 135487 2.26 MB/sec execute 127 sec latency 364.033 ms 2023-07-21T22:21:31.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 135487 2.24 MB/sec execute 128 sec latency 1364.119 ms 2023-07-21T22:21:31.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:31 smithi194 ceph-mon[98288]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 781 KiB/s wr, 94 op/s 2023-07-21T22:21:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:31 smithi134 ceph-mon[97873]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 781 KiB/s wr, 94 op/s 2023-07-21T22:21:31.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 -- 172.21.15.93:0/4163575168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03840ffb30 msgr2=0x7f03840fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 --2- 172.21.15.93:0/4163575168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03840ffb30 0x7f03840fffa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0380009990 tx=0x7f0380009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 -- 172.21.15.93:0/4163575168 shutdown_connections 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 --2- 172.21.15.93:0/4163575168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03841004e0 0x7f038410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 --2- 172.21.15.93:0/4163575168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0384105b60 0x7f0384105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 --2- 172.21.15.93:0/4163575168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03840ffb30 0x7f03840fffa0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.596+0000 7f038b515700 1 -- 172.21.15.93:0/4163575168 >> 172.21.15.93:0/4163575168 conn(0x7f03840fb650 msgr2=0x7f03840fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- 172.21.15.93:0/4163575168 shutdown_connections 2023-07-21T22:21:31.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- 172.21.15.93:0/4163575168 wait complete. 2023-07-21T22:21:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 Processor -- start 2023-07-21T22:21:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- start start 2023-07-21T22:21:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03840ffb30 0x7f0384199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105b60 0x7f03841937b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384193e20 con 0x7f03841004e0 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384193f90 con 0x7f0384105b60 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f038b515700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384194150 con 0x7f03840ffb30 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.597+0000 7f0388ab0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46666/0 (socket says 172.21.15.93:46666) 2023-07-21T22:21:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 -- 172.21.15.93:0/2598872777 learned_addr learned my addr 172.21.15.93:0/2598872777 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0389ab2700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105b60 0x7f03841937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f03892b1700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03840ffb30 0x7f0384199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 -- 172.21.15.93:0/2598872777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03840ffb30 msgr2=0x7f0384199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03840ffb30 0x7f0384199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 -- 172.21.15.93:0/2598872777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105b60 msgr2=0x7f03841937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105b60 0x7f03841937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 -- 172.21.15.93:0/2598872777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800095f0 con 0x7f03841004e0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f0388ab0700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f038000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f037a7fc700 1 -- 172.21.15.93:0/2598872777 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038001d070 con 0x7f03841004e0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f037a7fc700 1 -- 172.21.15.93:0/2598872777 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f038000fc20 con 0x7f03841004e0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03841943d0 con 0x7f03841004e0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f037a7fc700 1 -- 172.21.15.93:0/2598872777 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380017650 con 0x7f03841004e0 2023-07-21T22:21:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.598+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03841a0940 con 0x7f03841004e0 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.601+0000 7f037a7fc700 1 -- 172.21.15.93:0/2598872777 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0380021410 con 0x7f03841004e0 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.601+0000 7f037a7fc700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 0x7f03700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.602+0000 7f03892b1700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 0x7f03700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.602+0000 7f037a7fc700 1 -- 172.21.15.93:0/2598872777 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f03800ce250 con 0x7f03841004e0 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.602+0000 7f03892b1700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 0x7f03700a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f037400f830 tx=0x7f0374009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.602+0000 7f039769ab80 1 Processor -- start 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.602+0000 7f039769ab80 1 -- start start 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b210 0x5632a3c54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3b5ac50 con 0x5632a3ab16b0 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3bad9e0 con 0x5632a3c5b660 2023-07-21T22:21:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f039769ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3ba6e70 con 0x5632a3c5b210 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46672/0 (socket says 172.21.15.93:46672) 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 -- 172.21.15.93:0/1963618390 learned_addr learned my addr 172.21.15.93:0/1963618390 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f0388ab0700 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b210 0x5632a3c54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 -- 172.21.15.93:0/1963618390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b210 msgr2=0x5632a3c54f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b210 0x5632a3c54f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f0389ab2700 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 -- 172.21.15.93:0/1963618390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 msgr2=0x5632a3c554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 -- 172.21.15.93:0/1963618390 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a39d4d80 con 0x5632a3ab16b0 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f03892b1700 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0374003c30 tx=0x7f0374018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ded075791479ecb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f036f7fe700 1 -- 172.21.15.93:0/1963618390 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374003840 con 0x5632a3ab16b0 2023-07-21T22:21:31.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f036f7fe700 1 -- 172.21.15.93:0/1963618390 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0374004830 con 0x5632a3ab16b0 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.603+0000 7f036f7fe700 1 -- 172.21.15.93:0/1963618390 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374005500 con 0x5632a3ab16b0 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 -- 172.21.15.93:0/1963618390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 msgr2=0x5632a3c5acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0374003c30 tx=0x7f0374018000 comp rx=0 tx=0).stop 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 -- 172.21.15.93:0/1963618390 shutdown_connections 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c5acd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 --2- 172.21.15.93:0/1963618390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b210 0x5632a3c54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 -- 172.21.15.93:0/1963618390 >> 172.21.15.93:0/1963618390 conn(0x5632a3c4aae0 msgr2=0x5632a3c500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 -- 172.21.15.93:0/1963618390 shutdown_connections 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.604+0000 7f039769ab80 1 -- 172.21.15.93:0/1963618390 wait complete. 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 -- start start 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 Processor -- start 2023-07-21T22:21:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3b46a60 con 0x5632a3c5b210 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3b46760 con 0x5632a3c5b660 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3a2a8c0 con 0x5632a3ab16b0 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f039769ab80 10 client.0 _ll_register_callbacks cb 0x5632a3b44700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f0388ab0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f0388ab0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46680/0 (socket says 172.21.15.93:46680) 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.605+0000 7f0388ab0700 1 -- 172.21.15.93:0/4253140612 learned_addr learned my addr 172.21.15.93:0/4253140612 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0389ab2700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 -- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 msgr2=0x5632a3c615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f03892b1700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 -- 172.21.15.93:0/4253140612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 msgr2=0x5632a3c5eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c5eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 -- 172.21.15.93:0/4253140612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03740039a0 con 0x5632a3c5b210 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f03892b1700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f0388ab0700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f03800a3200 tx=0x7f0380005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348dbc3b384b94e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038009f3a0 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f03800a3730 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632a3b00490 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03800a10e0 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f039769ab80 10 client.16518 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5632a39bdc00 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03800b7080 con 0x5632a3c5b210 2023-07-21T22:21:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03800a3380 con 0x5632a3c5b210 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.606+0000 7f039769ab80 10 client.16518 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f039769ab80 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f039769ab80 20 client.16518 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3c6e7d0 0x5632a3c70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f039769ab80 4 client.16518 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632a3b9fe50 con 0x5632a3c6e7d0 2023-07-21T22:21:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f03892b1700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3c6e7d0 0x5632a3c70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f03892b1700 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3c6e7d0 0x5632a3c70be0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341361f4f8bc6f3 server_cookie=3da89e151ba37543 in_seq=0 out_seq=0 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.607+0000 7f036cff9700 10 client.16518 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f036cff9700 1 -- 172.21.15.93:0/4253140612 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5632a3b9fe50 con 0x5632a3c6e7d0 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f036cff9700 10 client.16518 handle_command_reply: tid=0 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 client.16518 shutdown 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 msgr2=0x5632a3c5e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f03800a3200 tx=0x7f0380005fd0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 shutdown_connections 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3c6e7d0 0x5632a3c70be0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b660 0x5632a3c5eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f036cff9700 0 client.16518 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 --2- 172.21.15.93:0/4253140612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b210 0x5632a3c5e8b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 >> 172.21.15.93:0/4253140612 conn(0x5632a3c4aae0 msgr2=0x5632a3c4f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 shutdown_connections 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 1 -- 172.21.15.93:0/4253140612 wait complete. 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.608+0000 7f039769ab80 20 client.16518 trim_cache size 0 max 16384 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 Processor -- start 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 -- start start 2023-07-21T22:21:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c4fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b660 0x5632a3c500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c51760 con 0x5632a3ab16b0 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c55f20 con 0x5632a3c5b210 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f039769ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c5a920 con 0x5632a3c5b660 2023-07-21T22:21:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46686/0 (socket says 172.21.15.93:46686) 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 -- 172.21.15.93:0/2537903807 learned_addr learned my addr 172.21.15.93:0/2537903807 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 -- 172.21.15.93:0/2537903807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b660 msgr2=0x5632a3c500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b660 0x5632a3c500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 -- 172.21.15.93:0/2537903807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 msgr2=0x5632a3c4fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c4fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 -- 172.21.15.93:0/2537903807 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632a39fb1f0 con 0x5632a3ab16b0 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.623+0000 7f03892b1700 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0374029040 tx=0x7f0374003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c8c1143ad7c52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f036ffff700 1 -- 172.21.15.93:0/2537903807 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374004b60 con 0x5632a3ab16b0 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f036ffff700 1 -- 172.21.15.93:0/2537903807 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f037400f4e0 con 0x5632a3ab16b0 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f036ffff700 1 -- 172.21.15.93:0/2537903807 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374004fc0 con 0x5632a3ab16b0 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- 172.21.15.93:0/2537903807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 msgr2=0x5632a3c61160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0374029040 tx=0x7f0374003680 comp rx=0 tx=0).stop 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- 172.21.15.93:0/2537903807 shutdown_connections 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3c5b660 0x5632a3c500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3ab16b0 0x5632a3c61160 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 --2- 172.21.15.93:0/2537903807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c4fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- 172.21.15.93:0/2537903807 >> 172.21.15.93:0/2537903807 conn(0x5632a3c4aae0 msgr2=0x5632a3c58ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- 172.21.15.93:0/2537903807 shutdown_connections 2023-07-21T22:21:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- 172.21.15.93:0/2537903807 wait complete. 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.624+0000 7f039769ab80 1 -- start start 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 Processor -- start 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c65870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c65db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c64420 con 0x5632a3c5b660 2023-07-21T22:21:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c64590 con 0x5632a3c5b210 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632a3c64750 con 0x5632a3ab16b0 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f039769ab80 10 client.0 _ll_register_callbacks cb 0x5632a3b44700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46696/0 (socket says 172.21.15.93:46696) 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 -- 172.21.15.93:0/450460284 learned_addr learned my addr 172.21.15.93:0/450460284 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f03892b1700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c65870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 -- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 msgr2=0x5632a3c65870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c65870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0388ab0700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c65db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 -- 172.21.15.93:0/450460284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 msgr2=0x5632a3c65db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c65db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 -- 172.21.15.93:0/450460284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632a39d4d80 con 0x5632a3c5b660 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f0389ab2700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f037c009670 tx=0x7f037c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e0da3ac0aca7d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.625+0000 7f036cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00e070 con 0x5632a3c5b660 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f037c016610 con 0x5632a3c5b660 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0374004830 con 0x5632a3c5b660 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037c0167d0 con 0x5632a3c5b660 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 10 client.16524 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5632a39d3250 con 0x5632a3c5b660 2023-07-21T22:21:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0374004830 con 0x5632a3c5b660 2023-07-21T22:21:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f037c01eb50 con 0x5632a3c5b660 2023-07-21T22:21:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 10 client.16524 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 20 client.16524 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3cc3ba0 0x5632a3cc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 4 client.16524 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:21:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632a3b61ff0 con 0x5632a3cc3ba0 2023-07-21T22:21:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.626+0000 7f03892b1700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3cc3ba0 0x5632a3cc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.627+0000 7f036cff9700 10 client.16524 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.627+0000 7f03892b1700 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3cc3ba0 0x5632a3cc5fb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d721b7de75a7a server_cookie=6ed18ff5453ff762 in_seq=0 out_seq=0 2023-07-21T22:21:31.733 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:31 smithi093 ceph-mon[96579]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 781 KiB/s wr, 94 op/s 2023-07-21T22:21:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f036cff9700 1 -- 172.21.15.93:0/450460284 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632a3b61ff0 con 0x5632a3cc3ba0 2023-07-21T22:21:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f036cff9700 10 client.16524 handle_command_reply: tid=0 2023-07-21T22:21:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 client.16524 shutdown 2023-07-21T22:21:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 msgr2=0x5632a3c63d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f037c009670 tx=0x7f037c007b10 comp rx=0 tx=0).stop 2023-07-21T22:21:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 shutdown_connections 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5632a3cc3ba0 0x5632a3cc5fb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5632a3c5b660 0x5632a3c63d10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f036cff9700 0 client.16524 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5632a3ab16b0 0x5632a3c65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 --2- 172.21.15.93:0/450460284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a3c5b210 0x5632a3c65db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 >> 172.21.15.93:0/450460284 conn(0x5632a3c4aae0 msgr2=0x5632a3c599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 shutdown_connections 2023-07-21T22:21:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.654+0000 7f039769ab80 1 -- 172.21.15.93:0/450460284 wait complete. 2023-07-21T22:21:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f039769ab80 20 client.16524 trim_cache size 0 max 16384 2023-07-21T22:21:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:21:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:21:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:21:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:21:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 msgr2=0x7f03700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 0x7f03700a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f037400f830 tx=0x7f0374009260 comp rx=0 tx=0).stop 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 msgr2=0x7f0384199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f038000f510 comp rx=0 tx=0).stop 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 shutdown_connections 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0384105b60 0x7f03841937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03840ffb30 0x7f0384199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f03700a12c0 0x7f03700a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 --2- 172.21.15.93:0/2598872777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03841004e0 0x7f0384199e90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 >> 172.21.15.93:0/2598872777 conn(0x7f03840fb650 msgr2=0x7f03840fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 shutdown_connections 2023-07-21T22:21:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:31.655+0000 7f038b515700 1 -- 172.21.15.93:0/2598872777 wait complete. 2023-07-21T22:21:32.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 135487 2.22 MB/sec execute 129 sec latency 2364.214 ms 2023-07-21T22:21:33.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 136350 2.25 MB/sec execute 130 sec latency 2869.032 ms 2023-07-21T22:21:33.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:33 smithi194 ceph-mon[98288]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 997 KiB/s wr, 144 op/s 2023-07-21T22:21:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:33 smithi134 ceph-mon[97873]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 997 KiB/s wr, 144 op/s 2023-07-21T22:21:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:33 smithi093 ceph-mon[96579]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 997 KiB/s wr, 144 op/s 2023-07-21T22:21:34.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 138610 2.33 MB/sec execute 131 sec latency 14.820 ms 2023-07-21T22:21:35.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 139090 2.32 MB/sec execute 132 sec latency 479.007 ms 2023-07-21T22:21:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:35 smithi194 ceph-mon[98288]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:21:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:35 smithi134 ceph-mon[97873]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:21:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:35 smithi093 ceph-mon[96579]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:21:36.325 INFO:tasks.workunit.client.0.smithi093.stdout: 1 139090 2.30 MB/sec execute 133 sec latency 1479.128 ms 2023-07-21T22:21:37.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 139090 2.29 MB/sec execute 134 sec latency 2479.202 ms 2023-07-21T22:21:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:37 smithi194 ceph-mon[98288]: pgmap v160: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:37.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:37 smithi134 ceph-mon[97873]: pgmap v160: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:37.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:37 smithi093 ceph-mon[96579]: pgmap v160: 97 pgs: 97 active+clean; 64 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 677 KiB/s wr, 80 op/s 2023-07-21T22:21:38.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 139897 2.30 MB/sec execute 135 sec latency 2989.152 ms 2023-07-21T22:21:39.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142151 2.39 MB/sec execute 136 sec latency 16.609 ms 2023-07-21T22:21:39.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:39 smithi194 ceph-mon[98288]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-07-21T22:21:39.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:39 smithi134 ceph-mon[97873]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-07-21T22:21:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:39 smithi093 ceph-mon[96579]: pgmap v161: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2023-07-21T22:21:40.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142633 2.37 MB/sec execute 137 sec latency 466.909 ms 2023-07-21T22:21:41.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142633 2.36 MB/sec execute 138 sec latency 1466.990 ms 2023-07-21T22:21:41.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:41 smithi194 ceph-mon[98288]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 806 KiB/s wr, 109 op/s 2023-07-21T22:21:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:41 smithi134 ceph-mon[97873]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 806 KiB/s wr, 109 op/s 2023-07-21T22:21:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:41 smithi093 ceph-mon[96579]: pgmap v162: 97 pgs: 97 active+clean; 70 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 806 KiB/s wr, 109 op/s 2023-07-21T22:21:42.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142633 2.34 MB/sec execute 139 sec latency 2467.055 ms 2023-07-21T22:21:43.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142701 2.32 MB/sec execute 140 sec latency 2973.053 ms 2023-07-21T22:21:43.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:43 smithi194 ceph-mon[98288]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-07-21T22:21:43.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:43 smithi134 ceph-mon[97873]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-07-21T22:21:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:43 smithi093 ceph-mon[96579]: pgmap v163: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.1 MiB/s wr, 166 op/s 2023-07-21T22:21:44.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142701 2.31 MB/sec execute 141 sec latency 1444.259 ms 2023-07-21T22:21:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 -- 172.21.15.93:0/2373860273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 msgr2=0x7f66b40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/2373860273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b40feea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009990 tx=0x7f66a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 -- 172.21.15.93:0/2373860273 shutdown_connections 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/2373860273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 0x7f66b410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/2373860273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/2373860273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b40feea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 -- 172.21.15.93:0/2373860273 >> 172.21.15.93:0/2373860273 conn(0x7f66b4074760 msgr2=0x7f66b4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 -- 172.21.15.93:0/2373860273 shutdown_connections 2023-07-21T22:21:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.770+0000 7f66b8f7e700 1 -- 172.21.15.93:0/2373860273 wait complete. 2023-07-21T22:21:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 Processor -- start 2023-07-21T22:21:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 -- start start 2023-07-21T22:21:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 0x7f66b4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b419d040 con 0x7f66b40fea30 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b419d1b0 con 0x7f66b4104a30 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b8f7e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b419d370 con 0x7f66b40ff3e0 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46706/0 (socket says 172.21.15.93:46706) 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b259c700 1 -- 172.21.15.93:0/3399838915 learned_addr learned my addr 172.21.15.93:0/3399838915 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 0x7f66b4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.771+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 -- 172.21.15.93:0/3399838915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 msgr2=0x7f66b4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 0x7f66b4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 -- 172.21.15.93:0/3399838915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 msgr2=0x7f66b419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 -- 172.21.15.93:0/3399838915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a40095f0 con 0x7f66b40fea30 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b259c700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b500 tx=0x7f66a800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66a37fe700 1 -- 172.21.15.93:0/3399838915 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8011830 con 0x7f66b40fea30 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66a37fe700 1 -- 172.21.15.93:0/3399838915 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a8011e50 con 0x7f66b40fea30 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b419d650 con 0x7f66b40fea30 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66a37fe700 1 -- 172.21.15.93:0/3399838915 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800f2e0 con 0x7f66b40fea30 2023-07-21T22:21:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.772+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b41a4e90 con 0x7f66b40fea30 2023-07-21T22:21:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.774+0000 7f66a37fe700 1 -- 172.21.15.93:0/3399838915 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f66a8010580 con 0x7f66b40fea30 2023-07-21T22:21:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.775+0000 7f66a37fe700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 0x7f669c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.775+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 0x7f669c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.775+0000 7f66a37fe700 1 -- 172.21.15.93:0/3399838915 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66a80cc670 con 0x7f66b40fea30 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.775+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 0x7f669c0a37f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66b4199670 tx=0x7f66a4005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.775+0000 7f66c5103b80 1 Processor -- start 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 -- start start 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b70f6b0 0x55d55b8b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 0x55d55b8b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b7b8c50 con 0x55d55b8b9660 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b80b9e0 con 0x55d55b8b9210 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66c5103b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b804e70 con 0x55d55b70f6b0 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46718/0 (socket says 172.21.15.93:46718) 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1297148376 learned_addr learned my addr 172.21.15.93:0/1297148376 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b259c700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b70f6b0 0x55d55b8b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 0x55d55b8b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1297148376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b70f6b0 msgr2=0x55d55b8b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b70f6b0 0x55d55b8b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1297148376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 msgr2=0x55d55b8b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 0x55d55b8b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1297148376 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55b632d80 con 0x55d55b8b9660 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 0x55d55b8b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.776+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0087c0 tx=0x7f66ac008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fc7fb4f8584eb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66a0ff9700 1 -- 172.21.15.93:0/1297148376 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac003c00 con 0x55d55b8b9660 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66a0ff9700 1 -- 172.21.15.93:0/1297148376 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac010040 con 0x55d55b8b9660 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66a0ff9700 1 -- 172.21.15.93:0/1297148376 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac012760 con 0x55d55b8b9660 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 -- 172.21.15.93:0/1297148376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 msgr2=0x55d55b8b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0087c0 tx=0x7f66ac008ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 -- 172.21.15.93:0/1297148376 shutdown_connections 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9660 0x55d55b8b34a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b70f6b0 0x55d55b8b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1297148376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9210 0x55d55b8b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.777+0000 7f66c5103b80 1 -- 172.21.15.93:0/1297148376 >> 172.21.15.93:0/1297148376 conn(0x55d55b8a8ae0 msgr2=0x55d55b8ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- 172.21.15.93:0/1297148376 shutdown_connections 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- 172.21.15.93:0/1297148376 wait complete. 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- start start 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 Processor -- start 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9660 0x55d55b8bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b7a4a60 con 0x55d55b70f6b0 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b7a4760 con 0x55d55b8b9660 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b6888c0 con 0x55d55b8b9210 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.778+0000 7f66c5103b80 10 client.0 _ll_register_callbacks cb 0x55d55b7a2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46720/0 (socket says 172.21.15.93:46720) 2023-07-21T22:21:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 -- 172.21.15.93:0/581078082 learned_addr learned my addr 172.21.15.93:0/581078082 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 -- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 msgr2=0x55d55b8bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9660 0x55d55b8bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 -- 172.21.15.93:0/581078082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9660 msgr2=0x55d55b8bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9660 0x55d55b8bcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 -- 172.21.15.93:0/581078082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0083f0 con 0x55d55b70f6b0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66b259c700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f66a8099510 tx=0x7f66a80992a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f1f5fe3cd3b7e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f6699ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80b1070 con 0x55d55b70f6b0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66a80b7e40 con 0x55d55b70f6b0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d55b75e490 con 0x55d55b70f6b0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a80b92c0 con 0x55d55b70f6b0 2023-07-21T22:21:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66c5103b80 10 client.16536 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d55b61bc00 con 0x55d55b70f6b0 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.779+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a80c8070 con 0x55d55b70f6b0 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66a80a17d0 con 0x55d55b70f6b0 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 20 client.16536 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8cca10 0x55d55b8cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 4 client.16536 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d55b7fde50 con 0x55d55b8cca10 2023-07-21T22:21:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8cca10 0x55d55b8cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f6699ffb700 10 client.16536 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.780+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8cca10 0x55d55b8cee20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3249f568e1f6623c server_cookie=fdd7c37dc96bd2f0 in_seq=0 out_seq=0 2023-07-21T22:21:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f6699ffb700 1 -- 172.21.15.93:0/581078082 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d55b7fde50 con 0x55d55b8cca10 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f6699ffb700 10 client.16536 handle_command_reply: tid=0 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 client.16536 shutdown 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 msgr2=0x55d55b8bf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f66a8099510 tx=0x7f66a80992a0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 shutdown_connections 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8cca10 0x55d55b8cee20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b8b9660 0x55d55b8bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b70f6b0 0x55d55b8bf550 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f6699ffb700 0 client.16536 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 --2- 172.21.15.93:0/581078082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 >> 172.21.15.93:0/581078082 conn(0x55d55b8a8ae0 msgr2=0x55d55b8ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 shutdown_connections 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 1 -- 172.21.15.93:0/581078082 wait complete. 2023-07-21T22:21:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.781+0000 7f66c5103b80 20 client.16536 trim_cache size 0 max 16384 2023-07-21T22:21:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 Processor -- start 2023-07-21T22:21:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 -- start start 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8aefb0 con 0x55d55b8b9210 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8b38a0 con 0x55d55b70f6b0 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66c5103b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8b3d30 con 0x55d55b8a8ae0 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46732/0 (socket says 172.21.15.93:46732) 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1175130431 learned_addr learned my addr 172.21.15.93:0/1175130431 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b259c700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1175130431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 msgr2=0x55d55b8b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1175130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 msgr2=0x55d55b8ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8ba1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b2d9d700 1 -- 172.21.15.93:0/1175130431 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55b6591f0 con 0x55d55b8b9210 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.796+0000 7f66b259c700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f66ac016040 tx=0x7f66ac006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd08ce9d2649211f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66a17fa700 1 -- 172.21.15.93:0/1175130431 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac007770 con 0x55d55b8b9210 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66a17fa700 1 -- 172.21.15.93:0/1175130431 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac003a00 con 0x55d55b8b9210 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66a17fa700 1 -- 172.21.15.93:0/1175130431 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0045c0 con 0x55d55b8b9210 2023-07-21T22:21:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 -- 172.21.15.93:0/1175130431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 msgr2=0x55d55b8c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f66ac016040 tx=0x7f66ac006000 comp rx=0 tx=0).stop 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 -- 172.21.15.93:0/1175130431 shutdown_connections 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8b9210 0x55d55b8c3b50 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 --2- 172.21.15.93:0/1175130431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 -- 172.21.15.93:0/1175130431 >> 172.21.15.93:0/1175130431 conn(0x55d55b8c9080 msgr2=0x55d55b8b9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 -- 172.21.15.93:0/1175130431 shutdown_connections 2023-07-21T22:21:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.797+0000 7f66c5103b80 1 -- 172.21.15.93:0/1175130431 wait complete. 2023-07-21T22:21:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 -- start start 2023-07-21T22:21:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 Processor -- start 2023-07-21T22:21:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8b54e0 con 0x55d55b8a8ae0 2023-07-21T22:21:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8b5650 con 0x55d55b70f6b0 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b9208d0 con 0x55d55b8b9210 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66c5103b80 10 client.0 _ll_register_callbacks cb 0x7f66a80996d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46736/0 (socket says 172.21.15.93:46736) 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 -- 172.21.15.93:0/389420275 learned_addr learned my addr 172.21.15.93:0/389420275 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b259c700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 -- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 msgr2=0x55d55b8b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 -- 172.21.15.93:0/389420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 msgr2=0x55d55b8c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b1d9b700 1 -- 172.21.15.93:0/389420275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d55b632d80 con 0x55d55b8a8ae0 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.798+0000 7f66b2d9d700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66b1d9b700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f66a40098e0 tx=0x7f66a401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a842dde65ae39044 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:44.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4032440 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f66a40325a0 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66ac010040 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a4004a70 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 10 client.16542 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d55b631250 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac010040 con 0x55d55b8a8ae0 2023-07-21T22:21:44.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f66a402f050 con 0x55d55b8a8ae0 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 20 client.16542 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8b9660 0x55d55b923f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 4 client.16542 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.799+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d55b7bfff0 con 0x55d55b8b9660 2023-07-21T22:21:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.800+0000 7f66b259c700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8b9660 0x55d55b923f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.800+0000 7f6699ffb700 10 client.16542 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.800+0000 7f66b259c700 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8b9660 0x55d55b923f80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a31079a4911b5af server_cookie=b60fa85823ae7960 in_seq=0 out_seq=0 2023-07-21T22:21:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.828+0000 7f6699ffb700 1 -- 172.21.15.93:0/389420275 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d55b7bfff0 con 0x55d55b8b9660 2023-07-21T22:21:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.828+0000 7f6699ffb700 10 client.16542 handle_command_reply: tid=0 2023-07-21T22:21:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.828+0000 7f66c5103b80 1 client.16542 shutdown 2023-07-21T22:21:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 msgr2=0x55d55b8b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f66a40098e0 tx=0x7f66a401a050 comp rx=0 tx=0).stop 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 shutdown_connections 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d55b8b9660 0x55d55b923f80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d55b8b9210 0x55d55b8b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d55b70f6b0 0x55d55b8c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 --2- 172.21.15.93:0/389420275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d55b8a8ae0 0x55d55b8b48d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 >> 172.21.15.93:0/389420275 conn(0x55d55b8c9080 msgr2=0x55d55b8ba2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f6699ffb700 0 client.16542 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 shutdown_connections 2023-07-21T22:21:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 1 -- 172.21.15.93:0/389420275 wait complete. 2023-07-21T22:21:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66c5103b80 20 client.16542 trim_cache size 0 max 16384 2023-07-21T22:21:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:21:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:21:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:21:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:21:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 msgr2=0x7f669c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 0x7f669c0a37f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66b4199670 tx=0x7f66a4005fd0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 msgr2=0x7f66b4197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b500 tx=0x7f66a800b810 comp rx=0 tx=0).stop 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 shutdown_connections 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b4104a30 0x7f66b419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f669c0a1340 0x7f669c0a37f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66b40fea30 0x7f66b4197e50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 --2- 172.21.15.93:0/3399838915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66b40ff3e0 0x7f66b4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.829+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 >> 172.21.15.93:0/3399838915 conn(0x7f66b4074760 msgr2=0x7f66b4067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.830+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 shutdown_connections 2023-07-21T22:21:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:44.830+0000 7f66b8f7e700 1 -- 172.21.15.93:0/3399838915 wait complete. 2023-07-21T22:21:45.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142701 2.29 MB/sec execute 142 sec latency 2444.352 ms 2023-07-21T22:21:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:45 smithi194 ceph-mon[98288]: pgmap v164: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:45.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:45 smithi134 ceph-mon[97873]: pgmap v164: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:45.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:45 smithi093 ceph-mon[96579]: pgmap v164: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:46.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142701 2.28 MB/sec execute 143 sec latency 3444.465 ms 2023-07-21T22:21:47.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 142701 2.26 MB/sec execute 144 sec latency 4444.569 ms 2023-07-21T22:21:47.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:47 smithi194 ceph-mon[98288]: pgmap v165: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:47 smithi134 ceph-mon[97873]: pgmap v165: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:47 smithi093 ceph-mon[96579]: pgmap v165: 97 pgs: 97 active+clean; 70 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 832 KiB/s wr, 107 op/s 2023-07-21T22:21:48.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 143597 2.28 MB/sec execute 145 sec latency 4952.215 ms 2023-07-21T22:21:49.326 INFO:tasks.workunit.client.0.smithi093.stdout: 1 145742 2.36 MB/sec execute 146 sec latency 14.761 ms 2023-07-21T22:21:49.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:49 smithi194 ceph-mon[98288]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-07-21T22:21:49.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:49 smithi134 ceph-mon[97873]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-07-21T22:21:49.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:49 smithi093 ceph-mon[96579]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2023-07-21T22:21:50.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146126 2.34 MB/sec execute 147 sec latency 591.830 ms 2023-07-21T22:21:51.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146126 2.33 MB/sec execute 148 sec latency 1591.920 ms 2023-07-21T22:21:51.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:51 smithi194 ceph-mon[98288]: pgmap v167: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 610 KiB/s wr, 77 op/s 2023-07-21T22:21:51.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:51 smithi134 ceph-mon[97873]: pgmap v167: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 610 KiB/s wr, 77 op/s 2023-07-21T22:21:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:51 smithi093 ceph-mon[96579]: pgmap v167: 97 pgs: 97 active+clean; 69 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 610 KiB/s wr, 77 op/s 2023-07-21T22:21:52.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146126 2.31 MB/sec execute 149 sec latency 2592.016 ms 2023-07-21T22:21:53.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146182 2.30 MB/sec execute 150 sec latency 3103.020 ms 2023-07-21T22:21:53.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:53 smithi194 ceph-mon[98288]: pgmap v168: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:21:53.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:53 smithi134 ceph-mon[97873]: pgmap v168: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:21:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:53 smithi093 ceph-mon[96579]: pgmap v168: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:21:54.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146182 2.28 MB/sec execute 151 sec latency 1447.397 ms 2023-07-21T22:21:55.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146182 2.27 MB/sec execute 152 sec latency 2447.478 ms 2023-07-21T22:21:55.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:55 smithi194 ceph-mon[98288]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:55.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:55 smithi134 ceph-mon[97873]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:55 smithi093 ceph-mon[96579]: pgmap v169: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:56.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146182 2.25 MB/sec execute 153 sec latency 3447.572 ms 2023-07-21T22:21:57.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146182 2.24 MB/sec execute 154 sec latency 4447.685 ms 2023-07-21T22:21:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:57 smithi194 ceph-mon[98288]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:57 smithi134 ceph-mon[97873]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:57 smithi093 ceph-mon[96579]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 773 KiB/s wr, 73 op/s 2023-07-21T22:21:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 -- 172.21.15.93:0/2121937333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e80685b0 msgr2=0x7f45e8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 --2- 172.21.15.93:0/2121937333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e80685b0 0x7f45e8104770 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f45d80099c0 tx=0x7f45d8009cd0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 -- 172.21.15.93:0/2121937333 shutdown_connections 2023-07-21T22:21:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 --2- 172.21.15.93:0/2121937333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e8104e50 0x7f45e810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 --2- 172.21.15.93:0/2121937333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 0x7f45e8067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 --2- 172.21.15.93:0/2121937333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e80685b0 0x7f45e8104770 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.956+0000 7f45ef39d700 1 -- 172.21.15.93:0/2121937333 >> 172.21.15.93:0/2121937333 conn(0x7f45e8074760 msgr2=0x7f45e8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.957+0000 7f45ef39d700 1 -- 172.21.15.93:0/2121937333 shutdown_connections 2023-07-21T22:21:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.957+0000 7f45ef39d700 1 -- 172.21.15.93:0/2121937333 wait complete. 2023-07-21T22:21:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.957+0000 7f45ef39d700 1 Processor -- start 2023-07-21T22:21:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.957+0000 7f45ef39d700 1 -- start start 2023-07-21T22:21:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 0x7f45e80fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e80685b0 0x7f45e80ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a42d0 con 0x7f45e8104e50 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a4410 con 0x7f45e8067c10 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e81a4580 con 0x7f45e80685b0 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed139700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 0x7f45e80fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46752/0 (socket says 172.21.15.93:46752) 2023-07-21T22:21:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 -- 172.21.15.93:0/1959189168 learned_addr learned my addr 172.21.15.93:0/1959189168 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ec938700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e80685b0 0x7f45e80ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 -- 172.21.15.93:0/1959189168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e80685b0 msgr2=0x7f45e80ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e80685b0 0x7f45e80ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 -- 172.21.15.93:0/1959189168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 msgr2=0x7f45e80fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 0x7f45e80fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 -- 172.21.15.93:0/1959189168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80095f0 con 0x7f45e8104e50 2023-07-21T22:21:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ed93a700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007ee0 tx=0x7f45e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45de7fc700 1 -- 172.21.15.93:0/1959189168 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45e8104e50 2023-07-21T22:21:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45de7fc700 1 -- 172.21.15.93:0/1959189168 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e000f040 con 0x7f45e8104e50 2023-07-21T22:21:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e81a4860 con 0x7f45e8104e50 2023-07-21T22:21:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.958+0000 7f45de7fc700 1 -- 172.21.15.93:0/1959189168 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0005580 con 0x7f45e8104e50 2023-07-21T22:21:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.959+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81a4cf0 con 0x7f45e8104e50 2023-07-21T22:21:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.961+0000 7f45de7fc700 1 -- 172.21.15.93:0/1959189168 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f45e000db20 con 0x7f45e8104e50 2023-07-21T22:21:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.962+0000 7f45de7fc700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 0x7f45d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.962+0000 7f45ed139700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 0x7f45d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.962+0000 7f45de7fc700 1 -- 172.21.15.93:0/1959189168 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f45e00cc430 con 0x7f45e8104e50 2023-07-21T22:21:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.962+0000 7f45ed139700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 0x7f45d40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005950 tx=0x7f45e400b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 Processor -- start 2023-07-21T22:21:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 -- start start 2023-07-21T22:21:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efde61c50 con 0x561efdf62660 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdeb49e0 con 0x561efddb86b0 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.963+0000 7f45fb522b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdeade70 con 0x561efdf62210 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ec938700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46760/0 (socket says 172.21.15.93:46760) 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 -- 172.21.15.93:0/598171022 learned_addr learned my addr 172.21.15.93:0/598171022 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ec938700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:33838/0 (socket says 172.21.15.93:33838) 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed139700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 -- 172.21.15.93:0/598171022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 msgr2=0x561efdf5bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 -- 172.21.15.93:0/598171022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 msgr2=0x561efdf61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 -- 172.21.15.93:0/598171022 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efdcdbd80 con 0x561efdf62660 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ec938700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45ed93a700 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f45e00a1440 tx=0x7f45e0095260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bf466d1254e10a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45d37fe700 1 -- 172.21.15.93:0/598171022 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0095040 con 0x561efdf62660 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45d37fe700 1 -- 172.21.15.93:0/598171022 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e00a1970 con 0x561efdf62660 2023-07-21T22:21:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.964+0000 7f45d37fe700 1 -- 172.21.15.93:0/598171022 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0095c30 con 0x561efdf62660 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 -- 172.21.15.93:0/598171022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 msgr2=0x561efdf5c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f45e00a1440 tx=0x7f45e0095260 comp rx=0 tx=0).stop 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 -- 172.21.15.93:0/598171022 shutdown_connections 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf5c4a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 --2- 172.21.15.93:0/598171022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 -- 172.21.15.93:0/598171022 >> 172.21.15.93:0/598171022 conn(0x561efdf51ae0 msgr2=0x561efdf570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 -- 172.21.15.93:0/598171022 shutdown_connections 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.965+0000 7f45fb522b80 1 -- 172.21.15.93:0/598171022 wait complete. 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 -- start start 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 Processor -- start 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf65890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf65fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efde4da60 con 0x561efdf62660 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efde4d760 con 0x561efddb86b0 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdd318c0 con 0x561efdf62210 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45fb522b80 10 client.0 _ll_register_callbacks cb 0x561efde4b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45ed139700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45ed139700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:57924/0 (socket says 172.21.15.93:57924) 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45ed139700 1 -- 172.21.15.93:0/488698135 learned_addr learned my addr 172.21.15.93:0/488698135 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.966+0000 7f45ec938700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf65890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed93a700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf65fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 -- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 msgr2=0x561efdf65890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf65890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 -- 172.21.15.93:0/488698135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 msgr2=0x561efdf65fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf65fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 -- 172.21.15.93:0/488698135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e810d210 con 0x561efddb86b0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45ed139700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45e40116f0 tx=0x7f45e40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6920abff1b76e8ca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4028470 con 0x561efddb86b0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561efde07490 con 0x561efddb86b0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f45e4017070 con 0x561efddb86b0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e40285d0 con 0x561efddb86b0 2023-07-21T22:21:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45fb522b80 10 client.25967 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.967+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f45e81a4cf0 con 0x561efddb86b0 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561efde07490 con 0x561efddb86b0 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f45e401f870 con 0x561efddb86b0 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 20 client.25967 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdf75c00 0x561efdf78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 4 client.25967 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561efdea6e50 con 0x561efdf75c00 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45ec938700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdf75c00 0x561efdf78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45d0ff9700 10 client.25967 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.968+0000 7f45ec938700 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdf75c00 0x561efdf78010 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eeba32ee4a096bb server_cookie=3bb8847caf2303b1 in_seq=0 out_seq=0 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45d0ff9700 1 -- 172.21.15.93:0/488698135 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561efdea6e50 con 0x561efdf75c00 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45d0ff9700 10 client.25967 handle_command_reply: tid=0 2023-07-21T22:21:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 client.25967 shutdown 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 msgr2=0x561efdf68610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45e40116f0 tx=0x7f45e40047c0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 shutdown_connections 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdf75c00 0x561efdf78010 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62660 0x561efdf65fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efddb86b0 0x561efdf68610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45d0ff9700 0 client.25967 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 --2- 172.21.15.93:0/488698135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62210 0x561efdf65890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 >> 172.21.15.93:0/488698135 conn(0x561efdf51ae0 msgr2=0x561efdf563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 shutdown_connections 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 1 -- 172.21.15.93:0/488698135 wait complete. 2023-07-21T22:21:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.969+0000 7f45fb522b80 20 client.25967 trim_cache size 0 max 16384 2023-07-21T22:21:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 Processor -- start 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 -- start start 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62660 0x561efdf631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdf588d0 con 0x561efddb86b0 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdf731a0 con 0x561efdf62210 2023-07-21T22:21:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45fb522b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdf58510 con 0x561efdf62660 2023-07-21T22:21:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ed139700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ed139700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46770/0 (socket says 172.21.15.93:46770) 2023-07-21T22:21:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ed139700 1 -- 172.21.15.93:0/2114749453 learned_addr learned my addr 172.21.15.93:0/2114749453 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ec938700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ed93a700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62660 0x561efdf631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ec938700 1 -- 172.21.15.93:0/2114749453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62660 msgr2=0x561efdf631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ec938700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62660 0x561efdf631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ec938700 1 -- 172.21.15.93:0/2114749453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 msgr2=0x561efdf61200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.988+0000 7f45ec938700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45ec938700 1 -- 172.21.15.93:0/2114749453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efdf7cc00 con 0x561efdf62210 2023-07-21T22:21:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45ed139700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45ec938700 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 0x561efdf5bf40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f45d80098e0 tx=0x7f45d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabb40f4ac61a716 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45d3fff700 1 -- 172.21.15.93:0/2114749453 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d802a070 con 0x561efdf62210 2023-07-21T22:21:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45d3fff700 1 -- 172.21.15.93:0/2114749453 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d802e070 con 0x561efdf62210 2023-07-21T22:21:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.989+0000 7f45d3fff700 1 -- 172.21.15.93:0/2114749453 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d801d040 con 0x561efdf62210 2023-07-21T22:21:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- 172.21.15.93:0/2114749453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 msgr2=0x561efdf5bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 0x561efdf5bf40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f45d80098e0 tx=0x7f45d8000bc0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- 172.21.15.93:0/2114749453 shutdown_connections 2023-07-21T22:21:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efdf62660 0x561efdf631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efddb86b0 0x561efdf61200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 --2- 172.21.15.93:0/2114749453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62210 0x561efdf5bf40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- 172.21.15.93:0/2114749453 >> 172.21.15.93:0/2114749453 conn(0x561efdf51ae0 msgr2=0x561efdf62aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- 172.21.15.93:0/2114749453 shutdown_connections 2023-07-21T22:21:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- 172.21.15.93:0/2114749453 wait complete. 2023-07-21T22:21:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 -- start start 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.990+0000 7f45fb522b80 1 Processor -- start 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efddb86b0 0x561efdf6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62660 0x561efdf69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdf69d90 con 0x561efdf62210 2023-07-21T22:21:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdf69f00 con 0x561efdf62660 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efdfc9a80 con 0x561efddb86b0 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 10 client.0 _ll_register_callbacks cb 0x7f45e4010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46778/0 (socket says 172.21.15.93:46778) 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 -- 172.21.15.93:0/3644500020 learned_addr learned my addr 172.21.15.93:0/3644500020 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ed139700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efddb86b0 0x561efdf6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ed93a700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62660 0x561efdf69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 -- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efddb86b0 msgr2=0x561efdf6cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efddb86b0 0x561efdf6cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 -- 172.21.15.93:0/3644500020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62660 msgr2=0x561efdf69680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62660 0x561efdf69680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 -- 172.21.15.93:0/3644500020 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e81a4cf0 con 0x561efdf62210 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45ec938700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45d80059e0 tx=0x7f45d8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760f1706d926fe5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d802a050 con 0x561efdf62210 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45e00a1970 con 0x561efdf62210 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.991+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f45d800b930 con 0x561efdf62210 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d801d040 con 0x561efdf62210 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 10 client.16560 fetch_fsmap learned FSMap version 41 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f45d8005360 con 0x561efdf62210 2023-07-21T22:21:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45e00a1970 con 0x561efdf62210 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f45d80099f0 con 0x561efdf62210 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 10 client.16560 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 20 client.16560 populate_metadata read hostname 'smithi093' 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdfcadb0 0x561efdfcd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 4 client.16560 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561efde68ff0 con 0x561efdfcadb0 2023-07-21T22:21:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45ed139700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdfcadb0 0x561efdfcd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45d0ff9700 10 client.16560 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:57.992+0000 7f45ed139700 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdfcadb0 0x561efdfcd1c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f779490a9bca58 server_cookie=fb9f0c979f6f402d in_seq=0 out_seq=0 2023-07-21T22:21:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.016+0000 7f45d0ff9700 1 -- 172.21.15.93:0/3644500020 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561efde68ff0 con 0x561efdfcadb0 2023-07-21T22:21:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.016+0000 7f45d0ff9700 10 client.16560 handle_command_reply: tid=0 2023-07-21T22:21:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.016+0000 7f45fb522b80 1 client.16560 shutdown 2023-07-21T22:21:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 msgr2=0x561efdf69140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45d80059e0 tx=0x7f45d8005a10 comp rx=0 tx=0).stop 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 shutdown_connections 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561efdfcadb0 0x561efdfcd1c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561efdf62660 0x561efdf69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45d0ff9700 0 client.16560 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561efddb86b0 0x561efdf6cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 --2- 172.21.15.93:0/3644500020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561efdf62210 0x561efdf69140 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 >> 172.21.15.93:0/3644500020 conn(0x561efdf51ae0 msgr2=0x561efdf61070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 shutdown_connections 2023-07-21T22:21:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 1 -- 172.21.15.93:0/3644500020 wait complete. 2023-07-21T22:21:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45fb522b80 20 client.16560 trim_cache size 0 max 16384 2023-07-21T22:21:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:21:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:21:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:21:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 msgr2=0x7f45d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 0x7f45d40a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005950 tx=0x7f45e400b410 comp rx=0 tx=0).stop 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 msgr2=0x7f45e81a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007ee0 tx=0x7f45e000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 shutdown_connections 2023-07-21T22:21:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f45d40a12c0 0x7f45d40a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45e8104e50 0x7f45e81a3d90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45e8067c10 0x7f45e80fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 --2- 172.21.15.93:0/1959189168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45e80685b0 0x7f45e80ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.017+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 >> 172.21.15.93:0/1959189168 conn(0x7f45e8074760 msgr2=0x7f45e8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.018+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 shutdown_connections 2023-07-21T22:21:58.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:21:58.018+0000 7f45ef39d700 1 -- 172.21.15.93:0/1959189168 wait complete. 2023-07-21T22:21:58.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 146991 2.25 MB/sec execute 155 sec latency 4955.047 ms 2023-07-21T22:21:59.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 149253 2.33 MB/sec execute 156 sec latency 14.508 ms 2023-07-21T22:21:59.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:21:59 smithi194 ceph-mon[98288]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 834 KiB/s wr, 89 op/s 2023-07-21T22:21:59.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:21:59 smithi134 ceph-mon[97873]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 834 KiB/s wr, 89 op/s 2023-07-21T22:21:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:21:59 smithi093 ceph-mon[96579]: pgmap v171: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 834 KiB/s wr, 89 op/s 2023-07-21T22:22:00.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 149667 2.31 MB/sec execute 157 sec latency 568.318 ms 2023-07-21T22:22:01.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 149791 2.30 MB/sec execute 158 sec latency 1169.640 ms 2023-07-21T22:22:01.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:01 smithi194 ceph-mon[98288]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 540 KiB/s wr, 69 op/s 2023-07-21T22:22:01.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:01 smithi134 ceph-mon[97873]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 540 KiB/s wr, 69 op/s 2023-07-21T22:22:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:01 smithi093 ceph-mon[96579]: pgmap v172: 97 pgs: 97 active+clean; 68 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 540 KiB/s wr, 69 op/s 2023-07-21T22:22:02.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 149791 2.29 MB/sec execute 159 sec latency 1289.022 ms 2023-07-21T22:22:03.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 150692 2.30 MB/sec execute 160 sec latency 1797.221 ms 2023-07-21T22:22:03.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:03 smithi194 ceph-mon[98288]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-07-21T22:22:03.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:03 smithi134 ceph-mon[97873]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-07-21T22:22:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:03 smithi093 ceph-mon[96579]: pgmap v173: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2023-07-21T22:22:04.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 152831 2.37 MB/sec execute 161 sec latency 14.325 ms 2023-07-21T22:22:05.327 INFO:tasks.workunit.client.0.smithi093.stdout: 1 153273 2.36 MB/sec execute 162 sec latency 533.650 ms 2023-07-21T22:22:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:05 smithi194 ceph-mon[98288]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:05.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:05 smithi134 ceph-mon[97873]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:05 smithi093 ceph-mon[96579]: pgmap v174: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:06.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 153273 2.35 MB/sec execute 163 sec latency 1533.687 ms 2023-07-21T22:22:07.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 153273 2.33 MB/sec execute 164 sec latency 2533.736 ms 2023-07-21T22:22:07.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:07 smithi194 ceph-mon[98288]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:07.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:07 smithi134 ceph-mon[97873]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:07 smithi093 ceph-mon[96579]: pgmap v175: 97 pgs: 97 active+clean; 74 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 702 KiB/s wr, 88 op/s 2023-07-21T22:22:08.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 154159 2.35 MB/sec execute 165 sec latency 3039.648 ms 2023-07-21T22:22:09.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156346 2.42 MB/sec execute 166 sec latency 13.615 ms 2023-07-21T22:22:09.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:09 smithi194 ceph-mon[98288]: pgmap v176: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-07-21T22:22:09.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:09 smithi134 ceph-mon[97873]: pgmap v176: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-07-21T22:22:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:09 smithi093 ceph-mon[96579]: pgmap v176: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2023-07-21T22:22:10.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156759 2.41 MB/sec execute 167 sec latency 550.654 ms 2023-07-21T22:22:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 -- 172.21.15.93:0/2133427665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c80ffa90 msgr2=0x7f47c80fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/2133427665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c80ffa90 0x7f47c80fff00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009990 tx=0x7f47c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 -- 172.21.15.93:0/2133427665 shutdown_connections 2023-07-21T22:22:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/2133427665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c8100440 0x7f47c810c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/2133427665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c8105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/2133427665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c80ffa90 0x7f47c80fff00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 -- 172.21.15.93:0/2133427665 >> 172.21.15.93:0/2133427665 conn(0x7f47c80fb650 msgr2=0x7f47c80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 -- 172.21.15.93:0/2133427665 shutdown_connections 2023-07-21T22:22:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.137+0000 7f47cf3e1700 1 -- 172.21.15.93:0/2133427665 wait complete. 2023-07-21T22:22:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 Processor -- start 2023-07-21T22:22:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 -- start start 2023-07-21T22:22:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 0x7f47c819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c81959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81961c0 con 0x7f47c8100440 2023-07-21T22:22:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8196330 con 0x7f47c8105aa0 2023-07-21T22:22:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cf3e1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81964f0 con 0x7f47c80ffa90 2023-07-21T22:22:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46792/0 (socket says 172.21.15.93:46792) 2023-07-21T22:22:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 -- 172.21.15.93:0/3465282484 learned_addr learned my addr 172.21.15.93:0/3465282484 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 0x7f47c819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 -- 172.21.15.93:0/3465282484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 msgr2=0x7f47c819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c81959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 0x7f47c819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 -- 172.21.15.93:0/3465282484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 msgr2=0x7f47c81959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c81959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 -- 172.21.15.93:0/3465282484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40095f0 con 0x7f47c8100440 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 0x7f47c819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c81959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.138+0000 7f47cc97c700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f47c4006010 tx=0x7f47c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.139+0000 7f47be7fc700 1 -- 172.21.15.93:0/3465282484 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c401d070 con 0x7f47c8100440 2023-07-21T22:22:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.139+0000 7f47be7fc700 1 -- 172.21.15.93:0/3465282484 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c400fb80 con 0x7f47c8100440 2023-07-21T22:22:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.139+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c8196770 con 0x7f47c8100440 2023-07-21T22:22:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.139+0000 7f47be7fc700 1 -- 172.21.15.93:0/3465282484 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4017650 con 0x7f47c8100440 2023-07-21T22:22:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.139+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a2990 con 0x7f47c8100440 2023-07-21T22:22:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.141+0000 7f47be7fc700 1 -- 172.21.15.93:0/3465282484 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f47c4022470 con 0x7f47c8100440 2023-07-21T22:22:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.142+0000 7f47be7fc700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.142+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.142+0000 7f47be7fc700 1 -- 172.21.15.93:0/3465282484 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f47c40cde10 con 0x7f47c8100440 2023-07-21T22:22:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.142+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 0x7f47b40a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47c8100bd0 tx=0x7f47b800b5f0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 Processor -- start 2023-07-21T22:22:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 -- start start 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a432620 0x556d6a42c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a42c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a32dc50 con 0x556d6a2846b0 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a3809e0 con 0x556d6a432620 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47db566b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a379e70 con 0x556d6a432a70 2023-07-21T22:22:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47cd17d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47cd17d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46798/0 (socket says 172.21.15.93:46798) 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47cd17d700 1 -- 172.21.15.93:0/3612623911 learned_addr learned my addr 172.21.15.93:0/3612623911 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47cc97c700 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a432620 0x556d6a42c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.143+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a42c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 -- 172.21.15.93:0/3612623911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 msgr2=0x556d6a42c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a42c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 -- 172.21.15.93:0/3612623911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a432620 msgr2=0x556d6a42c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a432620 0x556d6a42c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 -- 172.21.15.93:0/3612623911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d6a1a7d80 con 0x556d6a2846b0 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47b800fcc0 tx=0x7f47b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f714b6c505877386 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47b37fe700 1 -- 172.21.15.93:0/3612623911 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8011ad0 con 0x556d6a2846b0 2023-07-21T22:22:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47b37fe700 1 -- 172.21.15.93:0/3612623911 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b8011c30 con 0x556d6a2846b0 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.144+0000 7f47b37fe700 1 -- 172.21.15.93:0/3612623911 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b801d610 con 0x556d6a2846b0 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- 172.21.15.93:0/3612623911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 msgr2=0x556d6a4320e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47b800fcc0 tx=0x7f47b8012000 comp rx=0 tx=0).stop 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- 172.21.15.93:0/3612623911 shutdown_connections 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a42c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a2846b0 0x556d6a4320e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 --2- 172.21.15.93:0/3612623911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a432620 0x556d6a42c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- 172.21.15.93:0/3612623911 >> 172.21.15.93:0/3612623911 conn(0x556d6a41dae0 msgr2=0x556d6a4230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- 172.21.15.93:0/3612623911 shutdown_connections 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- 172.21.15.93:0/3612623911 wait complete. 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 -- start start 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.145+0000 7f47db566b80 1 Processor -- start 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a438970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a436250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a319a60 con 0x556d6a432620 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a319760 con 0x556d6a2846b0 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a1fd8c0 con 0x556d6a432a70 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 10 client.0 _ll_register_callbacks cb 0x556d6a317700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:11.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46806/0 (socket says 172.21.15.93:46806) 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 -- 172.21.15.93:0/4238746349 learned_addr learned my addr 172.21.15.93:0/4238746349 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cd17d700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a438970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cd97e700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a436250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 -- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 msgr2=0x556d6a436250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a436250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 -- 172.21.15.93:0/4238746349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 msgr2=0x556d6a438970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a438970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 -- 172.21.15.93:0/4238746349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b800f9a0 con 0x556d6a432620 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cd17d700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a438970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47cc97c700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f47c40cb5e0 tx=0x7f47c40a3350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9fb622a0614d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a2ec0 con 0x556d6a432620 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f47c40a3020 con 0x556d6a432620 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d6a2d3490 con 0x556d6a432620 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c40b9320 con 0x556d6a432620 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 10 client.16572 fetch_fsmap learned FSMap version 42 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.146+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556d6a190c00 con 0x556d6a432620 2023-07-21T22:22:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d6a2d3490 con 0x556d6a432620 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f47c4022920 con 0x556d6a432620 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 20 client.16572 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a445ba0 0x556d6a447fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 4 client.16572 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:11.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d6a372e50 con 0x556d6a445ba0 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.147+0000 7f47cd17d700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a445ba0 0x556d6a447fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47b0ff9700 10 client.16572 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47cd17d700 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a445ba0 0x556d6a447fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d295fe5fa359a2 server_cookie=f9025e1db0dc603b in_seq=0 out_seq=0 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47b0ff9700 1 -- 172.21.15.93:0/4238746349 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556d6a372e50 con 0x556d6a445ba0 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47b0ff9700 10 client.16572 handle_command_reply: tid=0 2023-07-21T22:22:11.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 client.16572 shutdown 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 msgr2=0x556d6a435bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f47c40cb5e0 tx=0x7f47c40a3350 comp rx=0 tx=0).stop 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 shutdown_connections 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a445ba0 0x556d6a447fb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432a70 0x556d6a436250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a438970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47b0ff9700 0 client.16572 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 --2- 172.21.15.93:0/4238746349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a435bf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.148+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 >> 172.21.15.93:0/4238746349 conn(0x556d6a41dae0 msgr2=0x556d6a4223e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.149+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 shutdown_connections 2023-07-21T22:22:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.149+0000 7f47db566b80 1 -- 172.21.15.93:0/4238746349 wait complete. 2023-07-21T22:22:11.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.149+0000 7f47db566b80 20 client.16572 trim_cache size 0 max 16384 2023-07-21T22:22:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.163+0000 7f47db566b80 1 Processor -- start 2023-07-21T22:22:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.163+0000 7f47db566b80 1 -- start start 2023-07-21T22:22:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.163+0000 7f47db566b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a41dae0 0x556d6a43cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47db566b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a2846b0 0x556d6a431440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47db566b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47db566b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a4371d0 con 0x556d6a432620 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47db566b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a431ce0 con 0x556d6a41dae0 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47db566b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a430920 con 0x556d6a2846b0 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46816/0 (socket says 172.21.15.93:46816) 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 -- 172.21.15.93:0/3017399062 learned_addr learned my addr 172.21.15.93:0/3017399062 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cc97c700 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a2846b0 0x556d6a431440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd17d700 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a41dae0 0x556d6a43cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 -- 172.21.15.93:0/3017399062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a2846b0 msgr2=0x556d6a431440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a2846b0 0x556d6a431440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 -- 172.21.15.93:0/3017399062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a41dae0 msgr2=0x556d6a43cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a41dae0 0x556d6a43cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 -- 172.21.15.93:0/3017399062 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d6a1ce1f0 con 0x556d6a432620 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47cd97e700 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f47c0007ea0 tx=0x7f47c000f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84a202113e1ecf2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47b3fff700 1 -- 172.21.15.93:0/3017399062 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c001a680 con 0x556d6a432620 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47b3fff700 1 -- 172.21.15.93:0/3017399062 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c000fe50 con 0x556d6a432620 2023-07-21T22:22:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.164+0000 7f47b3fff700 1 -- 172.21.15.93:0/3017399062 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0011cb0 con 0x556d6a432620 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- 172.21.15.93:0/3017399062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 msgr2=0x556d6a422b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f47c0007ea0 tx=0x7f47c000f630 comp rx=0 tx=0).stop 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- 172.21.15.93:0/3017399062 shutdown_connections 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a432620 0x556d6a422b60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a41dae0 0x556d6a43cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 --2- 172.21.15.93:0/3017399062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a2846b0 0x556d6a431440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- 172.21.15.93:0/3017399062 >> 172.21.15.93:0/3017399062 conn(0x556d6a441aa0 msgr2=0x556d6a430d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- 172.21.15.93:0/3017399062 shutdown_connections 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- 172.21.15.93:0/3017399062 wait complete. 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 -- start start 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.165+0000 7f47db566b80 1 Processor -- start 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a43d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432620 0x556d6a42e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a42eba0 con 0x556d6a41dae0 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a42ed10 con 0x556d6a2846b0 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6a499a60 con 0x556d6a432620 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 10 client.0 _ll_register_callbacks cb 0x7f47c409af90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46820/0 (socket says 172.21.15.93:46820) 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 -- 172.21.15.93:0/2885822650 learned_addr learned my addr 172.21.15.93:0/2885822650 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cd17d700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a43d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cd97e700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432620 0x556d6a42e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 -- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432620 msgr2=0x556d6a42e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432620 0x556d6a42e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 -- 172.21.15.93:0/2885822650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 msgr2=0x556d6a43d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a43d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 -- 172.21.15.93:0/2885822650 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c0007b00 con 0x556d6a41dae0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47cc97c700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f47c4097440 tx=0x7f47c4009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c99b2ad4af3d1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a30c0 con 0x556d6a41dae0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f47c4095ba0 con 0x556d6a41dae0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.166+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47b8011c30 con 0x556d6a41dae0 2023-07-21T22:22:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47c40b9320 con 0x556d6a41dae0 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 10 client.16578 fetch_fsmap learned FSMap version 42 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556d6a1a6250 con 0x556d6a41dae0 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b8011c30 con 0x556d6a41dae0 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f47c40cb020 con 0x556d6a41dae0 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 20 client.16578 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a432a70 0x556d6a49cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 4 client.16578 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d6a334ff0 con 0x556d6a432a70 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.167+0000 7f47cd17d700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a432a70 0x556d6a49cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.168+0000 7f47b0ff9700 10 client.16578 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.168+0000 7f47cd17d700 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a432a70 0x556d6a49cd20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6909dbdb685c93f0 server_cookie=1bf4a87046eaa059 in_seq=0 out_seq=0 2023-07-21T22:22:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47b0ff9700 1 -- 172.21.15.93:0/2885822650 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d6a334ff0 con 0x556d6a432a70 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47b0ff9700 10 client.16578 handle_command_reply: tid=0 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 client.16578 shutdown 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 msgr2=0x556d6a42df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f47c4097440 tx=0x7f47c4009380 comp rx=0 tx=0).stop 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 shutdown_connections 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556d6a432a70 0x556d6a49cd20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d6a432620 0x556d6a42e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47b0ff9700 0 client.16578 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d6a2846b0 0x556d6a43d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 --2- 172.21.15.93:0/2885822650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556d6a41dae0 0x556d6a42df90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 >> 172.21.15.93:0/2885822650 conn(0x556d6a441aa0 msgr2=0x556d6a4315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 shutdown_connections 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 1 -- 172.21.15.93:0/2885822650 wait complete. 2023-07-21T22:22:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.197+0000 7f47db566b80 20 client.16578 trim_cache size 0 max 16384 2023-07-21T22:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:22:11.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:22:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 msgr2=0x7f47b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 0x7f47b40a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47c8100bd0 tx=0x7f47b800b5f0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 msgr2=0x7f47c819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f47c4006010 tx=0x7f47c400f690 comp rx=0 tx=0).stop 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 shutdown_connections 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c8105aa0 0x7f47c81959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47c80ffa90 0x7f47c819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f47b40a12c0 0x7f47b40a3770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 --2- 172.21.15.93:0/3465282484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47c8100440 0x7f47c819bf50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:11.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 >> 172.21.15.93:0/3465282484 conn(0x7f47c80fb650 msgr2=0x7f47c80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 shutdown_connections 2023-07-21T22:22:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:11.198+0000 7f47cf3e1700 1 -- 172.21.15.93:0/3465282484 wait complete. 2023-07-21T22:22:11.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156759 2.39 MB/sec execute 168 sec latency 1550.696 ms 2023-07-21T22:22:11.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:11 smithi194 ceph-mon[98288]: pgmap v177: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:22:11.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:11 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:11.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:11 smithi134 ceph-mon[97873]: pgmap v177: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:22:11.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:11 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:11 smithi093 ceph-mon[96579]: pgmap v177: 97 pgs: 97 active+clean; 76 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-21T22:22:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:11 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:12.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156759 2.38 MB/sec execute 169 sec latency 2550.795 ms 2023-07-21T22:22:13.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156815 2.36 MB/sec execute 170 sec latency 3055.266 ms 2023-07-21T22:22:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:13 smithi194 ceph-mon[98288]: pgmap v178: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-07-21T22:22:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:13 smithi134 ceph-mon[97873]: pgmap v178: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-07-21T22:22:13.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:13 smithi093 ceph-mon[96579]: pgmap v178: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2023-07-21T22:22:14.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156815 2.35 MB/sec execute 171 sec latency 1453.835 ms 2023-07-21T22:22:15.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156815 2.34 MB/sec execute 172 sec latency 2453.885 ms 2023-07-21T22:22:15.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:15 smithi134 ceph-mon[97873]: pgmap v179: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:15.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:15 smithi093 ceph-mon[96579]: pgmap v179: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:15.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:15 smithi194 ceph-mon[98288]: pgmap v179: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:16.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156815 2.32 MB/sec execute 173 sec latency 3453.973 ms 2023-07-21T22:22:17.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 156815 2.31 MB/sec execute 174 sec latency 4454.086 ms 2023-07-21T22:22:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:17 smithi134 ceph-mon[97873]: pgmap v180: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:17.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:17 smithi093 ceph-mon[96579]: pgmap v180: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:17.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:17 smithi194 ceph-mon[98288]: pgmap v180: 97 pgs: 97 active+clean; 75 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2023-07-21T22:22:18.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 157702 2.32 MB/sec execute 175 sec latency 4960.041 ms 2023-07-21T22:22:19.328 INFO:tasks.workunit.client.0.smithi093.stdout: 1 159913 2.39 MB/sec execute 176 sec latency 15.491 ms 2023-07-21T22:22:19.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:19 smithi134 ceph-mon[97873]: pgmap v181: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-21T22:22:19.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:19 smithi093 ceph-mon[96579]: pgmap v181: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-21T22:22:19.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:19 smithi194 ceph-mon[98288]: pgmap v181: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2023-07-21T22:22:20.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 160245 2.38 MB/sec execute 177 sec latency 630.059 ms 2023-07-21T22:22:20.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:20 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:20.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:20 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:20.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:20 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:21.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 160245 2.36 MB/sec execute 178 sec latency 1630.168 ms 2023-07-21T22:22:21.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:21 smithi134 ceph-mon[97873]: pgmap v182: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 609 KiB/s wr, 79 op/s 2023-07-21T22:22:21.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:21 smithi093 ceph-mon[96579]: pgmap v182: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 609 KiB/s wr, 79 op/s 2023-07-21T22:22:21.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:21 smithi194 ceph-mon[98288]: pgmap v182: 97 pgs: 97 active+clean; 72 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 609 KiB/s wr, 79 op/s 2023-07-21T22:22:22.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 160245 2.35 MB/sec execute 179 sec latency 2630.254 ms 2023-07-21T22:22:23.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 160245 2.34 MB/sec execute 180 sec latency 3630.314 ms 2023-07-21T22:22:23.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:23 smithi134 ceph-mon[97873]: pgmap v183: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-21T22:22:23.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:23 smithi093 ceph-mon[96579]: pgmap v183: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-21T22:22:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:23 smithi194 ceph-mon[98288]: pgmap v183: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2023-07-21T22:22:24.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 160452 2.33 MB/sec execute 181 sec latency 4483.042 ms 2023-07-21T22:22:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 -- 172.21.15.93:0/4062658855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15080ff3e0 msgr2=0x7f150810b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 --2- 172.21.15.93:0/4062658855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15080ff3e0 0x7f150810b860 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f150000b0a0 tx=0x7f150000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 -- 172.21.15.93:0/4062658855 shutdown_connections 2023-07-21T22:22:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 --2- 172.21.15.93:0/4062658855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15080ff3e0 0x7f150810b860 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 --2- 172.21.15.93:0/4062658855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1508104a30 0x7f1508104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 --2- 172.21.15.93:0/4062658855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080fea30 0x7f15080feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 -- 172.21.15.93:0/4062658855 >> 172.21.15.93:0/4062658855 conn(0x7f1508074760 msgr2=0x7f1508074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 -- 172.21.15.93:0/4062658855 shutdown_connections 2023-07-21T22:22:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.330+0000 7f15103af700 1 -- 172.21.15.93:0/4062658855 wait complete. 2023-07-21T22:22:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 Processor -- start 2023-07-21T22:22:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 -- start start 2023-07-21T22:22:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 0x7f1508198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819d0e0 con 0x7f1508104a30 2023-07-21T22:22:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819d250 con 0x7f15080ff3e0 2023-07-21T22:22:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f15103af700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819d410 con 0x7f15080fea30 2023-07-21T22:22:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e14b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46836/0 (socket says 172.21.15.93:46836) 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e14b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:33912/0 (socket says 172.21.15.93:33912) 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 -- 172.21.15.93:0/2267607872 learned_addr learned my addr 172.21.15.93:0/2267607872 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150d94a700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 0x7f1508198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 -- 172.21.15.93:0/2267607872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 msgr2=0x7f1508197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 -- 172.21.15.93:0/2267607872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 msgr2=0x7f1508198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 0x7f1508198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.331+0000 7f150e94c700 1 -- 172.21.15.93:0/2267607872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009d00 con 0x7f1508104a30 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f150e14b700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f150d94a700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 0x7f1508198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f150e94c700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1500009fd0 tx=0x7f150000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f14ff7fe700 1 -- 172.21.15.93:0/2267607872 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000e040 con 0x7f1508104a30 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150819d690 con 0x7f1508104a30 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f14ff7fe700 1 -- 172.21.15.93:0/2267607872 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15000094c0 con 0x7f1508104a30 2023-07-21T22:22:24.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f14ff7fe700 1 -- 172.21.15.93:0/2267607872 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000052a0 con 0x7f1508104a30 2023-07-21T22:22:24.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.332+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15081a4f30 con 0x7f1508104a30 2023-07-21T22:22:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.335+0000 7f14ff7fe700 1 -- 172.21.15.93:0/2267607872 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1500005400 con 0x7f1508104a30 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.335+0000 7f14ff7fe700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 0x7f14f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 Processor -- start 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e14b700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 0x7f14f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f14ff7fe700 1 -- 172.21.15.93:0/2267607872 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f15000cde00 con 0x7f1508104a30 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 -- start start 2023-07-21T22:22:24.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e14b700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 0x7f14f40a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15080ffba0 tx=0x7f1504009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b527cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528660 0x55f77b5224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b427c50 con 0x55f77b37e6b0 2023-07-21T22:22:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b47a9e0 con 0x55f77b528210 2023-07-21T22:22:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f151c534b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b473e70 con 0x55f77b528660 2023-07-21T22:22:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e14b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b527cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150d94a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e14b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b527cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46840/0 (socket says 172.21.15.93:46840) 2023-07-21T22:22:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e14b700 1 -- 172.21.15.93:0/449543176 learned_addr learned my addr 172.21.15.93:0/449543176 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150d94a700 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58004/0 (socket says 172.21.15.93:58004) 2023-07-21T22:22:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.336+0000 7f150e94c700 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528660 0x55f77b5224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 -- 172.21.15.93:0/449543176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528660 msgr2=0x55f77b5224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528660 0x55f77b5224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 -- 172.21.15.93:0/449543176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 msgr2=0x55f77b527cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b527cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 -- 172.21.15.93:0/449543176 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77b2a1d80 con 0x55f77b528210 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f150d94a700 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009c90 tx=0x7f14f80113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7257a5f5e74498a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f14fcdac700 1 -- 172.21.15.93:0/449543176 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011c00 con 0x55f77b528210 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f14fcdac700 1 -- 172.21.15.93:0/449543176 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8011d60 con 0x55f77b528210 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.337+0000 7f14fcdac700 1 -- 172.21.15.93:0/449543176 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800f6a0 con 0x55f77b528210 2023-07-21T22:22:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 -- 172.21.15.93:0/449543176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 msgr2=0x55f77b521f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009c90 tx=0x7f14f80113e0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 -- 172.21.15.93:0/449543176 shutdown_connections 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528660 0x55f77b5224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b527cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 --2- 172.21.15.93:0/449543176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b521f40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 -- 172.21.15.93:0/449543176 >> 172.21.15.93:0/449543176 conn(0x55f77b517ae0 msgr2=0x55f77b51d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 -- 172.21.15.93:0/449543176 shutdown_connections 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.338+0000 7f151c534b80 1 -- 172.21.15.93:0/449543176 wait complete. 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 -- start start 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 Processor -- start 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 0x55f77b52bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b413a60 con 0x55f77b528660 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b413760 con 0x55f77b528210 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b2f78c0 con 0x55f77b37e6b0 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f151c534b80 10 client.0 _ll_register_callbacks cb 0x55f77b411700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150e14b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58010/0 (socket says 172.21.15.93:58010) 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 -- 172.21.15.93:0/3406882319 learned_addr learned my addr 172.21.15.93:0/3406882319 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150e94c700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 0x55f77b52bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 -- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 msgr2=0x55f77b52e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 -- 172.21.15.93:0/3406882319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 msgr2=0x55f77b52bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.339+0000 7f150d94a700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 0x55f77b52bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f150d94a700 1 -- 172.21.15.93:0/3406882319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f80098f0 con 0x55f77b528210 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f150e94c700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 0x55f77b52bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f150d94a700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009600 tx=0x7f14f80052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b666481113c628 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8012450 con 0x55f77b528210 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f77b3cd490 con 0x55f77b528210 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f14f80125b0 con 0x55f77b528210 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f8012770 con 0x55f77b528210 2023-07-21T22:22:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 10 client.25997 fetch_fsmap learned FSMap version 43 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f77b28ac00 con 0x55f77b528210 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f77b3cd490 con 0x55f77b528210 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f14f8022070 con 0x55f77b528210 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 20 client.25997 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b53bbb0 0x55f77b53dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 4 client.25997 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.340+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f77b46ce50 con 0x55f77b53bbb0 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.341+0000 7f150e14b700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b53bbb0 0x55f77b53dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.341+0000 7f150e14b700 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b53bbb0 0x55f77b53dfc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b463769d0a7a045 server_cookie=b1754fa153ab6c67 in_seq=0 out_seq=0 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.341+0000 7f14f1ffb700 10 client.25997 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f14f1ffb700 1 -- 172.21.15.93:0/3406882319 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f77b46ce50 con 0x55f77b53bbb0 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f14f1ffb700 10 client.25997 handle_command_reply: tid=0 2023-07-21T22:22:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 client.25997 shutdown 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 msgr2=0x55f77b52b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009600 tx=0x7f14f80052f0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 shutdown_connections 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b53bbb0 0x55f77b53dfc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528660 0x55f77b52bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f14f1ffb700 0 client.25997 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 --2- 172.21.15.93:0/3406882319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b528210 0x55f77b52b800 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 >> 172.21.15.93:0/3406882319 conn(0x55f77b517ae0 msgr2=0x55f77b51c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 shutdown_connections 2023-07-21T22:22:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 1 -- 172.21.15.93:0/3406882319 wait complete. 2023-07-21T22:22:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.342+0000 7f151c534b80 20 client.25997 trim_cache size 0 max 16384 2023-07-21T22:22:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 Processor -- start 2023-07-21T22:22:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 -- start start 2023-07-21T22:22:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b529140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528210 0x55f77b52e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b52cc90 con 0x55f77b37e6b0 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b51dcb0 con 0x55f77b517ae0 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f151c534b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b5330e0 con 0x55f77b528210 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150d94a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b529140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150e14b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150e14b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58016/0 (socket says 172.21.15.93:58016) 2023-07-21T22:22:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150d94a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b529140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46858/0 (socket says 172.21.15.93:46858) 2023-07-21T22:22:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150e14b700 1 -- 172.21.15.93:0/3902721534 learned_addr learned my addr 172.21.15.93:0/3902721534 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.358+0000 7f150e94c700 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528210 0x55f77b52e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 -- 172.21.15.93:0/3902721534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528210 msgr2=0x55f77b52e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528210 0x55f77b52e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 -- 172.21.15.93:0/3902721534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 msgr2=0x55f77b529140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b529140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 -- 172.21.15.93:0/3902721534 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77b2c81f0 con 0x55f77b517ae0 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f150e14b700 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f77b53e8a0 tx=0x7f1504014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33663525a8be455c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f14fd5ad700 1 -- 172.21.15.93:0/3902721534 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504024040 con 0x55f77b517ae0 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f14fd5ad700 1 -- 172.21.15.93:0/3902721534 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1504029e10 con 0x55f77b517ae0 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.359+0000 7f14fd5ad700 1 -- 172.21.15.93:0/3902721534 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504004b60 con 0x55f77b517ae0 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- 172.21.15.93:0/3902721534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 msgr2=0x55f77b532ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55f77b53e8a0 tx=0x7f1504014000 comp rx=0 tx=0).stop 2023-07-21T22:22:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- 172.21.15.93:0/3902721534 shutdown_connections 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b528210 0x55f77b52e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b532ba0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 --2- 172.21.15.93:0/3902721534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b37e6b0 0x55f77b529140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- 172.21.15.93:0/3902721534 >> 172.21.15.93:0/3902721534 conn(0x55f77b5380a0 msgr2=0x55f77b528a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- 172.21.15.93:0/3902721534 shutdown_connections 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- 172.21.15.93:0/3902721534 wait complete. 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 -- start start 2023-07-21T22:22:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.360+0000 7f151c534b80 1 Processor -- start 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b529e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b52aad0 con 0x55f77b528210 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b52ac40 con 0x55f77b517ae0 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77b58fa70 con 0x55f77b37e6b0 2023-07-21T22:22:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 10 client.0 _ll_register_callbacks cb 0x7f1504004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e14b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46864/0 (socket says 172.21.15.93:46864) 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 -- 172.21.15.93:0/2246008426 learned_addr learned my addr 172.21.15.93:0/2246008426 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150d94a700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b529e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 -- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 msgr2=0x55f77b52b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 -- 172.21.15.93:0/2246008426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 msgr2=0x55f77b529e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b529e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 -- 172.21.15.93:0/2246008426 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1504017070 con 0x55f77b528210 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f150e94c700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f150009af10 tx=0x7f150009add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9695bfd3a875554a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f14f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150009af80 con 0x55f77b528210 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f150009efb0 con 0x55f77b528210 2023-07-21T22:22:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.361+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14f8011d60 con 0x55f77b528210 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15000a3340 con 0x55f77b528210 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 10 client.16596 fetch_fsmap learned FSMap version 43 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f77b2a0250 con 0x55f77b528210 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8011d60 con 0x55f77b528210 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f15000a2eb0 con 0x55f77b528210 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:24.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 20 client.16596 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b528660 0x55f77b592ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 4 client.16596 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f77b42eff0 con 0x55f77b528660 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.362+0000 7f150e14b700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b528660 0x55f77b592ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.363+0000 7f14f1ffb700 10 client.16596 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:24.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.363+0000 7f150e14b700 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b528660 0x55f77b592ff0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1446b51c0b5c552 server_cookie=c358d65aed619e86 in_seq=0 out_seq=0 2023-07-21T22:22:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f14f1ffb700 1 -- 172.21.15.93:0/2246008426 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f77b42eff0 con 0x55f77b528660 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f14f1ffb700 10 client.16596 handle_command_reply: tid=0 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 client.16596 shutdown 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 msgr2=0x55f77b52a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f150009af10 tx=0x7f150009add0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 shutdown_connections 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f77b528660 0x55f77b592ff0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f77b528210 0x55f77b52a390 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f14f1ffb700 0 client.16596 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f77b37e6b0 0x55f77b52b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 --2- 172.21.15.93:0/2246008426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f77b517ae0 0x55f77b529e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 >> 172.21.15.93:0/2246008426 conn(0x55f77b5380a0 msgr2=0x55f77b5217c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 shutdown_connections 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 1 -- 172.21.15.93:0/2246008426 wait complete. 2023-07-21T22:22:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.386+0000 7f151c534b80 20 client.16596 trim_cache size 0 max 16384 2023-07-21T22:22:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:22:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:22:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:22:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:22:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 msgr2=0x7f14f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 0x7f14f40a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15080ffba0 tx=0x7f1504009260 comp rx=0 tx=0).stop 2023-07-21T22:22:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 msgr2=0x7f150819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1500009fd0 tx=0x7f150000bce0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 shutdown_connections 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f14f40a12c0 0x7f14f40a3770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1508104a30 0x7f150819caa0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15080fea30 0x7f1508197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 --2- 172.21.15.93:0/2267607872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080ff3e0 0x7f1508198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 >> 172.21.15.93:0/2267607872 conn(0x7f1508074760 msgr2=0x7f1508067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 shutdown_connections 2023-07-21T22:22:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:24.387+0000 7f15103af700 1 -- 172.21.15.93:0/2267607872 wait complete. 2023-07-21T22:22:25.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 162608 2.38 MB/sec execute 182 sec latency 17.731 ms 2023-07-21T22:22:25.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:25 smithi134 ceph-mon[97873]: pgmap v184: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:25 smithi093 ceph-mon[96579]: pgmap v184: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:25.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:25 smithi194 ceph-mon[98288]: pgmap v184: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:26.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 164096 2.41 MB/sec execute 183 sec latency 11.043 ms 2023-07-21T22:22:27.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 166055 2.45 MB/sec execute 184 sec latency 15.227 ms 2023-07-21T22:22:27.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:27 smithi134 ceph-mon[97873]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:27.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:27 smithi093 ceph-mon[96579]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:27.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:27 smithi194 ceph-mon[98288]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 590 KiB/s wr, 66 op/s 2023-07-21T22:22:28.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 167522 2.48 MB/sec execute 185 sec latency 14.335 ms 2023-07-21T22:22:29.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 169314 2.51 MB/sec execute 186 sec latency 17.917 ms 2023-07-21T22:22:29.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:29 smithi134 ceph-mon[97873]: pgmap v186: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2023-07-21T22:22:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:29 smithi093 ceph-mon[96579]: pgmap v186: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2023-07-21T22:22:29.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:29 smithi194 ceph-mon[98288]: pgmap v186: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2023-07-21T22:22:30.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 170892 2.56 MB/sec execute 187 sec latency 13.735 ms 2023-07-21T22:22:31.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 172882 2.59 MB/sec execute 188 sec latency 15.623 ms 2023-07-21T22:22:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:31 smithi134 ceph-mon[97873]: pgmap v187: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 989 KiB/s wr, 124 op/s 2023-07-21T22:22:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:31 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:31 smithi093 ceph-mon[96579]: pgmap v187: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 989 KiB/s wr, 124 op/s 2023-07-21T22:22:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:31 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:31.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:31 smithi194 ceph-mon[98288]: pgmap v187: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 989 KiB/s wr, 124 op/s 2023-07-21T22:22:31.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:31 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:32.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 174539 2.63 MB/sec execute 189 sec latency 12.718 ms 2023-07-21T22:22:33.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 176722 2.68 MB/sec execute 190 sec latency 15.458 ms 2023-07-21T22:22:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:33 smithi134 ceph-mon[97873]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-07-21T22:22:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:33 smithi093 ceph-mon[96579]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-07-21T22:22:33.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:33 smithi194 ceph-mon[98288]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2023-07-21T22:22:34.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 178088 2.70 MB/sec execute 191 sec latency 35.977 ms 2023-07-21T22:22:35.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 179856 2.73 MB/sec execute 192 sec latency 16.216 ms 2023-07-21T22:22:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:35 smithi093 ceph-mon[96579]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:35 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:35 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:35.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:35 smithi194 ceph-mon[98288]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:35.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:35 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:35.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:35 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:35 smithi134 ceph-mon[97873]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:35 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:35 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:36.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 181370 2.77 MB/sec execute 193 sec latency 57.057 ms 2023-07-21T22:22:36.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:36 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:36.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:36 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:36.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:36 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:22:36.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:36 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:36.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:36 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:36.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:36 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:22:36.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:36 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:36.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:36 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:36.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:36 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:22:37.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 182604 2.79 MB/sec execute 194 sec latency 92.160 ms 2023-07-21T22:22:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3966610067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640ff470 msgr2=0x7fd16410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3966610067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640ff470 0x7fd16410b900 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b0a0 tx=0x7fd15c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3966610067 shutdown_connections 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3966610067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640ff470 0x7fd16410b900 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3966610067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd164104ad0 0x7fd164104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3966610067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1640feac0 0x7fd1640fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3966610067 >> 172.21.15.93:0/3966610067 conn(0x7fd164075bd0 msgr2=0x7fd164075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3966610067 shutdown_connections 2023-07-21T22:22:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3966610067 wait complete. 2023-07-21T22:22:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 Processor -- start 2023-07-21T22:22:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.500+0000 7fd16bd5e700 1 -- start start 2023-07-21T22:22:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1640ff470 0x7fd1641971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd164104ad0 0x7fd1641a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641a20f0 con 0x7fd1640feac0 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641a2260 con 0x7fd164104ad0 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641a2420 con 0x7fd1640ff470 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46874/0 (socket says 172.21.15.93:46874) 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 -- 172.21.15.93:0/3597413178 learned_addr learned my addr 172.21.15.93:0/3597413178 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd1692f9700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1640ff470 0x7fd1641971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd164104ad0 0x7fd1641a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 -- 172.21.15.93:0/3597413178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1640ff470 msgr2=0x7fd1641971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1640ff470 0x7fd1641971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 -- 172.21.15.93:0/3597413178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd164104ad0 msgr2=0x7fd1641a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd164104ad0 0x7fd1641a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 -- 172.21.15.93:0/3597413178 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160009520 con 0x7fd1640feac0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd169afa700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd16000ea50 tx=0x7fd16000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd15affd700 1 -- 172.21.15.93:0/3597413178 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16000cbd0 con 0x7fd1640feac0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c009d00 con 0x7fd1640feac0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd15affd700 1 -- 172.21.15.93:0/3597413178 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd16000cd30 con 0x7fd1640feac0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd15affd700 1 -- 172.21.15.93:0/3597413178 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160018680 con 0x7fd1640feac0 2023-07-21T22:22:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.501+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641a2ad0 con 0x7fd1640feac0 2023-07-21T22:22:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.504+0000 7fd15affd700 1 -- 172.21.15.93:0/3597413178 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd160010430 con 0x7fd1640feac0 2023-07-21T22:22:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.504+0000 7fd15affd700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 0x7fd1500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.504+0000 7fd15affd700 1 -- 172.21.15.93:0/3597413178 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd160014070 con 0x7fd1640feac0 2023-07-21T22:22:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.504+0000 7fd1692f9700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 0x7fd1500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.505+0000 7fd1692f9700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 0x7fd1500a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1640698f0 tx=0x7fd154009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.505+0000 7fd177ee3b80 1 Processor -- start 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.505+0000 7fd177ee3b80 1 -- start start 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x56457668cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564576592c50 con 0x5645764e96b0 2023-07-21T22:22:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645765e59e0 con 0x564576693660 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645765dee70 con 0x564576693210 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd1692f9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x56457668cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46882/0 (socket says 172.21.15.93:46882) 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 -- 172.21.15.93:0/1640260975 learned_addr learned my addr 172.21.15.93:0/1640260975 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 -- 172.21.15.93:0/1640260975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 msgr2=0x56457668cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x56457668cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 -- 172.21.15.93:0/1640260975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 msgr2=0x56457668d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 -- 172.21.15.93:0/1640260975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56457640cd80 con 0x5645764e96b0 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd1692f9700 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x56457668cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd169afa700 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd160096d20 tx=0x7fd160099f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cf58e0146e1cf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd14ffff700 1 -- 172.21.15.93:0/1640260975 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16009e120 con 0x5645764e96b0 2023-07-21T22:22:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd14ffff700 1 -- 172.21.15.93:0/1640260975 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1600a2700 con 0x5645764e96b0 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.506+0000 7fd14ffff700 1 -- 172.21.15.93:0/1640260975 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600a2270 con 0x5645764e96b0 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 -- 172.21.15.93:0/1640260975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 msgr2=0x564576692cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd160096d20 tx=0x7fd160099f20 comp rx=0 tx=0).stop 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 -- 172.21.15.93:0/1640260975 shutdown_connections 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692cd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/1640260975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x56457668cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 -- 172.21.15.93:0/1640260975 >> 172.21.15.93:0/1640260975 conn(0x564576682ae0 msgr2=0x5645766880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 -- 172.21.15.93:0/1640260975 shutdown_connections 2023-07-21T22:22:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.507+0000 7fd177ee3b80 1 -- 172.21.15.93:0/1640260975 wait complete. 2023-07-21T22:22:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 -- start start 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 Processor -- start 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 0x5645766995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576696850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457657ea60 con 0x564576693660 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457657e760 con 0x5645764e96b0 2023-07-21T22:22:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645764628c0 con 0x564576693210 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd177ee3b80 10 client.0 _ll_register_callbacks cb 0x56457657c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd16a2fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd169afa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 0x5645766995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd16a2fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46892/0 (socket says 172.21.15.93:46892) 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd169afa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 0x5645766995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58050/0 (socket says 172.21.15.93:58050) 2023-07-21T22:22:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd16a2fb700 1 -- 172.21.15.93:0/2338905484 learned_addr learned my addr 172.21.15.93:0/2338905484 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.508+0000 7fd1692f9700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576696850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 -- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 msgr2=0x564576696850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576696850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 -- 172.21.15.93:0/2338905484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 msgr2=0x5645766995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 0x5645766995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 -- 172.21.15.93:0/2338905484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd154017070 con 0x564576693660 2023-07-21T22:22:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd1692f9700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576696850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00bf60 tx=0x7fd15c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334add2222cd09bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00e040 con 0x564576693660 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd15c0074e0 con 0x564576693660 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564576538490 con 0x564576693660 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd15c0076a0 con 0x564576693660 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 10 client.16608 fetch_fsmap learned FSMap version 45 2023-07-21T22:22:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fd1641a2ad0 con 0x564576693660 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564576538490 con 0x564576693660 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 45) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd15c025030 con 0x564576693660 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 20 client.16608 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766a6c50 0x5645766a9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 4 client.16608 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.509+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645765d7e50 con 0x5645766a6c50 2023-07-21T22:22:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.510+0000 7fd169afa700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766a6c50 0x5645766a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.510+0000 7fd14d7fa700 10 client.16608 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.510+0000 7fd169afa700 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766a6c50 0x5645766a9060 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30744c2f849b093b server_cookie=28355287e8a61fdb in_seq=0 out_seq=0 2023-07-21T22:22:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2338905484 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5645765d7e50 con 0x5645766a6c50 2023-07-21T22:22:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd14d7fa700 10 client.16608 handle_command_reply: tid=0 2023-07-21T22:22:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 client.16608 shutdown 2023-07-21T22:22:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 msgr2=0x564576696f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00bf60 tx=0x7fd15c00bf90 comp rx=0 tx=0).stop 2023-07-21T22:22:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 shutdown_connections 2023-07-21T22:22:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766a6c50 0x5645766a9060 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693660 0x564576696f90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd14d7fa700 0 client.16608 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645764e96b0 0x5645766995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2338905484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576696850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 >> 172.21.15.93:0/2338905484 conn(0x564576682ae0 msgr2=0x5645766873e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 shutdown_connections 2023-07-21T22:22:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2338905484 wait complete. 2023-07-21T22:22:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.511+0000 7fd177ee3b80 20 client.16608 trim_cache size 0 max 16384 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 Processor -- start 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 -- start start 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576687b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645766896b0 con 0x5645764e96b0 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564576692900 con 0x564576693660 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645766844f0 con 0x564576693210 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd169afa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46896/0 (socket says 172.21.15.93:46896) 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd169afa700 1 -- 172.21.15.93:0/666448986 learned_addr learned my addr 172.21.15.93:0/666448986 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd1692f9700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576687b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.527+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 -- 172.21.15.93:0/666448986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 msgr2=0x564576687b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576687b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 -- 172.21.15.93:0/666448986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 msgr2=0x56457668cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 -- 172.21.15.93:0/666448986 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645763f5c00 con 0x5645764e96b0 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd169afa700 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd160098600 tx=0x7fd16000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d58a3a6975e8580 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd158ff9700 1 -- 172.21.15.93:0/666448986 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600986d0 con 0x5645764e96b0 2023-07-21T22:22:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd158ff9700 1 -- 172.21.15.93:0/666448986 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd16009a510 con 0x5645764e96b0 2023-07-21T22:22:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.528+0000 7fd158ff9700 1 -- 172.21.15.93:0/666448986 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600a24d0 con 0x5645764e96b0 2023-07-21T22:22:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- 172.21.15.93:0/666448986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 msgr2=0x564576692040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd160098600 tx=0x7fd16000eef0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- 172.21.15.93:0/666448986 shutdown_connections 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x56457668cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5645764e96b0 0x564576692040 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/666448986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564576693210 0x564576687b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- 172.21.15.93:0/666448986 >> 172.21.15.93:0/666448986 conn(0x564576682ae0 msgr2=0x564576691930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- 172.21.15.93:0/666448986 shutdown_connections 2023-07-21T22:22:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- 172.21.15.93:0/666448986 wait complete. 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.529+0000 7fd177ee3b80 1 -- start start 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 Processor -- start 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645764e96b0 0x56457669b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x5645766f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56457669c6e0 con 0x564576693210 2023-07-21T22:22:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645766fa050 con 0x564576693660 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645766fa190 con 0x5645764e96b0 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd177ee3b80 10 client.0 _ll_register_callbacks cb 0x7fd15c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd1692f9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd1692f9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46904/0 (socket says 172.21.15.93:46904) 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd1692f9700 1 -- 172.21.15.93:0/2797005702 learned_addr learned my addr 172.21.15.93:0/2797005702 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd169afa700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645764e96b0 0x56457669b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.530+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x5645766f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 -- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645764e96b0 msgr2=0x56457669b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645764e96b0 0x56457669b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 -- 172.21.15.93:0/2797005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 msgr2=0x5645766f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x5645766f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 -- 172.21.15.93:0/2797005702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641a2ad0 con 0x564576693210 2023-07-21T22:22:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd16a2fb700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x5645766f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd1692f9700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd1540117d0 tx=0x7fd1540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb439b2939f4ea5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154022030 con 0x564576693210 2023-07-21T22:22:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd154028630 con 0x564576693210 2023-07-21T22:22:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1600a2700 con 0x564576693210 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1540287f0 con 0x564576693210 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 10 client.16614 fetch_fsmap learned FSMap version 45 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fd1600b1040 con 0x564576693210 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd154035020 con 0x564576693210 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 45) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd15401db40 con 0x564576693210 2023-07-21T22:22:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:22:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 10 client.16614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 20 client.16614 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.531+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766fba70 0x5645766fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.532+0000 7fd177ee3b80 4 client.16614 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:22:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.532+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564576599ff0 con 0x5645766fba70 2023-07-21T22:22:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.532+0000 7fd169afa700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766fba70 0x5645766fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.532+0000 7fd14d7fa700 10 client.16614 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.532+0000 7fd169afa700 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766fba70 0x5645766fde80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bfb980bf589ce7a server_cookie=26f7b28e83f933f6 in_seq=0 out_seq=0 2023-07-21T22:22:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.626+0000 7fd14d7fa700 1 -- 172.21.15.93:0/2797005702 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564576599ff0 con 0x5645766fba70 2023-07-21T22:22:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.626+0000 7fd14d7fa700 10 client.16614 handle_command_reply: tid=0 2023-07-21T22:22:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.626+0000 7fd177ee3b80 1 client.16614 shutdown 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 msgr2=0x56457669bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd1540117d0 tx=0x7fd1540047c0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 shutdown_connections 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5645766fba70 0x5645766fde80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564576693660 0x5645766f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd14d7fa700 0 client.16614 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645764e96b0 0x56457669b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 --2- 172.21.15.93:0/2797005702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564576693210 0x56457669bdd0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 >> 172.21.15.93:0/2797005702 conn(0x564576682ae0 msgr2=0x5645766921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 shutdown_connections 2023-07-21T22:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 1 -- 172.21.15.93:0/2797005702 wait complete. 2023-07-21T22:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd177ee3b80 20 client.16614 trim_cache size 0 max 16384 2023-07-21T22:22:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:22:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:22:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:22:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 msgr2=0x7fd1500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 0x7fd1500a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1640698f0 tx=0x7fd154009260 comp rx=0 tx=0).stop 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 msgr2=0x7fd164196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.627+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd16000ea50 tx=0x7fd16000ed60 comp rx=0 tx=0).stop 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 shutdown_connections 2023-07-21T22:22:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd164104ad0 0x7fd1641a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd1500a12c0 0x7fd1500a3770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1640feac0 0x7fd164196c80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 --2- 172.21.15.93:0/3597413178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd1640ff470 0x7fd1641971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 >> 172.21.15.93:0/3597413178 conn(0x7fd164075bd0 msgr2=0x7fd164102680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 shutdown_connections 2023-07-21T22:22:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:37.628+0000 7fd16bd5e700 1 -- 172.21.15.93:0/3597413178 wait complete. 2023-07-21T22:22:37.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:37 smithi093 ceph-mon[96579]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:37.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:37 smithi194 ceph-mon[98288]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:37.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:37 smithi134 ceph-mon[97873]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2023-07-21T22:22:38.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 184672 2.84 MB/sec execute 195 sec latency 56.495 ms 2023-07-21T22:22:39.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 185950 2.86 MB/sec execute 196 sec latency 16.305 ms 2023-07-21T22:22:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:39 smithi093 ceph-mon[96579]: pgmap v191: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2023-07-21T22:22:39.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:39 smithi194 ceph-mon[98288]: pgmap v191: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2023-07-21T22:22:39.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:39 smithi134 ceph-mon[97873]: pgmap v191: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 291 op/s 2023-07-21T22:22:40.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 188141 2.91 MB/sec execute 197 sec latency 13.454 ms 2023-07-21T22:22:40.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:40 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:40.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:40 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:40.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:40 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:22:41.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 189440 2.92 MB/sec execute 198 sec latency 18.834 ms 2023-07-21T22:22:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:41 smithi093 ceph-mon[96579]: pgmap v192: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-21T22:22:41.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:41 smithi194 ceph-mon[98288]: pgmap v192: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-21T22:22:41.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:41 smithi134 ceph-mon[97873]: pgmap v192: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-21T22:22:42.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 191343 2.96 MB/sec execute 199 sec latency 30.725 ms 2023-07-21T22:22:43.329 INFO:tasks.workunit.client.0.smithi093.stdout: 1 192526 2.97 MB/sec execute 200 sec latency 76.732 ms 2023-07-21T22:22:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:43 smithi093 ceph-mon[96579]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2023-07-21T22:22:43.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:43 smithi194 ceph-mon[98288]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2023-07-21T22:22:43.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:43 smithi134 ceph-mon[97873]: pgmap v193: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 331 op/s 2023-07-21T22:22:44.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 194755 3.02 MB/sec execute 201 sec latency 15.362 ms 2023-07-21T22:22:45.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 196156 3.04 MB/sec execute 202 sec latency 11.503 ms 2023-07-21T22:22:45.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:45 smithi093 ceph-mon[96579]: pgmap v194: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:45.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:45 smithi194 ceph-mon[98288]: pgmap v194: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:45.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:45 smithi134 ceph-mon[97873]: pgmap v194: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:46.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 197549 3.05 MB/sec execute 203 sec latency 16.001 ms 2023-07-21T22:22:47.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 199148 3.09 MB/sec execute 204 sec latency 21.187 ms 2023-07-21T22:22:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:47 smithi093 ceph-mon[96579]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:47 smithi194 ceph-mon[98288]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:47 smithi134 ceph-mon[97873]: pgmap v195: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2023-07-21T22:22:48.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 201094 3.11 MB/sec execute 205 sec latency 16.260 ms 2023-07-21T22:22:49.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 202669 3.16 MB/sec execute 206 sec latency 10.739 ms 2023-07-21T22:22:49.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:49 smithi093 ceph-mon[96579]: pgmap v196: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2023-07-21T22:22:49.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:49 smithi194 ceph-mon[98288]: pgmap v196: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2023-07-21T22:22:49.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:49 smithi134 ceph-mon[97873]: pgmap v196: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 323 op/s 2023-07-21T22:22:50.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 204012 3.17 MB/sec execute 207 sec latency 17.537 ms 2023-07-21T22:22:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.741+0000 7feb50d61700 1 -- 172.21.15.93:0/718853443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 msgr2=0x7feb4c104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.741+0000 7feb50d61700 1 --2- 172.21.15.93:0/718853443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c104e60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009990 tx=0x7feb3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 -- 172.21.15.93:0/718853443 shutdown_connections 2023-07-21T22:22:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 --2- 172.21.15.93:0/718853443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0ff430 0x7feb4c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 --2- 172.21.15.93:0/718853443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c104e60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 --2- 172.21.15.93:0/718853443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0fea80 0x7feb4c0feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 -- 172.21.15.93:0/718853443 >> 172.21.15.93:0/718853443 conn(0x7feb4c0769b0 msgr2=0x7feb4c076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 -- 172.21.15.93:0/718853443 shutdown_connections 2023-07-21T22:22:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 -- 172.21.15.93:0/718853443 wait complete. 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 Processor -- start 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.742+0000 7feb50d61700 1 -- start start 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0fea80 0x7feb4c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0ff430 0x7feb4c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a2020 con 0x7feb4c104a90 2023-07-21T22:22:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a2190 con 0x7feb4c0fea80 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c1a2350 con 0x7feb4c0ff430 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46920/0 (socket says 172.21.15.93:46920) 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0fea80 0x7feb4c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 -- 172.21.15.93:0/1577826740 learned_addr learned my addr 172.21.15.93:0/1577826740 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb49d9b700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0ff430 0x7feb4c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 -- 172.21.15.93:0/1577826740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0ff430 msgr2=0x7feb4c1970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0ff430 0x7feb4c1970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 -- 172.21.15.93:0/1577826740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0fea80 msgr2=0x7feb4c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0fea80 0x7feb4c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 -- 172.21.15.93:0/1577826740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0095f0 con 0x7feb4c104a90 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb4400ef30 tx=0x7feb4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb3b7fe700 1 -- 172.21.15.93:0/1577826740 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44018070 con 0x7feb4c104a90 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c1a25d0 con 0x7feb4c104a90 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb3b7fe700 1 -- 172.21.15.93:0/1577826740 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4400f040 con 0x7feb4c104a90 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb3b7fe700 1 -- 172.21.15.93:0/1577826740 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400cd00 con 0x7feb4c104a90 2023-07-21T22:22:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.743+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c1a2a90 con 0x7feb4c104a90 2023-07-21T22:22:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.746+0000 7feb3b7fe700 1 -- 172.21.15.93:0/1577826740 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7feb44003a00 con 0x7feb4c104a90 2023-07-21T22:22:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.746+0000 7feb3b7fe700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 0x7feb340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.747+0000 7feb4a59c700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 0x7feb340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.747+0000 7feb3b7fe700 1 -- 172.21.15.93:0/1577826740 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7feb440cc850 con 0x7feb4c104a90 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.747+0000 7feb4a59c700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 0x7feb340a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b2e0 tx=0x7feb3c005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.747+0000 7feb5cee6b80 1 Processor -- start 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 -- start start 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98d4ec50 con 0x564a98e4f660 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98da19e0 con 0x564a98ca56b0 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98d9ae70 con 0x564a98e4f210 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb49d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46928/0 (socket says 172.21.15.93:46928) 2023-07-21T22:22:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 -- 172.21.15.93:0/3199204059 learned_addr learned my addr 172.21.15.93:0/3199204059 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb49d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34006/0 (socket says 172.21.15.93:34006) 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 -- 172.21.15.93:0/3199204059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 msgr2=0x564a98e48f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 -- 172.21.15.93:0/3199204059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 msgr2=0x564a98e4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 -- 172.21.15.93:0/3199204059 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a98bc8d80 con 0x564a98e4f660 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb49d9b700 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4a59c700 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb440090a0 tx=0x7feb44099590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a7a5302bfdf9cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb38ff9700 1 -- 172.21.15.93:0/3199204059 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44004ad0 con 0x564a98e4f660 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb38ff9700 1 -- 172.21.15.93:0/3199204059 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb440999d0 con 0x564a98e4f660 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.748+0000 7feb38ff9700 1 -- 172.21.15.93:0/3199204059 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44099e90 con 0x564a98e4f660 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3199204059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 msgr2=0x564a98e494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feb440090a0 tx=0x7feb44099590 comp rx=0 tx=0).stop 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3199204059 shutdown_connections 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e494a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3199204059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3199204059 >> 172.21.15.93:0/3199204059 conn(0x564a98e3eae0 msgr2=0x564a98e440e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3199204059 shutdown_connections 2023-07-21T22:22:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.749+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3199204059 wait complete. 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 -- start start 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 Processor -- start 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 0x564a98e52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 0x564a98e52e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e53380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98d3aa60 con 0x564a98e4f660 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98d3a760 con 0x564a98e4f210 2023-07-21T22:22:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98c1e8c0 con 0x564a98ca56b0 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb5cee6b80 10 client.0 _ll_register_callbacks cb 0x564a98d38700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 0x564a98e52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb4ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e53380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 0x564a98e52900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34010/0 (socket says 172.21.15.93:34010) 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb49d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 0x564a98e52e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.750+0000 7feb4a59c700 1 -- 172.21.15.93:0/2555244802 learned_addr learned my addr 172.21.15.93:0/2555244802 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 -- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 msgr2=0x564a98e52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 0x564a98e52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 -- 172.21.15.93:0/2555244802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 msgr2=0x564a98e52e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 0x564a98e52e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 -- 172.21.15.93:0/2555244802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c10c1c0 con 0x564a98e4f660 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb49d9b700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 0x564a98e52e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e53380 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7feb4409dbc0 tx=0x7feb440c9f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f109789b857838d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4409da00 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7feb440a1d90 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a98cf4490 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb440990d0 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 10 client.16626 fetch_fsmap learned FSMap version 46 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7feb400098e0 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a98cf4490 con 0x564a98e4f660 2023-07-21T22:22:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feb440957c0 con 0x564a98e4f660 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 10 client.16626 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 20 client.16626 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.751+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98e64af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.752+0000 7feb5cee6b80 4 client.16626 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.752+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a98d93e50 con 0x564a98e626e0 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.752+0000 7feb4a59c700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98e64af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.752+0000 7feb31ffb700 10 client.16626 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.752+0000 7feb4a59c700 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98e64af0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1059979e92a3bfa server_cookie=45f4f64d12d9e79d in_seq=0 out_seq=0 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb31ffb700 1 -- 172.21.15.93:0/2555244802 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a98d93e50 con 0x564a98e626e0 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb31ffb700 10 client.16626 handle_command_reply: tid=0 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 client.16626 shutdown 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 msgr2=0x564a98e53380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e53380 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7feb4409dbc0 tx=0x7feb440c9f30 comp rx=0 tx=0).stop 2023-07-21T22:22:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 shutdown_connections 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98e64af0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f660 0x564a98e53380 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb31ffb700 0 client.16626 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98ca56b0 0x564a98e52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2555244802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f210 0x564a98e52e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 >> 172.21.15.93:0/2555244802 conn(0x564a98e3eae0 msgr2=0x564a98e433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 shutdown_connections 2023-07-21T22:22:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2555244802 wait complete. 2023-07-21T22:22:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.753+0000 7feb5cee6b80 20 client.16626 trim_cache size 0 max 16384 2023-07-21T22:22:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 Processor -- start 2023-07-21T22:22:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 -- start start 2023-07-21T22:22:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e59b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f660 0x564a98e531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98e44cb0 con 0x564a98ca56b0 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98e45510 con 0x564a98e4f660 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98e3bb50 con 0x564a98e4f210 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb4a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46938/0 (socket says 172.21.15.93:46938) 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb4a59c700 1 -- 172.21.15.93:0/3416078136 learned_addr learned my addr 172.21.15.93:0/3416078136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb49d9b700 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e59b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.768+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f660 0x564a98e531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 -- 172.21.15.93:0/3416078136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 msgr2=0x564a98e59b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e59b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 -- 172.21.15.93:0/3416078136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f660 msgr2=0x564a98e531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f660 0x564a98e531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 -- 172.21.15.93:0/3416078136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c1a2a90 con 0x564a98ca56b0 2023-07-21T22:22:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb4a59c700 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feb3c000c00 tx=0x7feb3c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4747a219c2a93795 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb397fa700 1 -- 172.21.15.93:0/3416078136 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c02b040 con 0x564a98ca56b0 2023-07-21T22:22:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb397fa700 1 -- 172.21.15.93:0/3416078136 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3c02c040 con 0x564a98ca56b0 2023-07-21T22:22:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.769+0000 7feb397fa700 1 -- 172.21.15.93:0/3416078136 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c004ca0 con 0x564a98ca56b0 2023-07-21T22:22:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3416078136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 msgr2=0x564a98e4e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feb3c000c00 tx=0x7feb3c01a050 comp rx=0 tx=0).stop 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3416078136 shutdown_connections 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98e4f660 0x564a98e531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98ca56b0 0x564a98e4e1d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/3416078136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f210 0x564a98e59b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3416078136 >> 172.21.15.93:0/3416078136 conn(0x564a98e3eae0 msgr2=0x564a98e510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3416078136 shutdown_connections 2023-07-21T22:22:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- 172.21.15.93:0/3416078136 wait complete. 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 -- start start 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 Processor -- start 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.770+0000 7feb5cee6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f660 0x564a98e56440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98eb5760 con 0x564a98e4f210 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98eb58d0 con 0x564a98ca56b0 2023-07-21T22:22:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb5cee6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a98eb5a90 con 0x564a98e4f660 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb5cee6b80 10 client.0 _ll_register_callbacks cb 0x7feb3c005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46946/0 (socket says 172.21.15.93:46946) 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 -- 172.21.15.93:0/2805582774 learned_addr learned my addr 172.21.15.93:0/2805582774 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb4ad9d700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f660 0x564a98e56440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb4a59c700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 -- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f660 msgr2=0x564a98e56440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f660 0x564a98e56440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 -- 172.21.15.93:0/2805582774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 msgr2=0x564a98e55980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e55980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 -- 172.21.15.93:0/2805582774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c00be60 con 0x564a98e4f210 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb49d9b700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feb4c069560 tx=0x7feb40012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223a7b7d52194f8a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40017040 con 0x564a98e4f210 2023-07-21T22:22:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7feb40012cd0 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb440999d0 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.771+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb40012e90 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 10 client.16632 fetch_fsmap learned FSMap version 46 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x564a98bb1c00 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4002b020 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feb40003cf0 con 0x564a98e4f210 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:22:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 20 client.16632 populate_metadata read hostname 'smithi093' 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98eb93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 4 client.16632 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a98d55ff0 con 0x564a98e626e0 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb4a59c700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98eb93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb31ffb700 10 client.16632 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.772+0000 7feb4a59c700 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98eb93e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7427658bbc5bbddc server_cookie=a8dbdc04d126a132 in_seq=0 out_seq=0 2023-07-21T22:22:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.802+0000 7feb31ffb700 1 -- 172.21.15.93:0/2805582774 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a98d55ff0 con 0x564a98e626e0 2023-07-21T22:22:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.802+0000 7feb31ffb700 10 client.16632 handle_command_reply: tid=0 2023-07-21T22:22:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.802+0000 7feb5cee6b80 1 client.16632 shutdown 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 msgr2=0x564a98e55ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feb4c069560 tx=0x7feb40012450 comp rx=0 tx=0).stop 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 shutdown_connections 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a98e626e0 0x564a98eb93e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a98e4f660 0x564a98e56440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a98ca56b0 0x564a98e55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 --2- 172.21.15.93:0/2805582774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a98e4f210 0x564a98e55ec0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 >> 172.21.15.93:0/2805582774 conn(0x564a98e3eae0 msgr2=0x564a98e4e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb31ffb700 0 client.16632 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 shutdown_connections 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 1 -- 172.21.15.93:0/2805582774 wait complete. 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.803+0000 7feb5cee6b80 20 client.16632 trim_cache size 0 max 16384 2023-07-21T22:22:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:22:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:22:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:22:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 msgr2=0x7feb340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 0x7feb340a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b2e0 tx=0x7feb3c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 msgr2=0x7feb4c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feb4400ef30 tx=0x7feb4400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 shutdown_connections 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feb340a12c0 0x7feb340a3770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feb4c104a90 0x7feb4c1a1a50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb4c0fea80 0x7feb4c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.805+0000 7feb50d61700 1 --2- 172.21.15.93:0/1577826740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb4c0ff430 0x7feb4c1970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.806+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 >> 172.21.15.93:0/1577826740 conn(0x7feb4c0769b0 msgr2=0x7feb4c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.806+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 shutdown_connections 2023-07-21T22:22:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:22:50.806+0000 7feb50d61700 1 -- 172.21.15.93:0/1577826740 wait complete. 2023-07-21T22:22:51.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 205949 3.22 MB/sec execute 208 sec latency 17.181 ms 2023-07-21T22:22:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:51 smithi093 ceph-mon[96579]: pgmap v197: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-07-21T22:22:51.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:51 smithi194 ceph-mon[98288]: pgmap v197: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-07-21T22:22:51.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:51 smithi134 ceph-mon[97873]: pgmap v197: 97 pgs: 97 active+clean; 95 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2023-07-21T22:22:52.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 207154 3.23 MB/sec execute 209 sec latency 28.269 ms 2023-07-21T22:22:53.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 209281 3.27 MB/sec execute 210 sec latency 21.097 ms 2023-07-21T22:22:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:53 smithi093 ceph-mon[96579]: pgmap v198: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2023-07-21T22:22:53.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:53 smithi194 ceph-mon[98288]: pgmap v198: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2023-07-21T22:22:53.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:53 smithi134 ceph-mon[97873]: pgmap v198: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 322 op/s 2023-07-21T22:22:54.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 210466 3.27 MB/sec execute 211 sec latency 13.774 ms 2023-07-21T22:22:55.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 212786 3.33 MB/sec execute 212 sec latency 15.850 ms 2023-07-21T22:22:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:55 smithi093 ceph-mon[96579]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:55.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:55 smithi194 ceph-mon[98288]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:55.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:55 smithi134 ceph-mon[97873]: pgmap v199: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:56.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 214006 3.33 MB/sec execute 213 sec latency 15.679 ms 2023-07-21T22:22:57.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 215884 3.36 MB/sec execute 214 sec latency 16.603 ms 2023-07-21T22:22:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:57 smithi093 ceph-mon[96579]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:57.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:57 smithi194 ceph-mon[98288]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:57.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:57 smithi134 ceph-mon[97873]: pgmap v200: 97 pgs: 97 active+clean; 94 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 208 op/s 2023-07-21T22:22:58.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 217214 3.38 MB/sec execute 215 sec latency 21.103 ms 2023-07-21T22:22:59.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 219004 3.40 MB/sec execute 216 sec latency 33.613 ms 2023-07-21T22:22:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:22:59 smithi093 ceph-mon[96579]: pgmap v201: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 317 op/s 2023-07-21T22:22:59.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:22:59 smithi194 ceph-mon[98288]: pgmap v201: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 317 op/s 2023-07-21T22:22:59.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:22:59 smithi134 ceph-mon[97873]: pgmap v201: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 317 op/s 2023-07-21T22:23:00.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 220561 3.44 MB/sec execute 217 sec latency 13.909 ms 2023-07-21T22:23:01.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 221883 3.45 MB/sec execute 218 sec latency 15.650 ms 2023-07-21T22:23:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:01 smithi093 ceph-mon[96579]: pgmap v202: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-07-21T22:23:01.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:01 smithi194 ceph-mon[98288]: pgmap v202: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-07-21T22:23:01.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:01 smithi134 ceph-mon[97873]: pgmap v202: 97 pgs: 97 active+clean; 100 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 216 op/s 2023-07-21T22:23:02.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 223518 3.49 MB/sec execute 219 sec latency 24.437 ms 2023-07-21T22:23:03.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 224734 3.49 MB/sec execute 220 sec latency 33.914 ms 2023-07-21T22:23:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:03 smithi093 ceph-mon[96579]: pgmap v203: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 352 op/s 2023-07-21T22:23:03.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:03 smithi194 ceph-mon[98288]: pgmap v203: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 352 op/s 2023-07-21T22:23:03.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:03 smithi134 ceph-mon[97873]: pgmap v203: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 352 op/s 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 -- 172.21.15.93:0/2136163555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e980fe3f0 msgr2=0x7f1e980fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/2136163555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e980fe3f0 0x7f1e980fe7c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1e940099c0 tx=0x7f1e94009cd0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 -- 172.21.15.93:0/2136163555 shutdown_connections 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/2136163555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e98103430 0x7f1e9810b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/2136163555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e980fe3f0 0x7f1e980fe7c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/2136163555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fed90 0x7f1e98102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 -- 172.21.15.93:0/2136163555 >> 172.21.15.93:0/2136163555 conn(0x7f1e98075f90 msgr2=0x7f1e98076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 -- 172.21.15.93:0/2136163555 shutdown_connections 2023-07-21T22:23:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.920+0000 7f1ea06fb700 1 -- 172.21.15.93:0/2136163555 wait complete. 2023-07-21T22:23:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 Processor -- start 2023-07-21T22:23:03.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 -- start start 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fe3f0 0x7f1e9819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980fed90 0x7f1e9819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e981961f0 con 0x7f1e98103430 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98196360 con 0x7f1e980fed90 2023-07-21T22:23:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1ea06fb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98196520 con 0x7f1e980fe3f0 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46962/0 (socket says 172.21.15.93:46962) 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 -- 172.21.15.93:0/4086624344 learned_addr learned my addr 172.21.15.93:0/4086624344 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9e497700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fe3f0 0x7f1e9819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980fed90 0x7f1e9819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 -- 172.21.15.93:0/4086624344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fe3f0 msgr2=0x7f1e9819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fe3f0 0x7f1e9819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 -- 172.21.15.93:0/4086624344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980fed90 msgr2=0x7f1e9819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980fed90 0x7f1e9819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.921+0000 7f1e9ec98700 1 -- 172.21.15.93:0/4086624344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940095f0 con 0x7f1e98103430 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000ef30 tx=0x7f1e9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1e8f7fe700 1 -- 172.21.15.93:0/4086624344 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90018070 con 0x7f1e98103430 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1e8f7fe700 1 -- 172.21.15.93:0/4086624344 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e9000f040 con 0x7f1e98103430 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e981967a0 con 0x7f1e98103430 2023-07-21T22:23:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1e8f7fe700 1 -- 172.21.15.93:0/4086624344 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000cd00 con 0x7f1e98103430 2023-07-21T22:23:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.922+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e981a2bd0 con 0x7f1e98103430 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.924+0000 7f1e8f7fe700 1 -- 172.21.15.93:0/4086624344 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1e90003a00 con 0x7f1e98103430 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.925+0000 7f1e8f7fe700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 0x7f1e840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.925+0000 7f1e9e497700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 0x7f1e840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.925+0000 7f1e8f7fe700 1 -- 172.21.15.93:0/4086624344 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e900cc7b0 con 0x7f1e98103430 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.925+0000 7f1eac880b80 1 Processor -- start 2023-07-21T22:23:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9e497700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 0x7f1e840a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009990 tx=0x7f1e94005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 -- start start 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd3f8c50 con 0x55d5fd4f9210 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd44b9e0 con 0x55d5fd34f6b0 2023-07-21T22:23:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1eac880b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd444e70 con 0x55d5fd4f9660 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46968/0 (socket says 172.21.15.93:46968) 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 -- 172.21.15.93:0/4278903606 learned_addr learned my addr 172.21.15.93:0/4278903606 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9e497700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 -- 172.21.15.93:0/4278903606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 msgr2=0x55d5fd4f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 -- 172.21.15.93:0/4278903606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 msgr2=0x55d5fd4f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9dc96700 1 -- 172.21.15.93:0/4278903606 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fd272d80 con 0x55d5fd4f9210 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.926+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009cb0 tx=0x7f1e88011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd33d106b50d338 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1e8cff9700 1 -- 172.21.15.93:0/4278903606 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011d20 con 0x55d5fd4f9210 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1e8cff9700 1 -- 172.21.15.93:0/4278903606 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e88011e80 con 0x55d5fd4f9210 2023-07-21T22:23:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1e8cff9700 1 -- 172.21.15.93:0/4278903606 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88005280 con 0x55d5fd4f9210 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1eac880b80 1 -- 172.21.15.93:0/4278903606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 msgr2=0x55d5fd4f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.927+0000 7f1eac880b80 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009cb0 tx=0x7f1e88011440 comp rx=0 tx=0).stop 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- 172.21.15.93:0/4278903606 shutdown_connections 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- 172.21.15.93:0/4278903606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f2f40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- 172.21.15.93:0/4278903606 >> 172.21.15.93:0/4278903606 conn(0x55d5fd4e8ae0 msgr2=0x55d5fd4ee0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- 172.21.15.93:0/4278903606 shutdown_connections 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- 172.21.15.93:0/4278903606 wait complete. 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- start start 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 Processor -- start 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4ff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd3e4a60 con 0x55d5fd4f9210 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd3e4760 con 0x55d5fd34f6b0 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd2c88c0 con 0x55d5fd4f9660 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1eac880b80 10 client.0 _ll_register_callbacks cb 0x55d5fd3e2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.928+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46974/0 (socket says 172.21.15.93:46974) 2023-07-21T22:23:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 -- 172.21.15.93:0/1735761511 learned_addr learned my addr 172.21.15.93:0/1735761511 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9e497700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 -- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 msgr2=0x55d5fd4fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 -- 172.21.15.93:0/1735761511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 msgr2=0x55d5fd4ff530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4ff530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 -- 172.21.15.93:0/1735761511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e981a2bd0 con 0x55d5fd4f9210 2023-07-21T22:23:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9e497700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005c50 tx=0x7f1e88005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f8173cf59b4aec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004260 con 0x55d5fd4f9210 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e880043c0 con 0x55d5fd4f9210 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5fd43de50 con 0x55d5fd4f9210 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e88004580 con 0x55d5fd4f9210 2023-07-21T22:23:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1eac880b80 10 client.16644 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.929+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f1e880098e0 con 0x55d5fd4f9210 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5fd43de50 con 0x55d5fd4f9210 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1e88005210 con 0x55d5fd4f9210 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 10 client.16644 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 20 client.16644 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd50ca10 0x55d5fd50ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 4 client.16644 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5fd3ffff0 con 0x55d5fd50ca10 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1e9e497700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd50ca10 0x55d5fd50ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1e81ffb700 10 client.16644 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.930+0000 7f1e9e497700 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd50ca10 0x55d5fd50ee20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b071224ee71c70 server_cookie=53ddec934e3739c3 in_seq=0 out_seq=0 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1e81ffb700 1 -- 172.21.15.93:0/1735761511 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5fd3ffff0 con 0x55d5fd50ca10 2023-07-21T22:23:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1e81ffb700 10 client.16644 handle_command_reply: tid=0 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 client.16644 shutdown 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 msgr2=0x55d5fd4fc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1e88005c50 tx=0x7f1e88005e30 comp rx=0 tx=0).stop 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 shutdown_connections 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd50ca10 0x55d5fd50ee20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1e81ffb700 0 client.16644 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 --2- 172.21.15.93:0/1735761511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4fc820 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 >> 172.21.15.93:0/1735761511 conn(0x55d5fd4e8ae0 msgr2=0x55d5fd4ed3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 shutdown_connections 2023-07-21T22:23:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 1 -- 172.21.15.93:0/1735761511 wait complete. 2023-07-21T22:23:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.931+0000 7f1eac880b80 20 client.16644 trim_cache size 0 max 16384 2023-07-21T22:23:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.946+0000 7f1eac880b80 1 Processor -- start 2023-07-21T22:23:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.946+0000 7f1eac880b80 1 -- start start 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd503b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9210 0x55d5fd4ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd509460 con 0x55d5fd4f9660 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd4eef00 con 0x55d5fd34f6b0 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1eac880b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd4fad80 con 0x55d5fd4f9210 2023-07-21T22:23:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46984/0 (socket says 172.21.15.93:46984) 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9210 0x55d5fd4ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 -- 172.21.15.93:0/417509920 learned_addr learned my addr 172.21.15.93:0/417509920 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9e497700 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd503b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 -- 172.21.15.93:0/417509920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9210 msgr2=0x55d5fd4ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9210 0x55d5fd4ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 -- 172.21.15.93:0/417509920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 msgr2=0x55d5fd503b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd503b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 -- 172.21.15.93:0/417509920 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fd25bc00 con 0x55d5fd4f9660 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1e900c9f80 tx=0x7f1e9009f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817fbb1ccbadfc1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e8d7fa700 1 -- 172.21.15.93:0/417509920 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90084040 con 0x55d5fd4f9660 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e8d7fa700 1 -- 172.21.15.93:0/417509920 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e9000ce60 con 0x55d5fd4f9660 2023-07-21T22:23:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.947+0000 7f1e8d7fa700 1 -- 172.21.15.93:0/417509920 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90099b90 con 0x55d5fd4f9660 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- 172.21.15.93:0/417509920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 msgr2=0x55d5fd4edb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1e900c9f80 tx=0x7f1e9009f040 comp rx=0 tx=0).stop 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- 172.21.15.93:0/417509920 shutdown_connections 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9660 0x55d5fd4edb60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd503b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 --2- 172.21.15.93:0/417509920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9210 0x55d5fd4ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- 172.21.15.93:0/417509920 >> 172.21.15.93:0/417509920 conn(0x55d5fd4e8ae0 msgr2=0x55d5fd4f6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- 172.21.15.93:0/417509920 shutdown_connections 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- 172.21.15.93:0/417509920 wait complete. 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 -- start start 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.948+0000 7f1eac880b80 1 Processor -- start 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd4f4e70 con 0x55d5fd4f9210 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd4f4fe0 con 0x55d5fd34f6b0 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fd4f51a0 con 0x55d5fd4f9660 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 10 client.0 _ll_register_callbacks cb 0x7f1e940037f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46988/0 (socket says 172.21.15.93:46988) 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 -- 172.21.15.93:0/91800723 learned_addr learned my addr 172.21.15.93:0/91800723 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9e497700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 -- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 msgr2=0x55d5fd4f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 -- 172.21.15.93:0/91800723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 msgr2=0x55d5fd4f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 -- 172.21.15.93:0/91800723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e981967a0 con 0x55d5fd4f9210 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9ec98700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e9dc96700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2236e5c77b742890 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011bd0 con 0x55d5fd4f9210 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f1e88022480 con 0x55d5fd4f9210 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5fd39e490 con 0x55d5fd4f9210 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e88004b00 con 0x55d5fd4f9210 2023-07-21T22:23:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 10 client.16650 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.949+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d5fd2991f0 con 0x55d5fd4f9210 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5fd39e490 con 0x55d5fd4f9210 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1e88022b40 con 0x55d5fd4f9210 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 20 client.16650 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd5616b0 0x55d5fd563ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 4 client.16650 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:23:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5fd293f10 con 0x55d5fd5616b0 2023-07-21T22:23:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1e9e497700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd5616b0 0x55d5fd563ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1e9e497700 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd5616b0 0x55d5fd563ac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4177e80949046c server_cookie=a04f46ade817e847 in_seq=0 out_seq=0 2023-07-21T22:23:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.950+0000 7f1e81ffb700 10 client.16650 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1e81ffb700 1 -- 172.21.15.93:0/91800723 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5fd293f10 con 0x55d5fd5616b0 2023-07-21T22:23:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1e81ffb700 10 client.16650 handle_command_reply: tid=0 2023-07-21T22:23:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 client.16650 shutdown 2023-07-21T22:23:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 msgr2=0x55d5fd4f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88004490 comp rx=0 tx=0).stop 2023-07-21T22:23:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 shutdown_connections 2023-07-21T22:23:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d5fd5616b0 0x55d5fd563ac0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d5fd4f9660 0x55d5fd4f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5fd34f6b0 0x55d5fd4f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1e81ffb700 0 client.16650 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 --2- 172.21.15.93:0/91800723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d5fd4f9210 0x55d5fd4f6140 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 >> 172.21.15.93:0/91800723 conn(0x55d5fd4e8ae0 msgr2=0x55d5fd4f1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 shutdown_connections 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 1 -- 172.21.15.93:0/91800723 wait complete. 2023-07-21T22:23:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.983+0000 7f1eac880b80 20 client.16650 trim_cache size 0 max 16384 2023-07-21T22:23:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:23:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:23:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:23:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:23:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 msgr2=0x7f1e840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 0x7f1e840a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009990 tx=0x7f1e94005fb0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 msgr2=0x7f1e98195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000ef30 tx=0x7f1e9000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 shutdown_connections 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1e840a12c0 0x7f1e840a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e98103430 0x7f1e98195a20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e980fe3f0 0x7f1e9819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 --2- 172.21.15.93:0/4086624344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e980fed90 0x7f1e9819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 >> 172.21.15.93:0/4086624344 conn(0x7f1e98075f90 msgr2=0x7f1e98102070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 shutdown_connections 2023-07-21T22:23:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:03.984+0000 7f1ea06fb700 1 -- 172.21.15.93:0/4086624344 wait complete. 2023-07-21T22:23:04.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 226871 3.53 MB/sec execute 221 sec latency 31.446 ms 2023-07-21T22:23:05.330 INFO:tasks.workunit.client.0.smithi093.stdout: 1 228093 3.54 MB/sec execute 222 sec latency 28.157 ms 2023-07-21T22:23:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:05 smithi093 ceph-mon[96579]: pgmap v204: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 244 op/s 2023-07-21T22:23:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:05 smithi093 ceph-mon[96579]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:23:05.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:05 smithi194 ceph-mon[98288]: pgmap v204: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 244 op/s 2023-07-21T22:23:05.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:05 smithi194 ceph-mon[98288]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:23:05.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:05 smithi134 ceph-mon[97873]: pgmap v204: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 244 op/s 2023-07-21T22:23:05.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:05 smithi134 ceph-mon[97873]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:23:06.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 229486 3.55 MB/sec execute 223 sec latency 15.971 ms 2023-07-21T22:23:07.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 231062 3.59 MB/sec execute 224 sec latency 11.814 ms 2023-07-21T22:23:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:07 smithi093 ceph-mon[96579]: pgmap v206: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 293 op/s 2023-07-21T22:23:07.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:07 smithi194 ceph-mon[98288]: pgmap v206: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 293 op/s 2023-07-21T22:23:07.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:07 smithi134 ceph-mon[97873]: pgmap v206: 97 pgs: 97 active+clean; 105 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 293 op/s 2023-07-21T22:23:08.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 232994 3.60 MB/sec execute 225 sec latency 39.086 ms 2023-07-21T22:23:09.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 234570 3.64 MB/sec execute 226 sec latency 66.444 ms 2023-07-21T22:23:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:09 smithi093 ceph-mon[96579]: pgmap v207: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:09 smithi194 ceph-mon[98288]: pgmap v207: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:09.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:09 smithi134 ceph-mon[97873]: pgmap v207: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:10.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 236543 3.65 MB/sec execute 227 sec latency 17.457 ms 2023-07-21T22:23:11.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 238151 3.69 MB/sec execute 228 sec latency 10.272 ms 2023-07-21T22:23:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:11 smithi093 ceph-mon[96579]: pgmap v208: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:11.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:11 smithi194 ceph-mon[98288]: pgmap v208: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:11.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:11 smithi134 ceph-mon[97873]: pgmap v208: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2023-07-21T22:23:12.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 239401 3.70 MB/sec execute 229 sec latency 24.883 ms 2023-07-21T22:23:13.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 241356 3.74 MB/sec execute 230 sec latency 52.313 ms 2023-07-21T22:23:13.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:13 smithi093 ceph-mon[96579]: pgmap v209: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:13.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:13 smithi093 ceph-mon[96579]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:23:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:13 smithi194 ceph-mon[98288]: pgmap v209: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:13 smithi194 ceph-mon[98288]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:23:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:13 smithi134 ceph-mon[97873]: pgmap v209: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:13 smithi134 ceph-mon[97873]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:23:14.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 242459 3.73 MB/sec execute 231 sec latency 14.076 ms 2023-07-21T22:23:15.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 244726 3.78 MB/sec execute 232 sec latency 34.817 ms 2023-07-21T22:23:15.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:15 smithi093 ceph-mon[96579]: pgmap v211: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2023-07-21T22:23:15.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:15 smithi194 ceph-mon[98288]: pgmap v211: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2023-07-21T22:23:15.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:15 smithi134 ceph-mon[97873]: pgmap v211: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 300 op/s 2023-07-21T22:23:16.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 245974 3.78 MB/sec execute 233 sec latency 14.000 ms 2023-07-21T22:23:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- 172.21.15.93:0/2786140945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa40ffde0 msgr2=0x7f5fa41001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 --2- 172.21.15.93:0/2786140945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa40ffde0 0x7f5fa41001b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00099c0 tx=0x7f5fa0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- 172.21.15.93:0/2786140945 shutdown_connections 2023-07-21T22:23:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 --2- 172.21.15.93:0/2786140945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa4104e20 0x7f5fa410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 --2- 172.21.15.93:0/2786140945 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa40ffde0 0x7f5fa41001b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 --2- 172.21.15.93:0/2786140945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa4104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- 172.21.15.93:0/2786140945 >> 172.21.15.93:0/2786140945 conn(0x7f5fa40fb6f0 msgr2=0x7f5fa40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- 172.21.15.93:0/2786140945 shutdown_connections 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- 172.21.15.93:0/2786140945 wait complete. 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 Processor -- start 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.097+0000 7f5fab786700 1 -- start start 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 0x7f5fa419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41960f0 con 0x7f5fa4104e20 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4196260 con 0x7f5fa4100780 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa4196420 con 0x7f5fa40ffde0 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 0x7f5fa419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47002/0 (socket says 172.21.15.93:47002) 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 0x7f5fa419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34080/0 (socket says 172.21.15.93:34080) 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa419c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58160/0 (socket says 172.21.15.93:58160) 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 -- 172.21.15.93:0/2444399242 learned_addr learned my addr 172.21.15.93:0/2444399242 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 -- 172.21.15.93:0/2444399242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 msgr2=0x7f5fa419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 0x7f5fa419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 -- 172.21.15.93:0/2444399242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 msgr2=0x7f5fa419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 -- 172.21.15.93:0/2444399242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa00095f0 con 0x7f5fa4104e20 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c007ee0 tx=0x7f5f9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5f9a7fc700 1 -- 172.21.15.93:0/2444399242 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00dd20 con 0x7f5fa4104e20 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5f9a7fc700 1 -- 172.21.15.93:0/2444399242 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f9c00f040 con 0x7f5fa4104e20 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa4196700 con 0x7f5fa4104e20 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5f9a7fc700 1 -- 172.21.15.93:0/2444399242 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0048a0 con 0x7f5fa4104e20 2023-07-21T22:23:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.098+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa41a2c10 con 0x7f5fa4104e20 2023-07-21T22:23:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.102+0000 7f5f9a7fc700 1 -- 172.21.15.93:0/2444399242 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5f9c003a00 con 0x7f5fa4104e20 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.102+0000 7f5f9a7fc700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 0x7f5f900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.102+0000 7f5fa9522700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 0x7f5f900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.102+0000 7f5f9a7fc700 1 -- 172.21.15.93:0/2444399242 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5f9c0cc530 con 0x7f5fa4104e20 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 Processor -- start 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 -- start start 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 0x7f5f900a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009fd0 tx=0x7f5fa0005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 0x55657326b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657326b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657316cc50 con 0x5565730c36b0 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565731bf9e0 con 0x556573271a70 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565731b8e70 con 0x556573271620 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47006/0 (socket says 172.21.15.93:47006) 2023-07-21T22:23:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 0x55657326b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 -- 172.21.15.93:0/1388046654 learned_addr learned my addr 172.21.15.93:0/1388046654 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657326b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 -- 172.21.15.93:0/1388046654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 msgr2=0x55657326b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 0x55657326b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 -- 172.21.15.93:0/1388046654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 msgr2=0x55657326b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.103+0000 7f5fa9522700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657326b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5fa9522700 1 -- 172.21.15.93:0/1388046654 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556572fe6d80 con 0x5565730c36b0 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 0x55657326b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657326b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5fa9522700 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00099c0 tx=0x7f5fa001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8214790e16e62d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5f8f7fe700 1 -- 172.21.15.93:0/1388046654 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000b990 con 0x5565730c36b0 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5f8f7fe700 1 -- 172.21.15.93:0/1388046654 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa001e040 con 0x5565730c36b0 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5f8f7fe700 1 -- 172.21.15.93:0/1388046654 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0030680 con 0x5565730c36b0 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.104+0000 7f5fb790bb80 1 -- 172.21.15.93:0/1388046654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 msgr2=0x5565732710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00099c0 tx=0x7f5fa001f000 comp rx=0 tx=0).stop 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 -- 172.21.15.93:0/1388046654 shutdown_connections 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657326b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5565730c36b0 0x5565732710e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/1388046654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271620 0x55657326b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 -- 172.21.15.93:0/1388046654 >> 172.21.15.93:0/1388046654 conn(0x55657325cae0 msgr2=0x5565732620e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 -- 172.21.15.93:0/1388046654 shutdown_connections 2023-07-21T22:23:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 -- 172.21.15.93:0/1388046654 wait complete. 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 -- start start 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.105+0000 7f5fb790bb80 1 Processor -- start 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 0x5565732779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271a70 0x556573275330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556573158a60 con 0x556573271620 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556573158760 con 0x5565730c36b0 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657303c8c0 con 0x556573271a70 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 10 client.0 _ll_register_callbacks cb 0x556573156700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47014/0 (socket says 172.21.15.93:47014) 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3664030515 learned_addr learned my addr 172.21.15.93:0/3664030515 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271a70 0x556573275330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 0x5565732779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271a70 msgr2=0x556573275330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271a70 0x556573275330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3664030515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 msgr2=0x5565732779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 0x5565732779a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3664030515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f940098e0 con 0x556573271620 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 0x5565732779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5f94012960 tx=0x7f5f94012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332cbce4afc5a32 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5f8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94017070 con 0x556573271620 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5f9400f7d0 con 0x556573271620 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.106+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556573112490 con 0x556573271620 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f9400f990 con 0x556573271620 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 10 client.16662 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f5fa41a2c10 con 0x556573271620 2023-07-21T22:23:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f94028080 con 0x556573271620 2023-07-21T22:23:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5f94022030 con 0x556573271620 2023-07-21T22:23:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 10 client.16662 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 20 client.16662 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556573285130 0x556573287540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 4 client.16662 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565731b1e50 con 0x556573285130 2023-07-21T22:23:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.107+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556573285130 0x556573287540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5f8cff9700 10 client.16662 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556573285130 0x556573287540 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342e4046c043a259 server_cookie=b8b62d471dd777d4 in_seq=0 out_seq=0 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3664030515 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565731b1e50 con 0x556573285130 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5f8cff9700 10 client.16662 handle_command_reply: tid=0 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5fb790bb80 1 client.16662 shutdown 2023-07-21T22:23:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 msgr2=0x556573274c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.108+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5f94012960 tx=0x7f5f94012c70 comp rx=0 tx=0).stop 2023-07-21T22:23:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 shutdown_connections 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556573285130 0x556573287540 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556573271a70 0x556573275330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565730c36b0 0x5565732779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5f8cff9700 0 client.16662 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3664030515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573274c90 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 >> 172.21.15.93:0/3664030515 conn(0x55657325cae0 msgr2=0x5565732613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 shutdown_connections 2023-07-21T22:23:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3664030515 wait complete. 2023-07-21T22:23:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.109+0000 7f5fb790bb80 20 client.16662 trim_cache size 0 max 16384 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 Processor -- start 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 -- start start 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x556573270550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x556573261b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657326c330 con 0x556573271620 2023-07-21T22:23:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657326c0e0 con 0x556573271a70 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556573273250 con 0x5565730c36b0 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47020/0 (socket says 172.21.15.93:47020) 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3237009451 learned_addr learned my addr 172.21.15.93:0/3237009451 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x556573270550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x556573261b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3237009451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 msgr2=0x556573270550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x556573270550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3237009451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 msgr2=0x556573261b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x556573261b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.126+0000 7f5fa8d21700 1 -- 172.21.15.93:0/3237009451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa410d230 con 0x556573271620 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5f94010cc0 tx=0x7f5f94010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56c32b0cf3a0173 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5f8ffff700 1 -- 172.21.15.93:0/3237009451 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94017040 con 0x556573271620 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5f8ffff700 1 -- 172.21.15.93:0/3237009451 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556573112490 con 0x556573271620 2023-07-21T22:23:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5f8ffff700 1 -- 172.21.15.93:0/3237009451 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94005040 con 0x556573271620 2023-07-21T22:23:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3237009451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 msgr2=0x556573277540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.127+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5f94010cc0 tx=0x7f5f94010e60 comp rx=0 tx=0).stop 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3237009451 shutdown_connections 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x556573261b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x556573270550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3237009451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x556573277540 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3237009451 >> 172.21.15.93:0/3237009451 conn(0x55657325cae0 msgr2=0x55657326fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3237009451 shutdown_connections 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3237009451 wait complete. 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 -- start start 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.128+0000 7f5fb790bb80 1 Processor -- start 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x55657327a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657327a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657327c5b0 con 0x556573271620 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657327aa80 con 0x556573271a70 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55657327ac40 con 0x5565730c36b0 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fb790bb80 10 client.0 _ll_register_callbacks cb 0x7f5f94012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34104/0 (socket says 172.21.15.93:34104) 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 -- 172.21.15.93:0/3945234112 learned_addr learned my addr 172.21.15.93:0/3945234112 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657327a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x55657327a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 -- 172.21.15.93:0/3945234112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 msgr2=0x55657327a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657327a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 -- 172.21.15.93:0/3945234112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 msgr2=0x55657327a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x55657327a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 -- 172.21.15.93:0/3945234112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa41a2c10 con 0x5565730c36b0 2023-07-21T22:23:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9d23700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657327a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5fa9522700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0003ab0 tx=0x7f5fa000b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53fc71a8752370f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5f8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.129+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0036040 con 0x5565730c36b0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5fa003b3b0 con 0x5565730c36b0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa001e040 con 0x5565730c36b0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa0004800 con 0x5565730c36b0 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 10 client.26017 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x556572fcfc00 con 0x5565730c36b0 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa0048020 con 0x5565730c36b0 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5fa000b9a0 con 0x5565730c36b0 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 10 client.26017 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 20 client.26017 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565732d9f00 0x5565732dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 4 client.26017 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:23:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556573173ff0 con 0x5565732d9f00 2023-07-21T22:23:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.130+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565732d9f00 0x5565732dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.131+0000 7f5f8cff9700 10 client.26017 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.131+0000 7f5fa8d21700 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565732d9f00 0x5565732dc310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ad21a5a845bcf3 server_cookie=75654ed846a354bf in_seq=0 out_seq=0 2023-07-21T22:23:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5f8cff9700 1 -- 172.21.15.93:0/3945234112 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556573173ff0 con 0x5565732d9f00 2023-07-21T22:23:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5f8cff9700 10 client.26017 handle_command_reply: tid=0 2023-07-21T22:23:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 client.26017 shutdown 2023-07-21T22:23:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 msgr2=0x55657327bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0003ab0 tx=0x7f5fa000b560 comp rx=0 tx=0).stop 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 shutdown_connections 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565732d9f00 0x5565732dc310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556573271a70 0x55657327a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565730c36b0 0x55657327bce0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 --2- 172.21.15.93:0/3945234112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556573271620 0x55657327a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 >> 172.21.15.93:0/3945234112 conn(0x55657325cae0 msgr2=0x5565732667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5f8cff9700 0 client.26017 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 shutdown_connections 2023-07-21T22:23:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 1 -- 172.21.15.93:0/3945234112 wait complete. 2023-07-21T22:23:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.171+0000 7f5fb790bb80 20 client.26017 trim_cache size 0 max 16384 2023-07-21T22:23:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:23:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:23:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:23:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:23:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 msgr2=0x7f5f900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 0x7f5f900a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009fd0 tx=0x7f5fa0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 msgr2=0x7f5fa4195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c007ee0 tx=0x7f5f9c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 shutdown_connections 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5f900a12c0 0x7f5f900a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5fa4104e20 0x7f5fa4195a80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fa40ffde0 0x7f5fa419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 --2- 172.21.15.93:0/2444399242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa4100780 0x7f5fa419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 >> 172.21.15.93:0/2444399242 conn(0x7f5fa40fb6f0 msgr2=0x7f5fa41037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 shutdown_connections 2023-07-21T22:23:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:17.172+0000 7f5fab786700 1 -- 172.21.15.93:0/2444399242 wait complete. 2023-07-21T22:23:17.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 247954 3.82 MB/sec execute 234 sec latency 39.242 ms 2023-07-21T22:23:17.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:17 smithi093 ceph-mon[96579]: pgmap v212: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:17.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:17 smithi194 ceph-mon[98288]: pgmap v212: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:17 smithi134 ceph-mon[97873]: pgmap v212: 97 pgs: 97 active+clean; 112 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2023-07-21T22:23:18.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 249322 3.83 MB/sec execute 235 sec latency 32.652 ms 2023-07-21T22:23:19.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 251350 3.86 MB/sec execute 236 sec latency 22.289 ms 2023-07-21T22:23:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:19 smithi093 ceph-mon[96579]: pgmap v213: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:19.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:19 smithi194 ceph-mon[98288]: pgmap v213: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:19.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:19 smithi134 ceph-mon[97873]: pgmap v213: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:20.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 252859 3.88 MB/sec execute 237 sec latency 14.164 ms 2023-07-21T22:23:21.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 254807 3.91 MB/sec execute 238 sec latency 79.725 ms 2023-07-21T22:23:21.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:21 smithi093 ceph-mon[96579]: pgmap v214: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:21.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:21 smithi194 ceph-mon[98288]: pgmap v214: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:21.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:21 smithi134 ceph-mon[97873]: pgmap v214: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2023-07-21T22:23:22.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 256323 3.92 MB/sec execute 239 sec latency 13.534 ms 2023-07-21T22:23:23.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 257602 3.93 MB/sec execute 240 sec latency 33.111 ms 2023-07-21T22:23:23.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:23 smithi093 ceph-mon[96579]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2023-07-21T22:23:23.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:23 smithi194 ceph-mon[98288]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2023-07-21T22:23:23.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:23 smithi134 ceph-mon[97873]: pgmap v215: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2023-07-21T22:23:24.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 259389 3.97 MB/sec execute 241 sec latency 15.108 ms 2023-07-21T22:23:25.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 261337 3.98 MB/sec execute 242 sec latency 16.095 ms 2023-07-21T22:23:25.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:25 smithi093 ceph-mon[96579]: pgmap v216: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:25 smithi194 ceph-mon[98288]: pgmap v216: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:25.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:25 smithi134 ceph-mon[97873]: pgmap v216: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:26.331 INFO:tasks.workunit.client.0.smithi093.stdout: 1 262969 4.01 MB/sec execute 243 sec latency 14.294 ms 2023-07-21T22:23:27.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 264942 4.03 MB/sec execute 244 sec latency 15.693 ms 2023-07-21T22:23:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:27 smithi093 ceph-mon[96579]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:27 smithi194 ceph-mon[98288]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:27.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:27 smithi134 ceph-mon[97873]: pgmap v217: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 245 op/s 2023-07-21T22:23:28.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 266534 4.06 MB/sec execute 245 sec latency 18.123 ms 2023-07-21T22:23:29.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 268502 4.07 MB/sec execute 246 sec latency 16.651 ms 2023-07-21T22:23:29.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:29 smithi093 ceph-mon[96579]: pgmap v218: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2023-07-21T22:23:29.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:29 smithi194 ceph-mon[98288]: pgmap v218: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2023-07-21T22:23:29.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:29 smithi134 ceph-mon[97873]: pgmap v218: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 363 op/s 2023-07-21T22:23:30.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 270112 4.10 MB/sec execute 247 sec latency 11.349 ms 2023-07-21T22:23:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 -- 172.21.15.93:0/2863783631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f40fe3f0 msgr2=0x7f80f40fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2863783631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fe7c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f80e40099c0 tx=0x7f80e4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 -- 172.21.15.93:0/2863783631 shutdown_connections 2023-07-21T22:23:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2863783631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f4103430 0x7f80f410b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2863783631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fe7c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2863783631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fed90 0x7f80f4102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 -- 172.21.15.93:0/2863783631 >> 172.21.15.93:0/2863783631 conn(0x7f80f4075f90 msgr2=0x7f80f4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 -- 172.21.15.93:0/2863783631 shutdown_connections 2023-07-21T22:23:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.287+0000 7f80f9d97700 1 -- 172.21.15.93:0/2863783631 wait complete. 2023-07-21T22:23:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 Processor -- start 2023-07-21T22:23:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 -- start start 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f40fed90 0x7f80f4100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a4350 con 0x7f80f4103430 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a4490 con 0x7f80f40fe3f0 2023-07-21T22:23:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f9d97700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41a4600 con 0x7f80f40fed90 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f40fed90 0x7f80f4100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47042/0 (socket says 172.21.15.93:47042) 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58200/0 (socket says 172.21.15.93:58200) 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.288+0000 7f80f3fff700 1 -- 172.21.15.93:0/2477712025 learned_addr learned my addr 172.21.15.93:0/2477712025 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 -- 172.21.15.93:0/2477712025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f40fed90 msgr2=0x7f80f4100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f40fed90 0x7f80f4100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 -- 172.21.15.93:0/2477712025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 msgr2=0x7f80f40fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 -- 172.21.15.93:0/2477712025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40095f0 con 0x7f80f4103430 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f3fff700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007ee0 tx=0x7f80ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f0ff9700 1 -- 172.21.15.93:0/2477712025 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec017070 con 0x7f80f4103430 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f0ff9700 1 -- 172.21.15.93:0/2477712025 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ec00f040 con 0x7f80f4103430 2023-07-21T22:23:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f41a48e0 con 0x7f80f4103430 2023-07-21T22:23:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f0ff9700 1 -- 172.21.15.93:0/2477712025 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec005580 con 0x7f80f4103430 2023-07-21T22:23:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.289+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a4df0 con 0x7f80f4103430 2023-07-21T22:23:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.292+0000 7f80f0ff9700 1 -- 172.21.15.93:0/2477712025 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f80ec00db20 con 0x7f80f4103430 2023-07-21T22:23:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.292+0000 7f80f0ff9700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 0x7f80dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.293+0000 7f80f0ff9700 1 -- 172.21.15.93:0/2477712025 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f80ec0cc410 con 0x7f80f4103430 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.293+0000 7f80f37fe700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 0x7f80dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.293+0000 7f80f37fe700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 0x7f80dc0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009990 tx=0x7f80e4005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.293+0000 7f8105f1cb80 1 Processor -- start 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.293+0000 7f8105f1cb80 1 -- start start 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc4660 0x555b43dbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43cc3c50 con 0x555b43dc4210 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43d169e0 con 0x555b43c1a6b0 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43d0fe70 con 0x555b43dc4660 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dbdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58208/0 (socket says 172.21.15.93:58208) 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc4660 0x555b43dbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dbdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47048/0 (socket says 172.21.15.93:47048) 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 -- 172.21.15.93:0/3725913877 learned_addr learned my addr 172.21.15.93:0/3725913877 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 -- 172.21.15.93:0/3725913877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc4660 msgr2=0x555b43dbe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc4660 0x555b43dbe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 -- 172.21.15.93:0/3725913877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 msgr2=0x555b43dbdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dbdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 -- 172.21.15.93:0/3725913877 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b43b3dd80 con 0x555b43c1a6b0 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80e40095a0 tx=0x7f80e401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcfbc377f5d30d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.294+0000 7f80d9ffb700 1 -- 172.21.15.93:0/3725913877 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003880 con 0x555b43c1a6b0 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f80d9ffb700 1 -- 172.21.15.93:0/3725913877 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e401e040 con 0x555b43c1a6b0 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f80d9ffb700 1 -- 172.21.15.93:0/3725913877 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40306e0 con 0x555b43c1a6b0 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3725913877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 msgr2=0x555b43dc3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80e40095a0 tx=0x7f80e401f000 comp rx=0 tx=0).stop 2023-07-21T22:23:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3725913877 shutdown_connections 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc4660 0x555b43dbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dc3cd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3725913877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3725913877 >> 172.21.15.93:0/3725913877 conn(0x555b43db3ae0 msgr2=0x555b43db90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3725913877 shutdown_connections 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.295+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3725913877 wait complete. 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 -- start start 2023-07-21T22:23:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 Processor -- start 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 0x555b43dcd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc4660 0x555b43dc7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43cafa60 con 0x555b43dc4210 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43caf760 con 0x555b43dc4660 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43b938c0 con 0x555b43c1a6b0 2023-07-21T22:23:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f8105f1cb80 10 client.0 _ll_register_callbacks cb 0x555b43cad700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47054/0 (socket says 172.21.15.93:47054) 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f80f2ffd700 1 -- 172.21.15.93:0/3031914589 learned_addr learned my addr 172.21.15.93:0/3031914589 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 0x555b43dcd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.296+0000 7f80f3fff700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc4660 0x555b43dc7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 -- 172.21.15.93:0/3031914589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc4660 msgr2=0x555b43dc7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc4660 0x555b43dc7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 -- 172.21.15.93:0/3031914589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 msgr2=0x555b43dcda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 -- 172.21.15.93:0/3031914589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f4067070 con 0x555b43c1a6b0 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 0x555b43dcd520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80e40058e0 tx=0x7f80e4030850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d9e103dd227748 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003880 con 0x555b43c1a6b0 2023-07-21T22:23:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b43c69490 con 0x555b43c1a6b0 2023-07-21T22:23:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f80e40309a0 con 0x555b43c1a6b0 2023-07-21T22:23:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e4030b60 con 0x555b43c1a6b0 2023-07-21T22:23:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f8105f1cb80 10 client.26029 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x555b43b26c00 con 0x555b43c1a6b0 2023-07-21T22:23:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.297+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b43c69490 con 0x555b43c1a6b0 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f80e400f6e0 con 0x555b43c1a6b0 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 10 client.26029 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 10 client.26029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 20 client.26029 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dd7530 0x555b43dd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 4 client.26029 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b43d08e50 con 0x555b43dd7530 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dd7530 0x555b43dd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f80d2ffd700 10 client.26029 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.298+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dd7530 0x555b43dd9910 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35294c107fdd2819 server_cookie=ab1c6c20818e9f9a in_seq=0 out_seq=0 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f80d2ffd700 1 -- 172.21.15.93:0/3031914589 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b43d08e50 con 0x555b43dd7530 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f80d2ffd700 10 client.26029 handle_command_reply: tid=0 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 client.26029 shutdown 2023-07-21T22:23:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 msgr2=0x555b43dcd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 0x555b43dcd520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80e40058e0 tx=0x7f80e4030850 comp rx=0 tx=0).stop 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 shutdown_connections 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dd7530 0x555b43dd9910 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc4660 0x555b43dc7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f80d2ffd700 0 client.26029 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43c1a6b0 0x555b43dcd520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3031914589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43dc4210 0x555b43dcda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 >> 172.21.15.93:0/3031914589 conn(0x555b43db3ae0 msgr2=0x555b43db83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 shutdown_connections 2023-07-21T22:23:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3031914589 wait complete. 2023-07-21T22:23:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.299+0000 7f8105f1cb80 20 client.26029 trim_cache size 0 max 16384 2023-07-21T22:23:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.314+0000 7f8105f1cb80 1 Processor -- start 2023-07-21T22:23:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.314+0000 7f8105f1cb80 1 -- start start 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43db3ae0 0x555b43dc7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43db0b70 con 0x555b43c1a6b0 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43dbf190 con 0x555b43dc2100 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43db5570 con 0x555b43db3ae0 2023-07-21T22:23:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58222/0 (socket says 172.21.15.93:58222) 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47064/0 (socket says 172.21.15.93:47064) 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 -- 172.21.15.93:0/3788505347 learned_addr learned my addr 172.21.15.93:0/3788505347 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43db3ae0 0x555b43dc7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 -- 172.21.15.93:0/3788505347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43db3ae0 msgr2=0x555b43dc7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43db3ae0 0x555b43dc7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 -- 172.21.15.93:0/3788505347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 msgr2=0x555b43dc7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 -- 172.21.15.93:0/3788505347 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b43dde540 con 0x555b43dc2100 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f3fff700 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.315+0000 7f80f37fe700 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d10 tx=0x7f80e40048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbc9bdaa849a09a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f80da7fc700 1 -- 172.21.15.93:0/3788505347 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4004000 con 0x555b43dc2100 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f80da7fc700 1 -- 172.21.15.93:0/3788505347 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e400fd10 con 0x555b43dc2100 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f80da7fc700 1 -- 172.21.15.93:0/3788505347 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400b930 con 0x555b43dc2100 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3788505347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 msgr2=0x555b43dbd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d10 tx=0x7f80e40048f0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3788505347 shutdown_connections 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43c1a6b0 0x555b43dc7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43dc2100 0x555b43dbd7c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/3788505347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43db3ae0 0x555b43dc7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3788505347 >> 172.21.15.93:0/3788505347 conn(0x555b43dc1d30 msgr2=0x555b43dc2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3788505347 shutdown_connections 2023-07-21T22:23:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.316+0000 7f8105f1cb80 1 -- 172.21.15.93:0/3788505347 wait complete. 2023-07-21T22:23:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 -- start start 2023-07-21T22:23:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 Processor -- start 2023-07-21T22:23:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dcb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 0x555b43e2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43dcc0d0 con 0x555b43db3ae0 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43e2a900 con 0x555b43c1a6b0 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b43e2aa70 con 0x555b43dc2100 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f8105f1cb80 10 client.0 _ll_register_callbacks cb 0x7f80e8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f80f37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dcb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47068/0 (socket says 172.21.15.93:47068) 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f80f2ffd700 1 -- 172.21.15.93:0/94746604 learned_addr learned my addr 172.21.15.93:0/94746604 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.317+0000 7f80f3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 0x555b43e2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 -- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 msgr2=0x555b43e2a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 0x555b43e2a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 -- 172.21.15.93:0/94746604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 msgr2=0x555b43dcb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 -- 172.21.15.93:0/94746604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f41a4df0 con 0x555b43c1a6b0 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f3fff700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 0x555b43e2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80f37fe700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dcb480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f80e40058e0 tx=0x7f80e4009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f468d92c500e11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400b8d0 con 0x555b43c1a6b0 2023-07-21T22:23:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f80e4030a50 con 0x555b43c1a6b0 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e401e040 con 0x555b43c1a6b0 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e4030830 con 0x555b43c1a6b0 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f8105f1cb80 10 client.26093 fetch_fsmap learned FSMap version 46 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x555b43b26c00 con 0x555b43c1a6b0 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.318+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e404e080 con 0x555b43c1a6b0 2023-07-21T22:23:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f80e4049070 con 0x555b43c1a6b0 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 10 client.26093 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 20 client.26093 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dc4210 0x555b43e2dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 4 client.26093 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b43ccaff0 con 0x555b43dc4210 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dc4210 0x555b43e2dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f80d2ffd700 10 client.26093 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.319+0000 7f80f2ffd700 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dc4210 0x555b43e2dd60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387f5e368207dc81 server_cookie=23228bbed57f87e7 in_seq=0 out_seq=0 2023-07-21T22:23:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f80d2ffd700 1 -- 172.21.15.93:0/94746604 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b43ccaff0 con 0x555b43dc4210 2023-07-21T22:23:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f80d2ffd700 10 client.26093 handle_command_reply: tid=0 2023-07-21T22:23:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 client.26093 shutdown 2023-07-21T22:23:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 msgr2=0x555b43dcb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dcb480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f80e40058e0 tx=0x7f80e4009fd0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 shutdown_connections 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555b43dc4210 0x555b43e2dd60 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555b43dc2100 0x555b43e2a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f80d2ffd700 0 client.26093 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b43c1a6b0 0x555b43dcb480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 --2- 172.21.15.93:0/94746604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555b43db3ae0 0x555b43dcb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 >> 172.21.15.93:0/94746604 conn(0x555b43dc1d30 msgr2=0x555b43dbd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 shutdown_connections 2023-07-21T22:23:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.352+0000 7f8105f1cb80 1 -- 172.21.15.93:0/94746604 wait complete. 2023-07-21T22:23:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f8105f1cb80 20 client.26093 trim_cache size 0 max 16384 2023-07-21T22:23:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:23:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:23:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:23:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 msgr2=0x7f80dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 0x7f80dc0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009990 tx=0x7f80e4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 msgr2=0x7f80f41a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f80ec007ee0 tx=0x7f80ec00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 shutdown_connections 2023-07-21T22:23:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f80dc0a12c0 0x7f80dc0a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80f4103430 0x7f80f41a3e10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f40fe3f0 0x7f80f40fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 --2- 172.21.15.93:0/2477712025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80f40fed90 0x7f80f4100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 >> 172.21.15.93:0/2477712025 conn(0x7f80f4075f90 msgr2=0x7f80f410b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 shutdown_connections 2023-07-21T22:23:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:30.353+0000 7f80f9d97700 1 -- 172.21.15.93:0/2477712025 wait complete. 2023-07-21T22:23:31.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 272032 4.12 MB/sec execute 248 sec latency 26.665 ms 2023-07-21T22:23:31.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:31 smithi093 ceph-mon[96579]: pgmap v219: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-07-21T22:23:31.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:31 smithi194 ceph-mon[98288]: pgmap v219: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-07-21T22:23:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:31 smithi134 ceph-mon[97873]: pgmap v219: 97 pgs: 97 active+clean; 123 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 250 op/s 2023-07-21T22:23:32.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 273626 4.15 MB/sec execute 249 sec latency 10.361 ms 2023-07-21T22:23:33.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 274928 4.15 MB/sec execute 250 sec latency 46.201 ms 2023-07-21T22:23:33.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:33 smithi093 ceph-mon[96579]: pgmap v220: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 384 op/s 2023-07-21T22:23:33.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:33 smithi194 ceph-mon[98288]: pgmap v220: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 384 op/s 2023-07-21T22:23:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:33 smithi134 ceph-mon[97873]: pgmap v220: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 384 op/s 2023-07-21T22:23:34.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 277012 4.19 MB/sec execute 251 sec latency 18.022 ms 2023-07-21T22:23:35.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 278302 4.19 MB/sec execute 252 sec latency 17.454 ms 2023-07-21T22:23:35.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:35 smithi093 ceph-mon[96579]: pgmap v221: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:35 smithi194 ceph-mon[98288]: pgmap v221: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:35 smithi134 ceph-mon[97873]: pgmap v221: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:36.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 280236 4.23 MB/sec execute 253 sec latency 13.964 ms 2023-07-21T22:23:36.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:36 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:36.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:36 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:36.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:36 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:37.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 281302 4.22 MB/sec execute 254 sec latency 21.679 ms 2023-07-21T22:23:37.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:37 smithi093 ceph-mon[96579]: pgmap v222: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:37.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:37 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:37.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:37 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:37.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:37 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:23:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:37 smithi194 ceph-mon[98288]: pgmap v222: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:37 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:37 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:37.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:37 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:23:37.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:37 smithi134 ceph-mon[97873]: pgmap v222: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 251 op/s 2023-07-21T22:23:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:37 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:37 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:37 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:23:38.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 282690 4.23 MB/sec execute 255 sec latency 16.885 ms 2023-07-21T22:23:39.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 284222 4.26 MB/sec execute 256 sec latency 52.395 ms 2023-07-21T22:23:39.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:39 smithi194 ceph-mon[98288]: pgmap v223: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 353 op/s 2023-07-21T22:23:39.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:39 smithi134 ceph-mon[97873]: pgmap v223: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 353 op/s 2023-07-21T22:23:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:39 smithi093 ceph-mon[96579]: pgmap v223: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 353 op/s 2023-07-21T22:23:40.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 286175 4.27 MB/sec execute 257 sec latency 17.179 ms 2023-07-21T22:23:41.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 287779 4.30 MB/sec execute 258 sec latency 10.873 ms 2023-07-21T22:23:41.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:41 smithi194 ceph-mon[98288]: pgmap v224: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:23:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:41 smithi134 ceph-mon[97873]: pgmap v224: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:23:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:41 smithi093 ceph-mon[96579]: pgmap v224: 97 pgs: 97 active+clean; 125 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:23:42.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 289108 4.30 MB/sec execute 259 sec latency 15.371 ms 2023-07-21T22:23:43.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 291103 4.34 MB/sec execute 260 sec latency 52.735 ms 2023-07-21T22:23:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 -- 172.21.15.93:0/3359359431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 msgr2=0x7f008410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 --2- 172.21.15.93:0/3359359431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f008410b860 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f007c00b0a0 tx=0x7f007c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 -- 172.21.15.93:0/3359359431 shutdown_connections 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 --2- 172.21.15.93:0/3359359431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f008410b860 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 --2- 172.21.15.93:0/3359359431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 0x7f0084104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 --2- 172.21.15.93:0/3359359431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f00840feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 -- 172.21.15.93:0/3359359431 >> 172.21.15.93:0/3359359431 conn(0x7f0084074760 msgr2=0x7f0084074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 -- 172.21.15.93:0/3359359431 shutdown_connections 2023-07-21T22:23:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.474+0000 7f008a01a700 1 -- 172.21.15.93:0/3359359431 wait complete. 2023-07-21T22:23:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 Processor -- start 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 -- start start 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 0x7f00841a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a20f0 con 0x7f00840ff3e0 2023-07-21T22:23:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a2260 con 0x7f0084104a30 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.475+0000 7f008a01a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841a2420 con 0x7f00840fea30 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f00837fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f00837fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34160/0 (socket says 172.21.15.93:34160) 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47082/0 (socket says 172.21.15.93:47082) 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f00837fe700 1 -- 172.21.15.93:0/1398449918 learned_addr learned my addr 172.21.15.93:0/1398449918 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0083fff700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 0x7f00841a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 -- 172.21.15.93:0/1398449918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 msgr2=0x7f0084196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 -- 172.21.15.93:0/1398449918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 msgr2=0x7f00841a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 0x7f00841a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 -- 172.21.15.93:0/1398449918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c009d00 con 0x7f00840ff3e0 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f00837fe700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f007800e9f0 tx=0x7f007800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0080ff9700 1 -- 172.21.15.93:0/1398449918 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007800c9c0 con 0x7f00840ff3e0 2023-07-21T22:23:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0080ff9700 1 -- 172.21.15.93:0/1398449918 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0078010430 con 0x7f00840ff3e0 2023-07-21T22:23:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00841a2700 con 0x7f00840ff3e0 2023-07-21T22:23:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f0080ff9700 1 -- 172.21.15.93:0/1398449918 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078018680 con 0x7f00840ff3e0 2023-07-21T22:23:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.476+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a2c10 con 0x7f00840ff3e0 2023-07-21T22:23:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.478+0000 7f0080ff9700 1 -- 172.21.15.93:0/1398449918 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0078010860 con 0x7f00840ff3e0 2023-07-21T22:23:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.479+0000 7f0080ff9700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 0x7f006c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.479+0000 7f00837fe700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 0x7f006c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.479+0000 7f0080ff9700 1 -- 172.21.15.93:0/1398449918 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0078014070 con 0x7f00840ff3e0 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.479+0000 7f00837fe700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 0x7f006c0a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0074005950 tx=0x7f0074009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 Processor -- start 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 -- start start 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be945e26b0 0x55be9478bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c660 0x55be947864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9468bc50 con 0x55be9478c210 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be946de9e0 con 0x55be945e26b0 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f009619fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be946d7e70 con 0x55be9478c660 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47090/0 (socket says 172.21.15.93:47090) 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 -- 172.21.15.93:0/3899486031 learned_addr learned my addr 172.21.15.93:0/3899486031 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f00837fe700 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be945e26b0 0x55be9478bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0083fff700 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c660 0x55be947864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 -- 172.21.15.93:0/3899486031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c660 msgr2=0x55be947864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c660 0x55be947864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 -- 172.21.15.93:0/3899486031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be945e26b0 msgr2=0x55be9478bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be945e26b0 0x55be9478bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 -- 172.21.15.93:0/3899486031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be94505d80 con 0x55be9478c210 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f00780a1e30 tx=0x7f0078099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6230448b15089e24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0069ffb700 1 -- 172.21.15.93:0/3899486031 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a2360 con 0x55be9478c210 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0069ffb700 1 -- 172.21.15.93:0/3899486031 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00780105a0 con 0x55be9478c210 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.480+0000 7f0069ffb700 1 -- 172.21.15.93:0/3899486031 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a1ed0 con 0x55be9478c210 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 -- 172.21.15.93:0/3899486031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 msgr2=0x55be94785f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f00780a1e30 tx=0x7f0078099e10 comp rx=0 tx=0).stop 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 -- 172.21.15.93:0/3899486031 shutdown_connections 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c660 0x55be947864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be945e26b0 0x55be9478bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 --2- 172.21.15.93:0/3899486031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9478c210 0x55be94785f40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 -- 172.21.15.93:0/3899486031 >> 172.21.15.93:0/3899486031 conn(0x55be9477bae0 msgr2=0x55be947810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 -- 172.21.15.93:0/3899486031 shutdown_connections 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.481+0000 7f009619fb80 1 -- 172.21.15.93:0/3899486031 wait complete. 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 -- start start 2023-07-21T22:23:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 Processor -- start 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c210 0x55be9478f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c660 0x55be9478fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be94677a60 con 0x55be945e26b0 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be94677760 con 0x55be9478c660 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9455b8c0 con 0x55be9478c210 2023-07-21T22:23:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f009619fb80 10 client.0 _ll_register_callbacks cb 0x55be94675700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47094/0 (socket says 172.21.15.93:47094) 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 -- 172.21.15.93:0/3349106142 learned_addr learned my addr 172.21.15.93:0/3349106142 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f0083fff700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c660 0x55be9478fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c210 0x55be9478f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 -- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c210 msgr2=0x55be9478f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c210 0x55be9478f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 -- 172.21.15.93:0/3349106142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c660 msgr2=0x55be9478fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c660 0x55be9478fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.482+0000 7f00837fe700 1 -- 172.21.15.93:0/3349106142 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00841a2c10 con 0x55be945e26b0 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f00837fe700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f007400ed70 tx=0x7f00740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d1afd0f3951958 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074022070 con 0x55be945e26b0 2023-07-21T22:23:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f007401dc90 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be94631490 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f007400ee60 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 10 client.16698 fetch_fsmap learned FSMap version 47 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55be944eec00 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be94631490 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 47) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f0074005430 con 0x55be945e26b0 2023-07-21T22:23:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 10 client.16698 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 20 client.16698 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9479fc00 0x55be947a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 4 client.16698 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.483+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be946d0e50 con 0x55be9479fc00 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.484+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9479fc00 0x55be947a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.484+0000 7f0062ffd700 10 client.16698 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.484+0000 7f0082ffd700 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9479fc00 0x55be947a2010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae6dea02aa93b9d server_cookie=b9d05b8fd517f36a in_seq=0 out_seq=0 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f0062ffd700 1 -- 172.21.15.93:0/3349106142 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be946d0e50 con 0x55be9479fc00 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f0062ffd700 10 client.16698 handle_command_reply: tid=0 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 client.16698 shutdown 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 msgr2=0x55be947925f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f007400ed70 tx=0x7f00740047c0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 shutdown_connections 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9479fc00 0x55be947a2010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c660 0x55be9478fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f0062ffd700 0 client.16698 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be945e26b0 0x55be947925f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 --2- 172.21.15.93:0/3349106142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be9478c210 0x55be9478f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 >> 172.21.15.93:0/3349106142 conn(0x55be9477bae0 msgr2=0x55be947803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 shutdown_connections 2023-07-21T22:23:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 1 -- 172.21.15.93:0/3349106142 wait complete. 2023-07-21T22:23:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.485+0000 7f009619fb80 20 client.16698 trim_cache size 0 max 16384 2023-07-21T22:23:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.500+0000 7f009619fb80 1 Processor -- start 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.500+0000 7f009619fb80 1 -- start start 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94792120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be946d0e50 con 0x55be9477bae0 2023-07-21T22:23:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9478bcd0 con 0x55be9478c210 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f009619fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be947829d0 con 0x55be945e26b0 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47102/0 (socket says 172.21.15.93:47102) 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 -- 172.21.15.93:0/4107707144 learned_addr learned my addr 172.21.15.93:0/4107707144 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f0082ffd700 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94792120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f0083fff700 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 -- 172.21.15.93:0/4107707144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 msgr2=0x55be94792120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94792120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 -- 172.21.15.93:0/4107707144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 msgr2=0x55be9478af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 -- 172.21.15.93:0/4107707144 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be9452c1f0 con 0x55be9477bae0 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f00837fe700 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f007401c010 tx=0x7f0074011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859956ac15afa80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f006a7fc700 1 -- 172.21.15.93:0/4107707144 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074022030 con 0x55be9477bae0 2023-07-21T22:23:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f006a7fc700 1 -- 172.21.15.93:0/4107707144 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f007400ee60 con 0x55be9477bae0 2023-07-21T22:23:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.501+0000 7f006a7fc700 1 -- 172.21.15.93:0/4107707144 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074026be0 con 0x55be9477bae0 2023-07-21T22:23:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- 172.21.15.93:0/4107707144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 msgr2=0x55be9478d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f007401c010 tx=0x7f0074011cb0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- 172.21.15.93:0/4107707144 shutdown_connections 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478d330 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 --2- 172.21.15.93:0/4107707144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94792120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- 172.21.15.93:0/4107707144 >> 172.21.15.93:0/4107707144 conn(0x55be9479b6b0 msgr2=0x55be9478a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- 172.21.15.93:0/4107707144 shutdown_connections 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- 172.21.15.93:0/4107707144 wait complete. 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 -- start start 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.502+0000 7f009619fb80 1 Processor -- start 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94796bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9478ede0 con 0x55be9477bae0 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be9478ef50 con 0x55be9478c210 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be947f3ac0 con 0x55be945e26b0 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 10 client.0 _ll_register_callbacks cb 0x7f007400e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47110/0 (socket says 172.21.15.93:47110) 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 -- 172.21.15.93:0/1531802678 learned_addr learned my addr 172.21.15.93:0/1531802678 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f00837fe700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94796bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0083fff700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 -- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 msgr2=0x55be94796bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94796bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 -- 172.21.15.93:0/1531802678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 msgr2=0x55be9478e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 -- 172.21.15.93:0/1531802678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be944eec00 con 0x55be9477bae0 2023-07-21T22:23:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0082ffd700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0078096410 tx=0x7f0078099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22869ff5dec5842d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0062ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007809a380 con 0x55be9477bae0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0078099400 con 0x55be9477bae0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.503+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00780105a0 con 0x55be9477bae0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00780995c0 con 0x55be9477bae0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 10 client.16704 fetch_fsmap learned FSMap version 47 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55be94504250 con 0x55be9477bae0 2023-07-21T22:23:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00780105a0 con 0x55be9477bae0 2023-07-21T22:23:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 47) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f00780ca050 con 0x55be9477bae0 2023-07-21T22:23:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 10 client.16704 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:23:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 20 client.16704 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9478c660 0x55be947f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 4 client.16704 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:23:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be94692ff0 con 0x55be9478c660 2023-07-21T22:23:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f00837fe700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9478c660 0x55be947f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f00837fe700 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9478c660 0x55be947f6e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458fe4bdbda6653a server_cookie=257f7f1b14e845d in_seq=0 out_seq=0 2023-07-21T22:23:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.504+0000 7f0062ffd700 10 client.16704 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f0062ffd700 1 -- 172.21.15.93:0/1531802678 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be94692ff0 con 0x55be9478c660 2023-07-21T22:23:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f0062ffd700 10 client.16704 handle_command_reply: tid=0 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 client.16704 shutdown 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 msgr2=0x55be9478e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0078096410 tx=0x7f0078099870 comp rx=0 tx=0).stop 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 shutdown_connections 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55be9478c660 0x55be947f6e80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be9478c210 0x55be9478e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55be945e26b0 0x55be94796bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 --2- 172.21.15.93:0/1531802678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55be9477bae0 0x55be9478e1d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 >> 172.21.15.93:0/1531802678 conn(0x55be9479b6b0 msgr2=0x55be9478b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.538+0000 7f0062ffd700 0 client.16704 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 shutdown_connections 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f009619fb80 1 -- 172.21.15.93:0/1531802678 wait complete. 2023-07-21T22:23:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f009619fb80 20 client.16704 trim_cache size 0 max 16384 2023-07-21T22:23:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:23:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:23:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:23:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 msgr2=0x7f006c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 0x7f006c0a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0074005950 tx=0x7f0074009260 comp rx=0 tx=0).stop 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 msgr2=0x7f0084197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f007800e9f0 tx=0x7f007800ed00 comp rx=0 tx=0).stop 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 shutdown_connections 2023-07-21T22:23:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0084104a30 0x7f00841a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00840fea30 0x7f0084196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f006c0a12c0 0x7f006c0a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 --2- 172.21.15.93:0/1398449918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00840ff3e0 0x7f0084197190 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 >> 172.21.15.93:0/1398449918 conn(0x7f0084074760 msgr2=0x7f00841025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 shutdown_connections 2023-07-21T22:23:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:43.539+0000 7f008a01a700 1 -- 172.21.15.93:0/1398449918 wait complete. 2023-07-21T22:23:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:43 smithi194 ceph-mon[98288]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2023-07-21T22:23:43.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:43 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:43.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:43 smithi134 ceph-mon[97873]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2023-07-21T22:23:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:43 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:43 smithi093 ceph-mon[96579]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 352 op/s 2023-07-21T22:23:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:43 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:44.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 292055 4.33 MB/sec execute 261 sec latency 46.287 ms 2023-07-21T22:23:45.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 294250 4.36 MB/sec execute 262 sec latency 19.098 ms 2023-07-21T22:23:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:45 smithi194 ceph-mon[98288]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:45.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:45 smithi134 ceph-mon[97873]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:45.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:45 smithi093 ceph-mon[96579]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:46.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 295579 4.37 MB/sec execute 263 sec latency 14.918 ms 2023-07-21T22:23:47.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 296994 4.38 MB/sec execute 264 sec latency 16.739 ms 2023-07-21T22:23:47.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:47 smithi194 ceph-mon[98288]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:47 smithi134 ceph-mon[97873]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:47 smithi093 ceph-mon[96579]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2023-07-21T22:23:48.332 INFO:tasks.workunit.client.0.smithi093.stdout: 1 298561 4.40 MB/sec execute 265 sec latency 18.491 ms 2023-07-21T22:23:49.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 300442 4.41 MB/sec execute 266 sec latency 25.854 ms 2023-07-21T22:23:49.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:49 smithi194 ceph-mon[98288]: pgmap v228: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2023-07-21T22:23:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:49 smithi134 ceph-mon[97873]: pgmap v228: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2023-07-21T22:23:49.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:49 smithi093 ceph-mon[96579]: pgmap v228: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2023-07-21T22:23:50.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 301984 4.44 MB/sec execute 267 sec latency 55.852 ms 2023-07-21T22:23:50.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:50 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:50.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:50 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:50.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:50 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:51.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 303217 4.44 MB/sec execute 268 sec latency 32.612 ms 2023-07-21T22:23:51.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:51 smithi194 ceph-mon[98288]: pgmap v229: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-07-21T22:23:51.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:51 smithi134 ceph-mon[97873]: pgmap v229: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-07-21T22:23:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:51 smithi093 ceph-mon[96579]: pgmap v229: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2023-07-21T22:23:52.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 305198 4.47 MB/sec execute 269 sec latency 38.096 ms 2023-07-21T22:23:53.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 306490 4.48 MB/sec execute 270 sec latency 15.793 ms 2023-07-21T22:23:53.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:53 smithi194 ceph-mon[98288]: pgmap v230: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-07-21T22:23:53.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:53 smithi134 ceph-mon[97873]: pgmap v230: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-07-21T22:23:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:53 smithi093 ceph-mon[96579]: pgmap v230: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 349 op/s 2023-07-21T22:23:54.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 308622 4.51 MB/sec execute 271 sec latency 55.745 ms 2023-07-21T22:23:55.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 309800 4.51 MB/sec execute 272 sec latency 13.374 ms 2023-07-21T22:23:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:55 smithi194 ceph-mon[98288]: pgmap v231: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:55 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:55.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:55 smithi134 ceph-mon[97873]: pgmap v231: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:55 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:55 smithi093 ceph-mon[96579]: pgmap v231: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:55 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:56.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 312000 4.54 MB/sec execute 273 sec latency 45.180 ms 2023-07-21T22:23:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 -- 172.21.15.93:0/3559743399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0ff3e0 msgr2=0x7efe9c10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 --2- 172.21.15.93:0/3559743399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c10b860 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 -- 172.21.15.93:0/3559743399 shutdown_connections 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 --2- 172.21.15.93:0/3559743399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c10b860 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 --2- 172.21.15.93:0/3559743399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c104a30 0x7efe9c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 --2- 172.21.15.93:0/3559743399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c0fea30 0x7efe9c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 -- 172.21.15.93:0/3559743399 >> 172.21.15.93:0/3559743399 conn(0x7efe9c074760 msgr2=0x7efe9c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 -- 172.21.15.93:0/3559743399 shutdown_connections 2023-07-21T22:23:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 -- 172.21.15.93:0/3559743399 wait complete. 2023-07-21T22:23:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.652+0000 7efea37f6700 1 Processor -- start 2023-07-21T22:23:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 -- start start 2023-07-21T22:23:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c104a30 0x7efe9c194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c194e80 con 0x7efe9c0fea30 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c194ff0 con 0x7efe9c104a30 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c1951b0 con 0x7efe9c0ff3e0 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47122/0 (socket says 172.21.15.93:47122) 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 -- 172.21.15.93:0/1548326295 learned_addr learned my addr 172.21.15.93:0/1548326295 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea0d91700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1d93700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c104a30 0x7efe9c194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 -- 172.21.15.93:0/1548326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c0ff3e0 msgr2=0x7efe9c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 -- 172.21.15.93:0/1548326295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c104a30 msgr2=0x7efe9c194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c104a30 0x7efe9c194810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 -- 172.21.15.93:0/1548326295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980095f0 con 0x7efe9c0fea30 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea1592700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efe9400efc0 tx=0x7efe9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efe927fc700 1 -- 172.21.15.93:0/1548326295 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94018070 con 0x7efe9c0fea30 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c195430 con 0x7efe9c0fea30 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efe927fc700 1 -- 172.21.15.93:0/1548326295 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9400f040 con 0x7efe9c0fea30 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.653+0000 7efe927fc700 1 -- 172.21.15.93:0/1548326295 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94004280 con 0x7efe9c0fea30 2023-07-21T22:23:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.654+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c1a19a0 con 0x7efe9c0fea30 2023-07-21T22:23:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.656+0000 7efe927fc700 1 -- 172.21.15.93:0/1548326295 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efe94003a00 con 0x7efe9c0fea30 2023-07-21T22:23:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.657+0000 7efe927fc700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 0x7efe880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.657+0000 7efea0d91700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 0x7efe880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.657+0000 7efe927fc700 1 -- 172.21.15.93:0/1548326295 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7efe940ccbf0 con 0x7efe9c0fea30 2023-07-21T22:23:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.657+0000 7efea0d91700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 0x7efe880a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0fc8d0 tx=0x7efe8c009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 Processor -- start 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 -- start start 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572823dfc50 con 0x5572824e0660 2023-07-21T22:23:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572824329e0 con 0x5572823366b0 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55728242be70 con 0x5572824e0210 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47134/0 (socket says 172.21.15.93:47134) 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea0d91700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 -- 172.21.15.93:0/1754432652 learned_addr learned my addr 172.21.15.93:0/1754432652 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1592700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 -- 172.21.15.93:0/1754432652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 msgr2=0x5572824d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 -- 172.21.15.93:0/1754432652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 msgr2=0x5572824dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.658+0000 7efea1d93700 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824dfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efea1d93700 1 -- 172.21.15.93:0/1754432652 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557282259d80 con 0x5572824e0660 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efea0d91700 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efea1592700 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efea1d93700 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f836d907d12e87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efe877fe700 1 -- 172.21.15.93:0/1754432652 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800ba50 con 0x5572824e0660 2023-07-21T22:23:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efe877fe700 1 -- 172.21.15.93:0/1754432652 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9800bbb0 con 0x5572824e0660 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.659+0000 7efe877fe700 1 -- 172.21.15.93:0/1754432652 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800f460 con 0x5572824e0660 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1754432652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 msgr2=0x5572824da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98005e80 comp rx=0 tx=0).stop 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1754432652 shutdown_connections 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x5572824da4a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1754432652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1754432652 >> 172.21.15.93:0/1754432652 conn(0x5572824cfae0 msgr2=0x5572824d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1754432652 shutdown_connections 2023-07-21T22:23:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1754432652 wait complete. 2023-07-21T22:23:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 -- start start 2023-07-21T22:23:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.660+0000 7efeaf97bb80 1 Processor -- start 2023-07-21T22:23:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572823cba60 con 0x5572824e0210 2023-07-21T22:23:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572823cb760 con 0x5572823366b0 2023-07-21T22:23:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572822af8c0 con 0x5572824e0660 2023-07-21T22:23:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 10 client.0 _ll_register_callbacks cb 0x5572823c9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47138/0 (socket says 172.21.15.93:47138) 2023-07-21T22:23:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 -- 172.21.15.93:0/3662473446 learned_addr learned my addr 172.21.15.93:0/3662473446 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea1592700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea1d93700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 -- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 msgr2=0x5572824e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 -- 172.21.15.93:0/3662473446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 msgr2=0x5572824e9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 -- 172.21.15.93:0/3662473446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c1a19a0 con 0x5572824e0210 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea1592700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efea0d91700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7efe8c01d8f0 tx=0x7efe8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e58862f525d4be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efe84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c022070 con 0x5572824e0210 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7efe8c004240 con 0x5572824e0210 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.661+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557282385490 con 0x5572824e0210 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8c004400 con 0x5572824e0210 2023-07-21T22:23:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 10 client.16716 fetch_fsmap learned FSMap version 49 2023-07-21T22:23:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7efe8c017070 con 0x5572824e0210 2023-07-21T22:23:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557282385490 con 0x5572824e0210 2023-07-21T22:23:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 49) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efe8c030760 con 0x5572824e0210 2023-07-21T22:23:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 10 client.16716 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:23:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 20 client.16716 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572824f39d0 0x5572824f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 4 client.16716 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557282424e50 con 0x5572824f39d0 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.662+0000 7efea1592700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572824f39d0 0x5572824f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efe84ff9700 10 client.16716 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efea1592700 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572824f39d0 0x5572824f5de0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596903a51539df01 server_cookie=16eb1b056ec9f940 in_seq=0 out_seq=0 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efe84ff9700 1 -- 172.21.15.93:0/3662473446 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557282424e50 con 0x5572824f39d0 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efe84ff9700 10 client.16716 handle_command_reply: tid=0 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 client.16716 shutdown 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 msgr2=0x5572824e9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7efe8c01d8f0 tx=0x7efe8c0047c0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 shutdown_connections 2023-07-21T22:23:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572824f39d0 0x5572824f5de0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efe84ff9700 0 client.16716 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/3662473446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0210 0x5572824e9aa0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 >> 172.21.15.93:0/3662473446 conn(0x5572824cfae0 msgr2=0x5572824d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 shutdown_connections 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.663+0000 7efeaf97bb80 1 -- 172.21.15.93:0/3662473446 wait complete. 2023-07-21T22:23:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.664+0000 7efeaf97bb80 20 client.16716 trim_cache size 0 max 16384 2023-07-21T22:23:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 Processor -- start 2023-07-21T22:23:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 -- start start 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824d4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572824de360 con 0x5572823366b0 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572824dad00 con 0x5572824e0210 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572824d1c00 con 0x5572824e0660 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efea1592700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efea1592700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47144/0 (socket says 172.21.15.93:47144) 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efea1592700 1 -- 172.21.15.93:0/2277208674 learned_addr learned my addr 172.21.15.93:0/2277208674 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efea0d91700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.679+0000 7efea1d93700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824d4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 -- 172.21.15.93:0/2277208674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 msgr2=0x5572824d4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824d4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 -- 172.21.15.93:0/2277208674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 msgr2=0x5572824d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 -- 172.21.15.93:0/2277208674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572824faa10 con 0x5572823366b0 2023-07-21T22:23:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea0d91700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efea1592700 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efe940ca3f0 tx=0x7efe9409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26809fbef31ed19 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efe87fff700 1 -- 172.21.15.93:0/2277208674 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9409dd80 con 0x5572823366b0 2023-07-21T22:23:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efe87fff700 1 -- 172.21.15.93:0/2277208674 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe940a2110 con 0x5572823366b0 2023-07-21T22:23:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.680+0000 7efe87fff700 1 -- 172.21.15.93:0/2277208674 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94099fb0 con 0x5572823366b0 2023-07-21T22:23:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- 172.21.15.93:0/2277208674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 msgr2=0x5572824df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7efe940ca3f0 tx=0x7efe9409f040 comp rx=0 tx=0).stop 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- 172.21.15.93:0/2277208674 shutdown_connections 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0660 0x5572824d4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572823366b0 0x5572824df180 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/2277208674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572824e0210 0x5572824d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- 172.21.15.93:0/2277208674 >> 172.21.15.93:0/2277208674 conn(0x5572824cfae0 msgr2=0x5572824e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- 172.21.15.93:0/2277208674 shutdown_connections 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- 172.21.15.93:0/2277208674 wait complete. 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 -- start start 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.681+0000 7efeaf97bb80 1 Processor -- start 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572824e8350 con 0x5572824e0660 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557282546dd0 con 0x5572823366b0 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557282546f10 con 0x5572824e0210 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 10 client.0 _ll_register_callbacks cb 0x5572823c9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47154/0 (socket says 172.21.15.93:47154) 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 -- 172.21.15.93:0/1050150760 learned_addr learned my addr 172.21.15.93:0/1050150760 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea0d91700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1592700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 -- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 msgr2=0x5572824e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 -- 172.21.15.93:0/1050150760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 msgr2=0x5572824e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 -- 172.21.15.93:0/1050150760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c017070 con 0x5572824e0660 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1592700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efea1d93700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efe98029000 tx=0x7efe9800ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efc8a0fafa780a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efe84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98030070 con 0x5572824e0660 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe9800bbb0 con 0x5572824e0660 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.682+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8527+0+0 (secure 0 0 0) 0x7efe98005030 con 0x5572824e0660 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe980051f0 con 0x5572824e0660 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 10 client.16722 fetch_fsmap learned FSMap version 49 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7efe9c195430 con 0x5572824e0660 2023-07-21T22:23:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9800bbb0 con 0x5572824e0660 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 49) v1 ==== 7482+0+0 (secure 0 0 0) 0x7efe9800f850 con 0x5572824e0660 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 10 client.16722 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 20 client.16722 populate_metadata read hostname 'smithi093' 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572825481c0 0x55728254a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 4 client.16722 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:23:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5572823e6ff0 con 0x5572825481c0 2023-07-21T22:23:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efea1592700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572825481c0 0x55728254a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efe84ff9700 10 client.16722 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.683+0000 7efea1592700 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572825481c0 0x55728254a5d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a294f2eb37753ab server_cookie=460d276b7afab607 in_seq=0 out_seq=0 2023-07-21T22:23:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efe84ff9700 1 -- 172.21.15.93:0/1050150760 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5572823e6ff0 con 0x5572825481c0 2023-07-21T22:23:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efe84ff9700 10 client.16722 handle_command_reply: tid=0 2023-07-21T22:23:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 client.16722 shutdown 2023-07-21T22:23:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 msgr2=0x557282546890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efe98029000 tx=0x7efe9800ba80 comp rx=0 tx=0).stop 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 shutdown_connections 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5572825481c0 0x55728254a5d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5572824e0660 0x557282546890 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572823366b0 0x5572824e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 --2- 172.21.15.93:0/1050150760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5572824e0210 0x5572824e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efe84ff9700 0 client.16722 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:23:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 >> 172.21.15.93:0/1050150760 conn(0x5572824cfae0 msgr2=0x5572824e1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 shutdown_connections 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.719+0000 7efeaf97bb80 1 -- 172.21.15.93:0/1050150760 wait complete. 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efeaf97bb80 20 client.16722 trim_cache size 0 max 16384 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:23:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:23:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:23:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 msgr2=0x7efe880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 0x7efe880a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0fc8d0 tx=0x7efe8c009260 comp rx=0 tx=0).stop 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 msgr2=0x7efe9c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efe9400efc0 tx=0x7efe9400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 shutdown_connections 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe9c104a30 0x7efe9c194810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7efe880a12c0 0x7efe880a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe9c0fea30 0x7efe9c19a9b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 --2- 172.21.15.93:0/1548326295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe9c0ff3e0 0x7efe9c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 >> 172.21.15.93:0/1548326295 conn(0x7efe9c074760 msgr2=0x7efe9c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 shutdown_connections 2023-07-21T22:23:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:23:56.720+0000 7efea37f6700 1 -- 172.21.15.93:0/1548326295 wait complete. 2023-07-21T22:23:57.236 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:23:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:23:57.236 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:23:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:23:57.236 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:23:56 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/889900016 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f27b2ff1e0 con 0x55f27b2d4c00 2023-07-21T22:23:57.236 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:23:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:23:57.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 313158 4.54 MB/sec execute 274 sec latency 25.603 ms 2023-07-21T22:23:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: pgmap v232: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:23:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:23:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/1717631683,v1:172.21.15.134:6837/1717631683] up:boot 2023-07-21T22:23:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:57 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:23:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: pgmap v232: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:57.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:23:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:23:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/1717631683,v1:172.21.15.134:6837/1717631683] up:boot 2023-07-21T22:23:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:57 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:23:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: pgmap v232: 97 pgs: 97 active+clean; 135 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 232 op/s 2023-07-21T22:23:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:23:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:23:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/1717631683,v1:172.21.15.134:6837/1717631683] up:boot 2023-07-21T22:23:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:23:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:57 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:23:58.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 314604 4.55 MB/sec execute 275 sec latency 17.378 ms 2023-07-21T22:23:59.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 316239 4.57 MB/sec execute 276 sec latency 19.246 ms 2023-07-21T22:23:59.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:23:59 smithi194 ceph-mon[98288]: pgmap v234: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:23:59.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:23:59 smithi134 ceph-mon[97873]: pgmap v234: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:23:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:23:59 smithi093 ceph-mon[96579]: pgmap v234: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:24:00.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 317978 4.58 MB/sec execute 277 sec latency 25.522 ms 2023-07-21T22:24:01.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 319512 4.61 MB/sec execute 278 sec latency 29.130 ms 2023-07-21T22:24:01.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:01 smithi194 ceph-mon[98288]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:24:01.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:01 smithi134 ceph-mon[97873]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:24:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:01 smithi093 ceph-mon[96579]: pgmap v235: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2023-07-21T22:24:02.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 320752 4.61 MB/sec execute 279 sec latency 15.782 ms 2023-07-21T22:24:03.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 322420 4.62 MB/sec execute 280 sec latency 15.704 ms 2023-07-21T22:24:03.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:03 smithi194 ceph-mon[98288]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:03.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:03 smithi134 ceph-mon[97873]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:03 smithi093 ceph-mon[96579]: pgmap v236: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:04.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 323585 4.63 MB/sec execute 281 sec latency 35.989 ms 2023-07-21T22:24:05.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 325370 4.64 MB/sec execute 282 sec latency 39.428 ms 2023-07-21T22:24:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:05 smithi194 ceph-mon[98288]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:05 smithi194 ceph-mon[98288]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:24:05.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:05 smithi134 ceph-mon[97873]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:05.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:05 smithi134 ceph-mon[97873]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:24:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:05 smithi093 ceph-mon[96579]: pgmap v237: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 330 op/s 2023-07-21T22:24:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:05 smithi093 ceph-mon[96579]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:24:06.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 326930 4.66 MB/sec execute 283 sec latency 23.923 ms 2023-07-21T22:24:07.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 328209 4.66 MB/sec execute 284 sec latency 28.177 ms 2023-07-21T22:24:07.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:07 smithi194 ceph-mon[98288]: pgmap v239: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-21T22:24:07.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:07 smithi134 ceph-mon[97873]: pgmap v239: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-21T22:24:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:07 smithi093 ceph-mon[96579]: pgmap v239: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2023-07-21T22:24:08.333 INFO:tasks.workunit.client.0.smithi093.stdout: 1 330170 4.70 MB/sec execute 285 sec latency 12.411 ms 2023-07-21T22:24:09.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 331415 4.70 MB/sec execute 286 sec latency 35.746 ms 2023-07-21T22:24:09.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:09 smithi194 ceph-mon[98288]: pgmap v240: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:09.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:09 smithi134 ceph-mon[97873]: pgmap v240: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:09 smithi093 ceph-mon[96579]: pgmap v240: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 -- 172.21.15.93:0/3791747894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7640ff3e0 msgr2=0x7fb76410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/3791747894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7640ff3e0 0x7fb76410b860 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00b0a0 tx=0x7fb75c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 -- 172.21.15.93:0/3791747894 shutdown_connections 2023-07-21T22:24:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/3791747894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7640ff3e0 0x7fb76410b860 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/3791747894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb764104a30 0x7fb764104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/3791747894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 0x7fb7640feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 -- 172.21.15.93:0/3791747894 >> 172.21.15.93:0/3791747894 conn(0x7fb764074760 msgr2=0x7fb764074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 -- 172.21.15.93:0/3791747894 shutdown_connections 2023-07-21T22:24:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.835+0000 7fb76c4cc700 1 -- 172.21.15.93:0/3791747894 wait complete. 2023-07-21T22:24:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 Processor -- start 2023-07-21T22:24:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 -- start start 2023-07-21T22:24:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 0x7fb76419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641960f0 con 0x7fb764104a30 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764196260 con 0x7fb7640ff3e0 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76c4cc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764196420 con 0x7fb7640fea30 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb769a67700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76a268700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 0x7fb76419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb769a67700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58328/0 (socket says 172.21.15.93:58328) 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47168/0 (socket says 172.21.15.93:47168) 2023-07-21T22:24:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb769a67700 1 -- 172.21.15.93:0/1547090163 learned_addr learned my addr 172.21.15.93:0/1547090163 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 -- 172.21.15.93:0/1547090163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 msgr2=0x7fb76419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 0x7fb76419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 -- 172.21.15.93:0/1547090163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 msgr2=0x7fb76419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb76aa69700 1 -- 172.21.15.93:0/1547090163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c009d00 con 0x7fb764104a30 2023-07-21T22:24:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.836+0000 7fb769a67700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00b070 tx=0x7fb75c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb75b7fe700 1 -- 172.21.15.93:0/1547090163 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00e040 con 0x7fb764104a30 2023-07-21T22:24:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7641966a0 con 0x7fb764104a30 2023-07-21T22:24:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb75b7fe700 1 -- 172.21.15.93:0/1547090163 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c01d470 con 0x7fb764104a30 2023-07-21T22:24:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb75b7fe700 1 -- 172.21.15.93:0/1547090163 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0052a0 con 0x7fb764104a30 2023-07-21T22:24:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.837+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7641a2c10 con 0x7fb764104a30 2023-07-21T22:24:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.839+0000 7fb75b7fe700 1 -- 172.21.15.93:0/1547090163 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb75c004a50 con 0x7fb764104a30 2023-07-21T22:24:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.839+0000 7fb75b7fe700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.839+0000 7fb76a268700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.839+0000 7fb75b7fe700 1 -- 172.21.15.93:0/1547090163 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb75c0ce000 con 0x7fb764104a30 2023-07-21T22:24:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb76a268700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 0x7fb7500a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7640ffba0 tx=0x7fb754009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb778651b80 1 Processor -- start 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb778651b80 1 -- start start 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb778651b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d3dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb778651b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.840+0000 7fb778651b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb778651b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115c3dc50 con 0x556115d3e210 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb778651b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115c909e0 con 0x556115b946b0 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb778651b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115c89e70 con 0x556115d3e660 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76a268700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d3dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47174/0 (socket says 172.21.15.93:47174) 2023-07-21T22:24:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb769a67700 1 -- 172.21.15.93:0/1290973441 learned_addr learned my addr 172.21.15.93:0/1290973441 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 -- 172.21.15.93:0/1290973441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 msgr2=0x556115d3dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d3dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 -- 172.21.15.93:0/1290973441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 msgr2=0x556115d37f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 -- 172.21.15.93:0/1290973441 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556115ab7d80 con 0x556115d3e660 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76a268700 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb769a67700 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb76aa69700 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d384a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0cb720 tx=0x7fb75c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a69064c60965af3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb758ff9700 1 -- 172.21.15.93:0/1290973441 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c01d5e0 con 0x556115d3e660 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb758ff9700 1 -- 172.21.15.93:0/1290973441 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c09b130 con 0x556115d3e660 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.841+0000 7fb758ff9700 1 -- 172.21.15.93:0/1290973441 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c09b390 con 0x556115d3e660 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 -- 172.21.15.93:0/1290973441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 msgr2=0x556115d384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d384a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0cb720 tx=0x7fb75c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 -- 172.21.15.93:0/1290973441 shutdown_connections 2023-07-21T22:24:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d384a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 --2- 172.21.15.93:0/1290973441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 -- 172.21.15.93:0/1290973441 >> 172.21.15.93:0/1290973441 conn(0x556115d2dae0 msgr2=0x556115d330e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 -- 172.21.15.93:0/1290973441 shutdown_connections 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.842+0000 7fb778651b80 1 -- 172.21.15.93:0/1290973441 wait complete. 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 -- start start 2023-07-21T22:24:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 Processor -- start 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d47300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115c29a60 con 0x556115d3e210 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115c29760 con 0x556115b946b0 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115b0d8c0 con 0x556115d3e660 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb778651b80 10 client.0 _ll_register_callbacks cb 0x556115c27700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47180/0 (socket says 172.21.15.93:47180) 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 -- 172.21.15.93:0/2836450263 learned_addr learned my addr 172.21.15.93:0/2836450263 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb76a268700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d47300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb76aa69700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 -- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 msgr2=0x556115d415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 -- 172.21.15.93:0/2836450263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 msgr2=0x556115d47300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d47300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 -- 172.21.15.93:0/2836450263 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb754017070 con 0x556115d3e210 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb76aa69700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb769a67700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb760012de0 tx=0x7fb76000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcfccf7fff870 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.843+0000 7fb74dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760017070 con 0x556115d3e210 2023-07-21T22:24:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb76000fb80 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556115be3490 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7600123a0 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 10 client.16740 fetch_fsmap learned FSMap version 51 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fb760009e10 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb760028080 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 51) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb760010430 con 0x556115d3e210 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:24:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 20 client.16740 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115d53cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 4 client.16740 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556115c82e50 con 0x556115d518e0 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.844+0000 7fb76a268700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115d53cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.845+0000 7fb76a268700 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115d53cf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83510de1ef8ab9ec server_cookie=2bd8e0c9a7b8c705 in_seq=0 out_seq=0 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.845+0000 7fb74dffb700 10 client.16740 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb74dffb700 1 -- 172.21.15.93:0/2836450263 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556115c82e50 con 0x556115d518e0 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb74dffb700 10 client.16740 handle_command_reply: tid=0 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 client.16740 shutdown 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 msgr2=0x556115d47840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb760012de0 tx=0x7fb76000f6a0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 shutdown_connections 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115d53cf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb74dffb700 0 client.16740 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115b946b0 0x556115d47300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 --2- 172.21.15.93:0/2836450263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d47840 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 >> 172.21.15.93:0/2836450263 conn(0x556115d2dae0 msgr2=0x556115d323e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 shutdown_connections 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 1 -- 172.21.15.93:0/2836450263 wait complete. 2023-07-21T22:24:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.846+0000 7fb778651b80 20 client.16740 trim_cache size 0 max 16384 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 Processor -- start 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 -- start start 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115b946b0 0x556115d37f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e660 0x556115d41f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115d33fb0 con 0x556115d3e210 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115d3c2c0 con 0x556115d3e660 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb778651b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115d58b90 con 0x556115b946b0 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47188/0 (socket says 172.21.15.93:47188) 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 -- 172.21.15.93:0/3810960098 learned_addr learned my addr 172.21.15.93:0/3810960098 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb76a268700 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115b946b0 0x556115d37f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb76aa69700 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e660 0x556115d41f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 -- 172.21.15.93:0/3810960098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115b946b0 msgr2=0x556115d37f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115b946b0 0x556115d37f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 -- 172.21.15.93:0/3810960098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e660 msgr2=0x556115d41f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e660 0x556115d41f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 -- 172.21.15.93:0/3810960098 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7641a2c10 con 0x556115d3e210 2023-07-21T22:24:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.861+0000 7fb769a67700 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb760010860 tx=0x7fb760005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0681efae622d6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb7597fa700 1 -- 172.21.15.93:0/3810960098 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760017040 con 0x556115d3e210 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb7597fa700 1 -- 172.21.15.93:0/3810960098 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556115be3490 con 0x556115d3e210 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb7597fa700 1 -- 172.21.15.93:0/3810960098 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760004280 con 0x556115d3e210 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 -- 172.21.15.93:0/3810960098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 msgr2=0x556115d58650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb760010860 tx=0x7fb760005dc0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 -- 172.21.15.93:0/3810960098 shutdown_connections 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e660 0x556115d41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115b946b0 0x556115d37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 --2- 172.21.15.93:0/3810960098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115d3e210 0x556115d58650 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 -- 172.21.15.93:0/3810960098 >> 172.21.15.93:0/3810960098 conn(0x556115d2dae0 msgr2=0x556115d41580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 -- 172.21.15.93:0/3810960098 shutdown_connections 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.862+0000 7fb778651b80 1 -- 172.21.15.93:0/3810960098 wait complete. 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 -- start start 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 Processor -- start 2023-07-21T22:24:09.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e210 0x556115d4a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115da48c0 con 0x556115b946b0 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115da4a30 con 0x556115d3e210 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556115da4bf0 con 0x556115d3e660 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb778651b80 10 client.0 _ll_register_callbacks cb 0x556115c27700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb76a268700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb76a268700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47192/0 (socket says 172.21.15.93:47192) 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb76a268700 1 -- 172.21.15.93:0/307586011 learned_addr learned my addr 172.21.15.93:0/307586011 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb769a67700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e210 0x556115d4a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.863+0000 7fb76aa69700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 -- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 msgr2=0x556115d4aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d4aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 -- 172.21.15.93:0/307586011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e210 msgr2=0x556115d4a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e210 0x556115d4a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 -- 172.21.15.93:0/307586011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb760009e10 con 0x556115b946b0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb76a268700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb7540117d0 tx=0x7fb7540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516a6dc1feef5a30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb74dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754022040 con 0x556115b946b0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb75402bcb0 con 0x556115b946b0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb75c09b130 con 0x556115b946b0 2023-07-21T22:24:09.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75402be70 con 0x556115b946b0 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb778651b80 10 client.16746 fetch_fsmap learned FSMap version 51 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x556115aa0c00 con 0x556115b946b0 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.864+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c09b130 con 0x556115b946b0 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 51) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb754003680 con 0x556115b946b0 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 20 client.16746 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:09.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115da83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 4 client.16746 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:24:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556115c44ff0 con 0x556115d518e0 2023-07-21T22:24:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb769a67700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115da83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb74dffb700 10 client.16746 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.865+0000 7fb769a67700 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115da83b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fef55b9740d7900 server_cookie=23c1ad8471c79dc7 in_seq=0 out_seq=0 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.899+0000 7fb74dffb700 1 -- 172.21.15.93:0/307586011 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556115c44ff0 con 0x556115d518e0 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.899+0000 7fb74dffb700 10 client.16746 handle_command_reply: tid=0 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.899+0000 7fb778651b80 1 client.16746 shutdown 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 msgr2=0x556115d4a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb7540117d0 tx=0x7fb7540047c0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 shutdown_connections 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556115d518e0 0x556115da83b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556115d3e660 0x556115d4aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556115b946b0 0x556115d4a030 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 --2- 172.21.15.93:0/307586011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556115d3e210 0x556115d4a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 >> 172.21.15.93:0/307586011 conn(0x556115d2dae0 msgr2=0x556115d470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb74dffb700 0 client.16746 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 shutdown_connections 2023-07-21T22:24:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 1 -- 172.21.15.93:0/307586011 wait complete. 2023-07-21T22:24:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.900+0000 7fb778651b80 20 client.16746 trim_cache size 0 max 16384 2023-07-21T22:24:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:24:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:24:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:24:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 msgr2=0x7fb7500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 0x7fb7500a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7640ffba0 tx=0x7fb754009260 comp rx=0 tx=0).stop 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 msgr2=0x7fb764195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00b070 tx=0x7fb75c0078e0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 shutdown_connections 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb764104a30 0x7fb764195a80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7640fea30 0x7fb76419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 --2- 172.21.15.93:0/1547090163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7640ff3e0 0x7fb76419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 >> 172.21.15.93:0/1547090163 conn(0x7fb764074760 msgr2=0x7fb764102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 shutdown_connections 2023-07-21T22:24:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:09.901+0000 7fb76c4cc700 1 -- 172.21.15.93:0/1547090163 wait complete. 2023-07-21T22:24:10.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 333562 4.73 MB/sec execute 287 sec latency 27.320 ms 2023-07-21T22:24:10.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:10 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:10.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:10 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:10.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:10 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:11.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 334830 4.73 MB/sec execute 288 sec latency 16.814 ms 2023-07-21T22:24:11.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:11 smithi194 ceph-mon[98288]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:11.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:11 smithi134 ceph-mon[97873]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:11.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:11 smithi093 ceph-mon[96579]: pgmap v241: 97 pgs: 97 active+clean; 141 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 301 op/s 2023-07-21T22:24:12.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 336201 4.74 MB/sec execute 289 sec latency 16.125 ms 2023-07-21T22:24:13.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 337643 4.75 MB/sec execute 290 sec latency 14.117 ms 2023-07-21T22:24:13.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:13 smithi093 ceph-mon[96579]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:13.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:13 smithi093 ceph-mon[96579]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:24:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:13 smithi194 ceph-mon[98288]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:13 smithi194 ceph-mon[98288]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:24:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:13 smithi134 ceph-mon[97873]: pgmap v242: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:13 smithi134 ceph-mon[97873]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:24:14.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 339069 4.75 MB/sec execute 291 sec latency 46.213 ms 2023-07-21T22:24:15.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 340894 4.78 MB/sec execute 292 sec latency 22.782 ms 2023-07-21T22:24:15.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:15 smithi093 ceph-mon[96579]: pgmap v244: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-07-21T22:24:15.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:15 smithi194 ceph-mon[98288]: pgmap v244: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-07-21T22:24:15.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:15 smithi134 ceph-mon[97873]: pgmap v244: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2023-07-21T22:24:16.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 342474 4.78 MB/sec execute 293 sec latency 16.188 ms 2023-07-21T22:24:17.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 344423 4.81 MB/sec execute 294 sec latency 23.172 ms 2023-07-21T22:24:17.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:17 smithi093 ceph-mon[96579]: pgmap v245: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:17.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:17 smithi194 ceph-mon[98288]: pgmap v245: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:17 smithi134 ceph-mon[97873]: pgmap v245: 97 pgs: 97 active+clean; 141 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:24:18.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 345657 4.81 MB/sec execute 295 sec latency 15.155 ms 2023-07-21T22:24:19.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 347261 4.83 MB/sec execute 296 sec latency 61.017 ms 2023-07-21T22:24:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:19 smithi093 ceph-mon[96579]: pgmap v246: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:19.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:19 smithi194 ceph-mon[98288]: pgmap v246: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:19.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:19 smithi134 ceph-mon[97873]: pgmap v246: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:20.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 348669 4.84 MB/sec execute 297 sec latency 38.729 ms 2023-07-21T22:24:21.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 350859 4.86 MB/sec execute 298 sec latency 17.111 ms 2023-07-21T22:24:21.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:21 smithi093 ceph-mon[96579]: pgmap v247: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:21.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:21 smithi194 ceph-mon[98288]: pgmap v247: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:21.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:21 smithi134 ceph-mon[97873]: pgmap v247: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 273 op/s 2023-07-21T22:24:22.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 352293 4.87 MB/sec execute 299 sec latency 13.643 ms 2023-07-21T22:24:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.015+0000 7f4f29a47700 1 -- 172.21.15.93:0/3654635538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f241093e0 msgr2=0x7f4f2410b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.015+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3654635538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f241093e0 0x7f4f2410b7c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b0a0 tx=0x7f4f1c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 -- 172.21.15.93:0/3654635538 shutdown_connections 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3654635538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f241093e0 0x7f4f2410b7c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3654635538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f24100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3654635538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f24100740 0x7f4f24108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 -- 172.21.15.93:0/3654635538 >> 172.21.15.93:0/3654635538 conn(0x7f4f24075270 msgr2=0x7f4f24075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 -- 172.21.15.93:0/3654635538 shutdown_connections 2023-07-21T22:24:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 -- 172.21.15.93:0/3654635538 wait complete. 2023-07-21T22:24:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 Processor -- start 2023-07-21T22:24:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.016+0000 7f4f29a47700 1 -- start start 2023-07-21T22:24:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f2419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f241093e0 0x7f4f24194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24195100 con 0x7f4f24100740 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24195270 con 0x7f4f240ffe30 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24195430 con 0x7f4f241093e0 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47206/0 (socket says 172.21.15.93:47206) 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 -- 172.21.15.93:0/3998246844 learned_addr learned my addr 172.21.15.93:0/3998246844 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f2419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f237fe700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f241093e0 0x7f4f24194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 -- 172.21.15.93:0/3998246844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f241093e0 msgr2=0x7f4f24194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f241093e0 0x7f4f24194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 -- 172.21.15.93:0/3998246844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 msgr2=0x7f4f2419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f2419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 -- 172.21.15.93:0/3998246844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c009d00 con 0x7f4f24100740 2023-07-21T22:24:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f2419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f227fc700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800e9a0 tx=0x7f4f1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f28a45700 1 -- 172.21.15.93:0/3998246844 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18009790 con 0x7f4f24100740 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f24195710 con 0x7f4f24100740 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f28a45700 1 -- 172.21.15.93:0/3998246844 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f18010430 con 0x7f4f24100740 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f28a45700 1 -- 172.21.15.93:0/3998246844 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18018680 con 0x7f4f24100740 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.017+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f241a1c20 con 0x7f4f24100740 2023-07-21T22:24:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.020+0000 7f4f28a45700 1 -- 172.21.15.93:0/3998246844 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4f180105a0 con 0x7f4f24100740 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.020+0000 7f4f28a45700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 0x7f4f0c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.020+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 0x7f4f0c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.020+0000 7f4f28a45700 1 -- 172.21.15.93:0/3998246844 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f18014070 con 0x7f4f24100740 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.021+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 0x7f4f0c0a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011490 tx=0x7f4f14009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.021+0000 7f4f35bccb80 1 Processor -- start 2023-07-21T22:24:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.021+0000 7f4f35bccb80 1 -- start start 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a176480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a1865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a07bc50 con 0x55d33a176a10 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a0ce9e0 con 0x55d339fd26b0 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a0c7e70 con 0x55d33a177d40 2023-07-21T22:24:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a176480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58374/0 (socket says 172.21.15.93:58374) 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 -- 172.21.15.93:0/3817890952 learned_addr learned my addr 172.21.15.93:0/3817890952 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 -- 172.21.15.93:0/3817890952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 msgr2=0x55d33a176480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a176480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 -- 172.21.15.93:0/3817890952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 msgr2=0x55d33a1865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a1865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 -- 172.21.15.93:0/3817890952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d339ef5d80 con 0x55d339fd26b0 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f140043c0 tx=0x7f4f14014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c7a2871de15f39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f09ffb700 1 -- 172.21.15.93:0/3817890952 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14003d10 con 0x55d339fd26b0 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f09ffb700 1 -- 172.21.15.93:0/3817890952 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f14003e70 con 0x55d339fd26b0 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.022+0000 7f4f09ffb700 1 -- 172.21.15.93:0/3817890952 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400e770 con 0x55d339fd26b0 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 -- 172.21.15.93:0/3817890952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 msgr2=0x55d33a175f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f140043c0 tx=0x7f4f14014000 comp rx=0 tx=0).stop 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 -- 172.21.15.93:0/3817890952 shutdown_connections 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a1865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a175f40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/3817890952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a176480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 -- 172.21.15.93:0/3817890952 >> 172.21.15.93:0/3817890952 conn(0x55d33a16bae0 msgr2=0x55d33a1710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 -- 172.21.15.93:0/3817890952 shutdown_connections 2023-07-21T22:24:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.023+0000 7f4f35bccb80 1 -- 172.21.15.93:0/3817890952 wait complete. 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 -- start start 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 Processor -- start 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 0x55d33a1865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a184020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a067a60 con 0x55d339fd26b0 2023-07-21T22:24:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a067760 con 0x55d33a176a10 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d339f4b8c0 con 0x55d33a177d40 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.024+0000 7f4f35bccb80 10 client.0 _ll_register_callbacks cb 0x55d33a065700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 0x55d33a1865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58382/0 (socket says 172.21.15.93:58382) 2023-07-21T22:24:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 0x55d33a1865f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47218/0 (socket says 172.21.15.93:47218) 2023-07-21T22:24:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 -- 172.21.15.93:0/768434514 learned_addr learned my addr 172.21.15.93:0/768434514 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f237fe700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a184020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 -- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 msgr2=0x55d33a184020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a184020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 -- 172.21.15.93:0/768434514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 msgr2=0x55d33a1865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 0x55d33a1865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 -- 172.21.15.93:0/768434514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f14017070 con 0x55d33a176a10 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f227fc700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f180963d0 tx=0x7f4f18099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6624a774484a748c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1809a380 con 0x55d33a176a10 2023-07-21T22:24:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d33a021490 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f180a2370 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f180b8300 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f35bccb80 10 client.26162 fetch_fsmap learned FSMap version 51 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.025+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d339edec00 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f180c8080 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4f180b8020 con 0x55d33a176a10 2023-07-21T22:24:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 10 client.26162 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 20 client.26162 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a196330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 4 client.26162 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d33a0c0e50 con 0x55d33a193f20 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f02ffd700 10 client.26162 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.026+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a196330 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27151aeb18e0b451 server_cookie=dafed3dab415bf6e in_seq=0 out_seq=0 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f02ffd700 1 -- 172.21.15.93:0/768434514 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d33a0c0e50 con 0x55d33a193f20 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f02ffd700 10 client.26162 handle_command_reply: tid=0 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 client.26162 shutdown 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 msgr2=0x55d33a1838e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f180963d0 tx=0x7f4f18099fa0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 shutdown_connections 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a196330 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a184020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f02ffd700 0 client.26162 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d339fd26b0 0x55d33a1865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/768434514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a1838e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 >> 172.21.15.93:0/768434514 conn(0x55d33a16bae0 msgr2=0x55d33a1703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 shutdown_connections 2023-07-21T22:24:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 1 -- 172.21.15.93:0/768434514 wait complete. 2023-07-21T22:24:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.027+0000 7f4f35bccb80 20 client.26162 trim_cache size 0 max 16384 2023-07-21T22:24:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.042+0000 7f4f35bccb80 1 Processor -- start 2023-07-21T22:24:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 -- start start 2023-07-21T22:24:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a18af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a17dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a17e7d0 con 0x55d33a176a10 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a186d70 con 0x55d339fd26b0 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a0c0e50 con 0x55d33a177d40 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47228/0 (socket says 172.21.15.93:47228) 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 -- 172.21.15.93:0/54853457 learned_addr learned my addr 172.21.15.93:0/54853457 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a18af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f237fe700 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a17dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 -- 172.21.15.93:0/54853457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 msgr2=0x55d33a17dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a17dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 -- 172.21.15.93:0/54853457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 msgr2=0x55d33a18af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a18af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 -- 172.21.15.93:0/54853457 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33a19af60 con 0x55d33a176a10 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a18af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f227fc700 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4f18096420 tx=0x7f4f18010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c623a803856727a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f0a7fc700 1 -- 172.21.15.93:0/54853457 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18099270 con 0x55d33a176a10 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f0a7fc700 1 -- 172.21.15.93:0/54853457 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d33a021490 con 0x55d33a176a10 2023-07-21T22:24:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.043+0000 7f4f0a7fc700 1 -- 172.21.15.93:0/54853457 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180ca480 con 0x55d33a176a10 2023-07-21T22:24:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- 172.21.15.93:0/54853457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 msgr2=0x55d33a17d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4f18096420 tx=0x7f4f18010a90 comp rx=0 tx=0).stop 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- 172.21.15.93:0/54853457 shutdown_connections 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d33a177d40 0x55d33a17dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d339fd26b0 0x55d33a18af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/54853457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a176a10 0x55d33a17d570 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- 172.21.15.93:0/54853457 >> 172.21.15.93:0/54853457 conn(0x55d33a16bae0 msgr2=0x55d33a185180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- 172.21.15.93:0/54853457 shutdown_connections 2023-07-21T22:24:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- 172.21.15.93:0/54853457 wait complete. 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.044+0000 7f4f35bccb80 1 -- start start 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 Processor -- start 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a181380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a18b4e0 con 0x55d33a177d40 2023-07-21T22:24:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a1800e0 con 0x55d33a176a10 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33a1802a0 con 0x55d339fd26b0 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f35bccb80 10 client.0 _ll_register_callbacks cb 0x7f4f1800e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f227fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a181380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47236/0 (socket says 172.21.15.93:47236) 2023-07-21T22:24:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f22ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34312/0 (socket says 172.21.15.93:34312) 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 -- 172.21.15.93:0/1060734905 learned_addr learned my addr 172.21.15.93:0/1060734905 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 -- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 msgr2=0x55d33a180e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 -- 172.21.15.93:0/1060734905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 msgr2=0x55d33a181380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a181380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f237fe700 1 -- 172.21.15.93:0/1060734905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d339ef5d80 con 0x55d33a177d40 2023-07-21T22:24:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f227fc700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a181380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.045+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f237fe700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b0a0 tx=0x7f4f1c004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e385b415307ea8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00e040 con 0x55d33a177d40 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4f1c009450 con 0x55d33a177d40 2023-07-21T22:24:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f14003e70 con 0x55d33a177d40 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1c0074e0 con 0x55d33a177d40 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 10 client.16761 fetch_fsmap learned FSMap version 51 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d339edec00 con 0x55d33a177d40 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f14003e70 con 0x55d33a177d40 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 51) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4f1c00b0d0 con 0x55d33a177d40 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 10 client.16761 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:24:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 20 client.16761 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a1eab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 4 client.16761 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:24:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.046+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d33a082ff0 con 0x55d33a193f20 2023-07-21T22:24:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.047+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a1eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.047+0000 7f4f02ffd700 10 client.16761 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.047+0000 7f4f22ffd700 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a1eab60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c916a026b7c7ab server_cookie=b2f577d057471887 in_seq=0 out_seq=0 2023-07-21T22:24:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f02ffd700 1 -- 172.21.15.93:0/1060734905 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d33a082ff0 con 0x55d33a193f20 2023-07-21T22:24:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f02ffd700 10 client.16761 handle_command_reply: tid=0 2023-07-21T22:24:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 client.16761 shutdown 2023-07-21T22:24:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 msgr2=0x55d33a17fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b0a0 tx=0x7f4f1c004c40 comp rx=0 tx=0).stop 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 shutdown_connections 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d33a193f20 0x55d33a1eab60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d33a177d40 0x55d33a17fba0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d339fd26b0 0x55d33a180e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 --2- 172.21.15.93:0/1060734905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33a176a10 0x55d33a181380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 >> 172.21.15.93:0/1060734905 conn(0x55d33a16bae0 msgr2=0x55d33a185a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.082+0000 7f4f02ffd700 0 client.16761 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.085+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 shutdown_connections 2023-07-21T22:24:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.085+0000 7f4f35bccb80 1 -- 172.21.15.93:0/1060734905 wait complete. 2023-07-21T22:24:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.085+0000 7f4f35bccb80 20 client.16761 trim_cache size 0 max 16384 2023-07-21T22:24:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:24:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:24:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:24:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:24:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 msgr2=0x7f4f0c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 0x7f4f0c0a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f14011490 tx=0x7f4f14009260 comp rx=0 tx=0).stop 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 msgr2=0x7f4f2419b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800e9a0 tx=0x7f4f1800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 shutdown_connections 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f241093e0 0x7f4f24194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f240ffe30 0x7f4f2419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4f0c0a12c0 0x7f4f0c0a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 --2- 172.21.15.93:0/3998246844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f24100740 0x7f4f2419b170 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 >> 172.21.15.93:0/3998246844 conn(0x7f4f24075270 msgr2=0x7f4f240fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 shutdown_connections 2023-07-21T22:24:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:23.086+0000 7f4f29a47700 1 -- 172.21.15.93:0/3998246844 wait complete. 2023-07-21T22:24:23.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 353626 4.87 MB/sec execute 300 sec latency 20.089 ms 2023-07-21T22:24:23.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:23 smithi093 ceph-mon[96579]: pgmap v248: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2023-07-21T22:24:23.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:23 smithi194 ceph-mon[98288]: pgmap v248: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2023-07-21T22:24:23.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:23 smithi134 ceph-mon[97873]: pgmap v248: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2023-07-21T22:24:24.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 355166 4.89 MB/sec execute 301 sec latency 23.132 ms 2023-07-21T22:24:25.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 356900 4.89 MB/sec execute 302 sec latency 25.010 ms 2023-07-21T22:24:25.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:25 smithi093 ceph-mon[96579]: pgmap v249: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:25.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:25 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:25 smithi194 ceph-mon[98288]: pgmap v249: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:25 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:25.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:25 smithi134 ceph-mon[97873]: pgmap v249: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:25 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:26.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 358500 4.92 MB/sec execute 303 sec latency 122.378 ms 2023-07-21T22:24:27.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 359898 4.92 MB/sec execute 304 sec latency 17.886 ms 2023-07-21T22:24:27.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:27 smithi093 ceph-mon[96579]: pgmap v250: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:27 smithi194 ceph-mon[98288]: pgmap v250: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:27.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:27 smithi134 ceph-mon[97873]: pgmap v250: 97 pgs: 97 active+clean; 154 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2023-07-21T22:24:28.334 INFO:tasks.workunit.client.0.smithi093.stdout: 1 362036 4.95 MB/sec execute 305 sec latency 15.247 ms 2023-07-21T22:24:29.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 363217 4.95 MB/sec execute 306 sec latency 18.185 ms 2023-07-21T22:24:29.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:29 smithi194 ceph-mon[98288]: pgmap v251: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2023-07-21T22:24:29.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:29 smithi134 ceph-mon[97873]: pgmap v251: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2023-07-21T22:24:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:29 smithi093 ceph-mon[96579]: pgmap v251: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 349 op/s 2023-07-21T22:24:30.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 364964 4.96 MB/sec execute 307 sec latency 35.222 ms 2023-07-21T22:24:31.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 366318 4.97 MB/sec execute 308 sec latency 17.846 ms 2023-07-21T22:24:31.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:31 smithi194 ceph-mon[98288]: pgmap v252: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2023-07-21T22:24:31.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:31 smithi134 ceph-mon[97873]: pgmap v252: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2023-07-21T22:24:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:31 smithi093 ceph-mon[96579]: pgmap v252: 97 pgs: 97 active+clean; 153 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 238 op/s 2023-07-21T22:24:32.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 368356 4.99 MB/sec execute 309 sec latency 23.225 ms 2023-07-21T22:24:33.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 369796 5.00 MB/sec execute 310 sec latency 14.168 ms 2023-07-21T22:24:33.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:33 smithi194 ceph-mon[98288]: pgmap v253: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 348 op/s 2023-07-21T22:24:33.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:33 smithi134 ceph-mon[97873]: pgmap v253: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 348 op/s 2023-07-21T22:24:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:33 smithi093 ceph-mon[96579]: pgmap v253: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 348 op/s 2023-07-21T22:24:34.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 371254 5.00 MB/sec execute 311 sec latency 30.535 ms 2023-07-21T22:24:35.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 372859 5.03 MB/sec execute 312 sec latency 25.018 ms 2023-07-21T22:24:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:35 smithi194 ceph-mon[98288]: pgmap v254: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:35 smithi134 ceph-mon[97873]: pgmap v254: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:35 smithi093 ceph-mon[96579]: pgmap v254: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.199+0000 7f9d15d27700 1 -- 172.21.15.93:0/663826383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10104a30 msgr2=0x7f9d10104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.199+0000 7f9d15d27700 1 --2- 172.21.15.93:0/663826383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10104a30 0x7f9d10104e00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009990 tx=0x7f9d00009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 -- 172.21.15.93:0/663826383 shutdown_connections 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 --2- 172.21.15.93:0/663826383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d100ff3e0 0x7f9d1010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 --2- 172.21.15.93:0/663826383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d10104a30 0x7f9d10104e00 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 --2- 172.21.15.93:0/663826383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 0x7f9d100feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 -- 172.21.15.93:0/663826383 >> 172.21.15.93:0/663826383 conn(0x7f9d10074760 msgr2=0x7f9d10074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 -- 172.21.15.93:0/663826383 shutdown_connections 2023-07-21T22:24:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 -- 172.21.15.93:0/663826383 wait complete. 2023-07-21T22:24:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 Processor -- start 2023-07-21T22:24:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 -- start start 2023-07-21T22:24:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.200+0000 7f9d15d27700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 0x7f9d1019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d10104a30 0x7f9d101948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10194f50 con 0x7f9d100ff3e0 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101950c0 con 0x7f9d10104a30 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10195280 con 0x7f9d100fea30 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47248/0 (socket says 172.21.15.93:47248) 2023-07-21T22:24:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 -- 172.21.15.93:0/1575138996 learned_addr learned my addr 172.21.15.93:0/1575138996 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 0x7f9d1019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d10104a30 0x7f9d101948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 -- 172.21.15.93:0/1575138996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 msgr2=0x7f9d1019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 0x7f9d1019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 -- 172.21.15.93:0/1575138996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d10104a30 msgr2=0x7f9d101948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d10104a30 0x7f9d101948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 -- 172.21.15.93:0/1575138996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000095f0 con 0x7f9d100ff3e0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0effd700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400d950 tx=0x7f9d0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0cff9700 1 -- 172.21.15.93:0/1575138996 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04009940 con 0x7f9d100ff3e0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0cff9700 1 -- 172.21.15.93:0/1575138996 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d04010460 con 0x7f9d100ff3e0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10195560 con 0x7f9d100ff3e0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.201+0000 7f9d0cff9700 1 -- 172.21.15.93:0/1575138996 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400f620 con 0x7f9d100ff3e0 2023-07-21T22:24:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.202+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d101a1a70 con 0x7f9d100ff3e0 2023-07-21T22:24:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.204+0000 7f9d0cff9700 1 -- 172.21.15.93:0/1575138996 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9d040108d0 con 0x7f9d100ff3e0 2023-07-21T22:24:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.204+0000 7f9d0cff9700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 0x7f9cf80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.204+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 0x7f9cf80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.204+0000 7f9d0cff9700 1 -- 172.21.15.93:0/1575138996 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9d040ccb20 con 0x7f9d100ff3e0 2023-07-21T22:24:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.205+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 0x7f9cf80a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000b5c0 tx=0x7f9d00005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.205+0000 7f9d21eacb80 1 Processor -- start 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.205+0000 7f9d21eacb80 1 -- start start 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 0x558e83086cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82f86c50 con 0x558e83087660 2023-07-21T22:24:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82fd99e0 con 0x558e83087210 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82fd2e70 con 0x558e82edd6b0 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47258/0 (socket says 172.21.15.93:47258) 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2238257045 learned_addr learned my addr 172.21.15.93:0/2238257045 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 0x558e83086cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0effd700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2238257045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 msgr2=0x558e83086cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 0x558e83086cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2238257045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087210 msgr2=0x558e83080f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2238257045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e82e00d80 con 0x558e83087660 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 0x558e83086cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009d70 tx=0x7f9d080077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51495ea38aa45fbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9cf5ffb700 1 -- 172.21.15.93:0/2238257045 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08014610 con 0x558e83087660 2023-07-21T22:24:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9cf5ffb700 1 -- 172.21.15.93:0/2238257045 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0800f050 con 0x558e83087660 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.206+0000 7f9cf5ffb700 1 -- 172.21.15.93:0/2238257045 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08013960 con 0x558e83087660 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2238257045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 msgr2=0x558e830814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009d70 tx=0x7f9d080077e0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2238257045 shutdown_connections 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e830814a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e82edd6b0 0x558e83086cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2238257045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2238257045 >> 172.21.15.93:0/2238257045 conn(0x558e83076ae0 msgr2=0x558e8307c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2238257045 shutdown_connections 2023-07-21T22:24:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.207+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2238257045 wait complete. 2023-07-21T22:24:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.208+0000 7f9d21eacb80 1 -- start start 2023-07-21T22:24:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.208+0000 7f9d21eacb80 1 Processor -- start 2023-07-21T22:24:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.208+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e830905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087210 0x558e83090b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82f72a60 con 0x558e83087210 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82f72760 con 0x558e82edd6b0 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82e568c0 con 0x558e83087660 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d21eacb80 10 client.0 _ll_register_callbacks cb 0x558e82f70700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e830905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e830905f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58422/0 (socket says 172.21.15.93:58422) 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34340/0 (socket says 172.21.15.93:34340) 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0f7fe700 1 -- 172.21.15.93:0/2809871012 learned_addr learned my addr 172.21.15.93:0/2809871012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0effd700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087210 0x558e83090b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2809871012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 msgr2=0x558e830905f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e830905f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2809871012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087210 msgr2=0x558e83090b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087210 0x558e83090b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2809871012 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1010c160 con 0x558e83087660 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558e83087a70 tx=0x7f9d08014af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3559f8e6f660e1cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.209+0000 7f9ceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08006000 con 0x558e83087660 2023-07-21T22:24:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e82f2c490 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9d08013540 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d080090b0 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 10 client.26125 fetch_fsmap learned FSMap version 52 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f9d080099a0 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e82f2c490 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 52) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d08027580 con 0x558e83087660 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 10 client.26125 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T22:24:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 20 client.26125 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e8309ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 4 client.26125 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e82fcbe50 con 0x558e8309aa00 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.210+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e8309ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.211+0000 7f9ceeffd700 10 client.26125 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.211+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e8309ce10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94946fcc68205a90 server_cookie=beb5bf95a195f42c in_seq=0 out_seq=0 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2809871012 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e82fcbe50 con 0x558e8309aa00 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9ceeffd700 10 client.26125 handle_command_reply: tid=0 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 client.26125 shutdown 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 msgr2=0x558e8308a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x558e83087a70 tx=0x7f9d08014af0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 shutdown_connections 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e8309ce10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087660 0x558e8308a880 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9ceeffd700 0 client.26125 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e830905f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2809871012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087210 0x558e83090b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 >> 172.21.15.93:0/2809871012 conn(0x558e83076ae0 msgr2=0x558e8307b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 shutdown_connections 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.212+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2809871012 wait complete. 2023-07-21T22:24:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.213+0000 7f9d21eacb80 20 client.26125 trim_cache size 0 max 16384 2023-07-21T22:24:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 Processor -- start 2023-07-21T22:24:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 -- start start 2023-07-21T22:24:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 0x558e8307bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e82f2c490 con 0x558e82edd6b0 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e83073b20 con 0x558e83087660 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e83078c00 con 0x558e83087210 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47266/0 (socket says 172.21.15.93:47266) 2023-07-21T22:24:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 -- 172.21.15.93:0/175939276 learned_addr learned my addr 172.21.15.93:0/175939276 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0effd700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 0x558e8307bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 -- 172.21.15.93:0/175939276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 msgr2=0x558e83080f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 -- 172.21.15.93:0/175939276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 msgr2=0x558e8307bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 0x558e8307bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0f7fe700 1 -- 172.21.15.93:0/175939276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e82de9c00 con 0x558e82edd6b0 2023-07-21T22:24:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.228+0000 7f9d0ffff700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 0x558e8307bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:24:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d00005f50 tx=0x7f9d00005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a96e7de2846a137 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9cf67fc700 1 -- 172.21.15.93:0/175939276 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0002a040 con 0x558e82edd6b0 2023-07-21T22:24:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9cf67fc700 1 -- 172.21.15.93:0/175939276 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0002b040 con 0x558e82edd6b0 2023-07-21T22:24:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9cf67fc700 1 -- 172.21.15.93:0/175939276 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000f460 con 0x558e82edd6b0 2023-07-21T22:24:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 -- 172.21.15.93:0/175939276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 msgr2=0x558e83088330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d00005f50 tx=0x7f9d00005000 comp rx=0 tx=0).stop 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 -- 172.21.15.93:0/175939276 shutdown_connections 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e83087660 0x558e8307bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e82edd6b0 0x558e83088330 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/175939276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83080f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 -- 172.21.15.93:0/175939276 >> 172.21.15.93:0/175939276 conn(0x558e83076ae0 msgr2=0x558e8308a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.229+0000 7f9d21eacb80 1 -- 172.21.15.93:0/175939276 shutdown_connections 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 -- 172.21.15.93:0/175939276 wait complete. 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 -- start start 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 Processor -- start 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e83090250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83090790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e8308e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8308eda0 con 0x558e83087660 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8308ef10 con 0x558e82edd6b0 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.230+0000 7f9d21eacb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8308f0d0 con 0x558e83087210 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d21eacb80 10 client.0 _ll_register_callbacks cb 0x7f9d04095f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e8308e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e83090250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83090790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e8308e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47276/0 (socket says 172.21.15.93:47276) 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0ffff700 1 -- 172.21.15.93:0/2064338867 learned_addr learned my addr 172.21.15.93:0/2064338867 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 -- 172.21.15.93:0/2064338867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 msgr2=0x558e83090250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e83090250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 -- 172.21.15.93:0/2064338867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 msgr2=0x558e8308e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e8308e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 -- 172.21.15.93:0/2064338867 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d00003910 con 0x558e83087210 2023-07-21T22:24:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d0effd700 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83090790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ca270 tx=0x7f9d04099830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d850d22a62ebee29 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9ceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04099e50 con 0x558e83087210 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9d04099fb0 con 0x558e83087210 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.231+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d0800f050 con 0x558e83087210 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d040990e0 con 0x558e83087210 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 10 client.26131 fetch_fsmap learned FSMap version 52 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x558e82e271f0 con 0x558e83087210 2023-07-21T22:24:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d040c8020 con 0x558e83087210 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 52) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d04098b50 con 0x558e83087210 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 10 client.26131 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 20 client.26131 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e830f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 4 client.26131 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:24:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e82f8dff0 con 0x558e8309aa00 2023-07-21T22:24:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.232+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e830f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.233+0000 7f9ceeffd700 10 client.26131 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.233+0000 7f9d0f7fe700 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e830f18a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d794a59261b54178 server_cookie=8bfada9954621800 in_seq=0 out_seq=0 2023-07-21T22:24:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.271+0000 7f9ceeffd700 1 -- 172.21.15.93:0/2064338867 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e82f8dff0 con 0x558e8309aa00 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.271+0000 7f9ceeffd700 10 client.26131 handle_command_reply: tid=0 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 client.26131 shutdown 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 msgr2=0x558e83090790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83090790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d040ca270 tx=0x7f9d04099830 comp rx=0 tx=0).stop 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 shutdown_connections 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558e8309aa00 0x558e830f18a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e83087660 0x558e8308e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e82edd6b0 0x558e83090250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9ceeffd700 0 client.26131 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 --2- 172.21.15.93:0/2064338867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e83087210 0x558e83090790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 >> 172.21.15.93:0/2064338867 conn(0x558e83076ae0 msgr2=0x558e83089030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 shutdown_connections 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 1 -- 172.21.15.93:0/2064338867 wait complete. 2023-07-21T22:24:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.272+0000 7f9d21eacb80 20 client.26131 trim_cache size 0 max 16384 2023-07-21T22:24:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:24:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:24:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:24:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:24:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 msgr2=0x7f9cf80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 0x7f9cf80a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000b5c0 tx=0x7f9d00005fd0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 msgr2=0x7f9d1019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400d950 tx=0x7f9d0400dc60 comp rx=0 tx=0).stop 2023-07-21T22:24:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 shutdown_connections 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d10104a30 0x7f9d101948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d100fea30 0x7f9d1019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cf80a1340 0x7f9cf80a37f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 --2- 172.21.15.93:0/1575138996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d100ff3e0 0x7f9d1019afc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.274+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 >> 172.21.15.93:0/1575138996 conn(0x7f9d10074760 msgr2=0x7f9d101026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.275+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 shutdown_connections 2023-07-21T22:24:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:36.275+0000 7f9d15d27700 1 -- 172.21.15.93:0/1575138996 wait complete. 2023-07-21T22:24:36.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 374522 5.03 MB/sec execute 313 sec latency 37.607 ms 2023-07-21T22:24:37.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 376346 5.05 MB/sec execute 314 sec latency 22.228 ms 2023-07-21T22:24:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:37 smithi194 ceph-mon[98288]: pgmap v255: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:37 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:37 smithi134 ceph-mon[97873]: pgmap v255: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:37 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:37.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:37 smithi093 ceph-mon[96579]: pgmap v255: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2023-07-21T22:24:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:37 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:38.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 377592 5.05 MB/sec execute 315 sec latency 15.756 ms 2023-07-21T22:24:38.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:38.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:38.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:24:38.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:24:38.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:24:39.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 379566 5.08 MB/sec execute 316 sec latency 29.439 ms 2023-07-21T22:24:39.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:39 smithi194 ceph-mon[98288]: pgmap v256: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2023-07-21T22:24:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:39 smithi134 ceph-mon[97873]: pgmap v256: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2023-07-21T22:24:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:39 smithi093 ceph-mon[96579]: pgmap v256: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2023-07-21T22:24:40.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 380747 5.07 MB/sec execute 317 sec latency 24.452 ms 2023-07-21T22:24:41.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 382949 5.10 MB/sec execute 318 sec latency 22.054 ms 2023-07-21T22:24:41.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:41 smithi194 ceph-mon[98288]: pgmap v257: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-07-21T22:24:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:41 smithi134 ceph-mon[97873]: pgmap v257: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-07-21T22:24:41.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:41 smithi093 ceph-mon[96579]: pgmap v257: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2023-07-21T22:24:42.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 384084 5.10 MB/sec execute 319 sec latency 23.110 ms 2023-07-21T22:24:43.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 385328 5.10 MB/sec execute 320 sec latency 22.023 ms 2023-07-21T22:24:43.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:43 smithi194 ceph-mon[98288]: pgmap v258: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2023-07-21T22:24:43.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:43 smithi134 ceph-mon[97873]: pgmap v258: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2023-07-21T22:24:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:43 smithi093 ceph-mon[96579]: pgmap v258: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2023-07-21T22:24:44.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 387018 5.12 MB/sec execute 321 sec latency 18.087 ms 2023-07-21T22:24:45.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 388335 5.12 MB/sec execute 322 sec latency 54.008 ms 2023-07-21T22:24:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:45 smithi194 ceph-mon[98288]: pgmap v259: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:45.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:45 smithi134 ceph-mon[97873]: pgmap v259: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:45.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:45 smithi093 ceph-mon[96579]: pgmap v259: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:46.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 390447 5.15 MB/sec execute 323 sec latency 20.428 ms 2023-07-21T22:24:47.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 391672 5.15 MB/sec execute 324 sec latency 16.157 ms 2023-07-21T22:24:47.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:47 smithi194 ceph-mon[98288]: pgmap v260: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:47.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:47 smithi134 ceph-mon[97873]: pgmap v260: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:47 smithi093 ceph-mon[96579]: pgmap v260: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2023-07-21T22:24:48.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 393708 5.17 MB/sec execute 325 sec latency 39.300 ms 2023-07-21T22:24:49.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 394893 5.17 MB/sec execute 326 sec latency 14.800 ms 2023-07-21T22:24:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- 172.21.15.93:0/88547783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 msgr2=0x7fb31c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 --2- 172.21.15.93:0/88547783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c104e00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- 172.21.15.93:0/88547783 shutdown_connections 2023-07-21T22:24:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 --2- 172.21.15.93:0/88547783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0ff3e0 0x7fb31c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 --2- 172.21.15.93:0/88547783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c104e00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 --2- 172.21.15.93:0/88547783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0fea30 0x7fb31c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- 172.21.15.93:0/88547783 >> 172.21.15.93:0/88547783 conn(0x7fb31c074760 msgr2=0x7fb31c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- 172.21.15.93:0/88547783 shutdown_connections 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- 172.21.15.93:0/88547783 wait complete. 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 Processor -- start 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 -- start start 2023-07-21T22:24:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.393+0000 7fb322f67700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0fea30 0x7fb31c196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0ff3e0 0x7fb31c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1a2040 con 0x7fb31c104a30 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1a21b0 con 0x7fb31c0ff3e0 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c1a2370 con 0x7fb31c0fea30 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47292/0 (socket says 172.21.15.93:47292) 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 -- 172.21.15.93:0/3274690040 learned_addr learned my addr 172.21.15.93:0/3274690040 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb320d03700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0fea30 0x7fb31c196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb31bfff700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0ff3e0 0x7fb31c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 -- 172.21.15.93:0/3274690040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0fea30 msgr2=0x7fb31c196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0fea30 0x7fb31c196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 -- 172.21.15.93:0/3274690040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0ff3e0 msgr2=0x7fb31c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0ff3e0 0x7fb31c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 -- 172.21.15.93:0/3274690040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100095f0 con 0x7fb31c104a30 2023-07-21T22:24:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb321504700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ef30 tx=0x7fb31400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb319ffb700 1 -- 172.21.15.93:0/3274690040 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314018070 con 0x7fb31c104a30 2023-07-21T22:24:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb319ffb700 1 -- 172.21.15.93:0/3274690040 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb31400f040 con 0x7fb31c104a30 2023-07-21T22:24:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb319ffb700 1 -- 172.21.15.93:0/3274690040 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400cd00 con 0x7fb31c104a30 2023-07-21T22:24:49.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.394+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c1a25f0 con 0x7fb31c104a30 2023-07-21T22:24:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.395+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c1a2b00 con 0x7fb31c104a30 2023-07-21T22:24:49.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.397+0000 7fb319ffb700 1 -- 172.21.15.93:0/3274690040 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb314003a00 con 0x7fb31c104a30 2023-07-21T22:24:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.398+0000 7fb319ffb700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 0x7fb30c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.398+0000 7fb319ffb700 1 -- 172.21.15.93:0/3274690040 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb3140cc7c0 con 0x7fb31c104a30 2023-07-21T22:24:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.398+0000 7fb320d03700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 0x7fb30c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.398+0000 7fb320d03700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 0x7fb30c0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b2e0 tx=0x7fb310005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 Processor -- start 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 -- start start 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d6acc50 con 0x55954d7b1620 2023-07-21T22:24:49.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d6ff9e0 con 0x55954d6036b0 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d6f8e70 con 0x55954d7b1a70 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47298/0 (socket says 172.21.15.93:47298) 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb31bfff700 1 -- 172.21.15.93:0/816207337 learned_addr learned my addr 172.21.15.93:0/816207337 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb320d03700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.399+0000 7fb321504700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 -- 172.21.15.93:0/816207337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 msgr2=0x55954d7b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 -- 172.21.15.93:0/816207337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 msgr2=0x55954d7ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 -- 172.21.15.93:0/816207337 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55954d526d80 con 0x55954d7b1a70 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb320d03700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb31bfff700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb321504700 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7ab8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3140c9f10 tx=0x7fb314095d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f34b3b73d34a445 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb306ffd700 1 -- 172.21.15.93:0/816207337 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31400ce60 con 0x55954d7b1a70 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb306ffd700 1 -- 172.21.15.93:0/816207337 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb314099920 con 0x55954d7b1a70 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.400+0000 7fb306ffd700 1 -- 172.21.15.93:0/816207337 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314099de0 con 0x55954d7b1a70 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/816207337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 msgr2=0x55954d7ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7ab8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3140c9f10 tx=0x7fb314095d90 comp rx=0 tx=0).stop 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/816207337 shutdown_connections 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7ab8b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/816207337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/816207337 >> 172.21.15.93:0/816207337 conn(0x55954d79cae0 msgr2=0x55954d7a20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/816207337 shutdown_connections 2023-07-21T22:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/816207337 wait complete. 2023-07-21T22:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 -- start start 2023-07-21T22:24:49.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.401+0000 7fb32f0ecb80 1 Processor -- start 2023-07-21T22:24:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1a70 0x55954d7b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d698a60 con 0x55954d6036b0 2023-07-21T22:24:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d698760 con 0x55954d7b1a70 2023-07-21T22:24:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d57c8c0 con 0x55954d7b1620 2023-07-21T22:24:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb32f0ecb80 10 client.0 _ll_register_callbacks cb 0x55954d696700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb320d03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34384/0 (socket says 172.21.15.93:34384) 2023-07-21T22:24:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 -- 172.21.15.93:0/1256318957 learned_addr learned my addr 172.21.15.93:0/1256318957 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb320d03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47302/0 (socket says 172.21.15.93:47302) 2023-07-21T22:24:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb321504700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1a70 0x55954d7b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 -- 172.21.15.93:0/1256318957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1a70 msgr2=0x55954d7b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1a70 0x55954d7b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 -- 172.21.15.93:0/1256318957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 msgr2=0x55954d7ba990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 -- 172.21.15.93:0/1256318957 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c10c160 con 0x55954d7b1620 2023-07-21T22:24:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb320d03700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb31bfff700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb308012a10 tx=0x7fb308012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c1952517485cec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.402+0000 7fb2fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:24:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308017070 con 0x55954d7b1620 2023-07-21T22:24:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55954d652490 con 0x55954d7b1620 2023-07-21T22:24:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb308004020 con 0x55954d7b1620 2023-07-21T22:24:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3080041e0 con 0x55954d7b1620 2023-07-21T22:24:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 10 client.26143 fetch_fsmap learned FSMap version 53 2023-07-21T22:24:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fb308009920 con 0x55954d7b1620 2023-07-21T22:24:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55954d652490 con 0x55954d7b1620 2023-07-21T22:24:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb3080037a0 con 0x55954d7b1620 2023-07-21T22:24:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 10 client.26143 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:24:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 20 client.26143 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d7c4ea0 0x55954d7c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 4 client.26143 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55954d6f1e50 con 0x55954d7c4ea0 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.403+0000 7fb320d03700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d7c4ea0 0x55954d7c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.404+0000 7fb2fbfff700 10 client.26143 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.404+0000 7fb320d03700 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d7c4ea0 0x55954d7c72b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ebdd1e065805ba server_cookie=be49e18ecfca30b7 in_seq=0 out_seq=0 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.404+0000 7fb2fbfff700 1 -- 172.21.15.93:0/1256318957 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55954d6f1e50 con 0x55954d7c4ea0 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.404+0000 7fb2fbfff700 10 client.26143 handle_command_reply: tid=0 2023-07-21T22:24:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.404+0000 7fb32f0ecb80 1 client.26143 shutdown 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 msgr2=0x55954d7baed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb308012a10 tx=0x7fb308012d20 comp rx=0 tx=0).stop 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 shutdown_connections 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d7c4ea0 0x55954d7c72b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1a70 0x55954d7b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7ba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb2fbfff700 0 client.26143 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1256318957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1620 0x55954d7baed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 >> 172.21.15.93:0/1256318957 conn(0x55954d79cae0 msgr2=0x55954d7a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 shutdown_connections 2023-07-21T22:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1256318957 wait complete. 2023-07-21T22:24:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.405+0000 7fb32f0ecb80 20 client.26143 trim_cache size 0 max 16384 2023-07-21T22:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 Processor -- start 2023-07-21T22:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 -- start start 2023-07-21T22:24:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1620 0x55954d7a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d7a2cb0 con 0x55954d6036b0 2023-07-21T22:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d79ec00 con 0x55954d7b1620 2023-07-21T22:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d7b0d30 con 0x55954d7b1a70 2023-07-21T22:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb320d03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb320d03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47310/0 (socket says 172.21.15.93:47310) 2023-07-21T22:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb320d03700 1 -- 172.21.15.93:0/1562637692 learned_addr learned my addr 172.21.15.93:0/1562637692 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb321504700 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.421+0000 7fb31bfff700 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1620 0x55954d7a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 -- 172.21.15.93:0/1562637692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 msgr2=0x55954d7a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 -- 172.21.15.93:0/1562637692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1620 msgr2=0x55954d7a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1620 0x55954d7a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 -- 172.21.15.93:0/1562637692 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c1a2b00 con 0x55954d6036b0 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb320d03700 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966a104340fda3d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb3077fe700 1 -- 172.21.15.93:0/1562637692 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31002d040 con 0x55954d6036b0 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb3077fe700 1 -- 172.21.15.93:0/1562637692 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb31002e040 con 0x55954d6036b0 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.422+0000 7fb3077fe700 1 -- 172.21.15.93:0/1562637692 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310034490 con 0x55954d6036b0 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1562637692 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 msgr2=0x55954d7b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310020000 comp rx=0 tx=0).stop 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1562637692 shutdown_connections 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d7a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d6036b0 0x55954d7b05b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/1562637692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d7b1620 0x55954d7a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1562637692 >> 172.21.15.93:0/1562637692 conn(0x55954d79cae0 msgr2=0x55954d7b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1562637692 shutdown_connections 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/1562637692 wait complete. 2023-07-21T22:24:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 -- start start 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.423+0000 7fb32f0ecb80 1 Processor -- start 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d817d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d7b97a0 con 0x55954d7b1620 2023-07-21T22:24:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d8182a0 con 0x55954d6036b0 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954d818460 con 0x55954d7b1a70 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 10 client.0 _ll_register_callbacks cb 0x7fb3080124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47318/0 (socket says 172.21.15.93:47318) 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 -- 172.21.15.93:0/866903207 learned_addr learned my addr 172.21.15.93:0/866903207 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb320d03700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb321504700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d817d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 -- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 msgr2=0x55954d817d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d817d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 -- 172.21.15.93:0/866903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 msgr2=0x55954d7b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 -- 172.21.15.93:0/866903207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310005350 con 0x55954d7b1620 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb320d03700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb31bfff700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb308000c00 tx=0x7fb30800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c0aa3ed4d1885b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb2fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308017040 con 0x55954d7b1620 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb3080050d0 con 0x55954d7b1620 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.424+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb314099920 con 0x55954d7b1620 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb308005290 con 0x55954d7b1620 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 10 client.16797 fetch_fsmap learned FSMap version 53 2023-07-21T22:24:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55954d50fc00 con 0x55954d7b1620 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb314099920 con 0x55954d7b1620 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb308003d20 con 0x55954d7b1620 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 10 client.16797 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 20 client.16797 populate_metadata read hostname 'smithi093' 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d819b70 0x55954d81bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 4 client.16797 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:24:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55954d6b3ff0 con 0x55954d819b70 2023-07-21T22:24:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb320d03700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d819b70 0x55954d81bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb2fbfff700 10 client.16797 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:49.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.425+0000 7fb320d03700 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d819b70 0x55954d81bf80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373fe32429b41206 server_cookie=5048ca9174b45b82 in_seq=0 out_seq=0 2023-07-21T22:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb2fbfff700 1 -- 172.21.15.93:0/866903207 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55954d6b3ff0 con 0x55954d819b70 2023-07-21T22:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb2fbfff700 10 client.16797 handle_command_reply: tid=0 2023-07-21T22:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 client.16797 shutdown 2023-07-21T22:24:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 msgr2=0x55954d7b9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb308000c00 tx=0x7fb30800fae0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 shutdown_connections 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55954d819b70 0x55954d81bf80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55954d7b1a70 0x55954d817d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb2fbfff700 0 client.16797 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55954d6036b0 0x55954d7b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 --2- 172.21.15.93:0/866903207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55954d7b1620 0x55954d7b9040 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 >> 172.21.15.93:0/866903207 conn(0x55954d79cae0 msgr2=0x55954d7a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 shutdown_connections 2023-07-21T22:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.467+0000 7fb32f0ecb80 1 -- 172.21.15.93:0/866903207 wait complete. 2023-07-21T22:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb32f0ecb80 20 client.16797 trim_cache size 0 max 16384 2023-07-21T22:24:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:24:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 msgr2=0x7fb30c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 0x7fb30c0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b2e0 tx=0x7fb310005fb0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 msgr2=0x7fb31c1a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb31400ef30 tx=0x7fb31400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 shutdown_connections 2023-07-21T22:24:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb30c0a1340 0x7fb30c0a37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31c104a30 0x7fb31c1a1a40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb31c0fea30 0x7fb31c196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 --2- 172.21.15.93:0/3274690040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb31c0ff3e0 0x7fb31c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 >> 172.21.15.93:0/3274690040 conn(0x7fb31c074760 msgr2=0x7fb31c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.468+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 shutdown_connections 2023-07-21T22:24:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:24:49.469+0000 7fb322f67700 1 -- 172.21.15.93:0/3274690040 wait complete. 2023-07-21T22:24:49.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:49 smithi194 ceph-mon[98288]: pgmap v261: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2023-07-21T22:24:49.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:49 smithi134 ceph-mon[97873]: pgmap v261: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2023-07-21T22:24:49.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:49 smithi093 ceph-mon[96579]: pgmap v261: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 342 op/s 2023-07-21T22:24:50.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 396258 5.17 MB/sec execute 327 sec latency 42.339 ms 2023-07-21T22:24:50.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:50 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:50.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:50 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:50.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:50 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:24:51.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 397744 5.19 MB/sec execute 328 sec latency 45.047 ms 2023-07-21T22:24:51.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:51 smithi194 ceph-mon[98288]: pgmap v262: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-21T22:24:51.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:51 smithi134 ceph-mon[97873]: pgmap v262: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-21T22:24:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:51 smithi093 ceph-mon[96579]: pgmap v262: 97 pgs: 97 active+clean; 166 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 218 op/s 2023-07-21T22:24:52.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 399004 5.19 MB/sec execute 329 sec latency 16.352 ms 2023-07-21T22:24:53.335 INFO:tasks.workunit.client.0.smithi093.stdout: 1 401013 5.21 MB/sec execute 330 sec latency 25.706 ms 2023-07-21T22:24:53.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:53 smithi134 ceph-mon[97873]: pgmap v263: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 337 op/s 2023-07-21T22:24:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:53 smithi093 ceph-mon[96579]: pgmap v263: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 337 op/s 2023-07-21T22:24:53.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:53 smithi194 ceph-mon[98288]: pgmap v263: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 337 op/s 2023-07-21T22:24:54.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 402294 5.21 MB/sec execute 331 sec latency 16.047 ms 2023-07-21T22:24:55.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 403758 5.22 MB/sec execute 332 sec latency 51.609 ms 2023-07-21T22:24:55.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:55 smithi134 ceph-mon[97873]: pgmap v264: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:55 smithi093 ceph-mon[96579]: pgmap v264: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:55.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:55 smithi194 ceph-mon[98288]: pgmap v264: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:56.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 405190 5.23 MB/sec execute 333 sec latency 12.746 ms 2023-07-21T22:24:57.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 407332 5.25 MB/sec execute 334 sec latency 19.200 ms 2023-07-21T22:24:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:57 smithi134 ceph-mon[97873]: pgmap v265: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:57 smithi093 ceph-mon[96579]: pgmap v265: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:57.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:57 smithi194 ceph-mon[98288]: pgmap v265: 97 pgs: 97 active+clean; 165 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2023-07-21T22:24:58.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 408738 5.25 MB/sec execute 335 sec latency 13.079 ms 2023-07-21T22:24:59.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 410743 5.27 MB/sec execute 336 sec latency 17.449 ms 2023-07-21T22:24:59.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:24:59 smithi134 ceph-mon[97873]: pgmap v266: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-21T22:24:59.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:24:59 smithi093 ceph-mon[96579]: pgmap v266: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-21T22:24:59.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:24:59 smithi194 ceph-mon[98288]: pgmap v266: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 337 op/s 2023-07-21T22:25:00.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 412111 5.28 MB/sec execute 337 sec latency 56.613 ms 2023-07-21T22:25:01.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 413299 5.28 MB/sec execute 338 sec latency 26.499 ms 2023-07-21T22:25:01.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:01 smithi134 ceph-mon[97873]: pgmap v267: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T22:25:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:01 smithi093 ceph-mon[96579]: pgmap v267: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T22:25:01.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:01 smithi194 ceph-mon[98288]: pgmap v267: 97 pgs: 97 active+clean; 171 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2023-07-21T22:25:02.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 415349 5.30 MB/sec execute 339 sec latency 12.568 ms 2023-07-21T22:25:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/2398508333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b40fec00 msgr2=0x7fe8b41046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/2398508333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b40fec00 0x7fe8b41046d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/2398508333 shutdown_connections 2023-07-21T22:25:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/2398508333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b4104db0 0x7fe8b410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/2398508333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fe260 0x7fe8b40fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/2398508333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b40fec00 0x7fe8b41046d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.584+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/2398508333 >> 172.21.15.93:0/2398508333 conn(0x7fe8b4075f90 msgr2=0x7fe8b4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.585+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/2398508333 shutdown_connections 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.585+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/2398508333 wait complete. 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.585+0000 7fe8b9b4e700 1 Processor -- start 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.585+0000 7fe8b9b4e700 1 -- start start 2023-07-21T22:25:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b40fe260 0x7fe8b4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b4104db0 0x7fe8b419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b419aeb0 con 0x7fe8b40fec00 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b419b020 con 0x7fe8b4104db0 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b419b1e0 con 0x7fe8b40fe260 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47332/0 (socket says 172.21.15.93:47332) 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/403477443 learned_addr learned my addr 172.21.15.93:0/403477443 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b3fff700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b4104db0 0x7fe8b419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b40fe260 0x7fe8b4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/403477443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b40fe260 msgr2=0x7fe8b4195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b40fe260 0x7fe8b4195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/403477443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b4104db0 msgr2=0x7fe8b419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b4104db0 0x7fe8b419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/403477443 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80095f0 con 0x7fe8b40fec00 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b0ff9700 1 -- 172.21.15.93:0/403477443 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a801d070 con 0x7fe8b40fec00 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b419b460 con 0x7fe8b40fec00 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b0ff9700 1 -- 172.21.15.93:0/403477443 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a8022470 con 0x7fe8b40fec00 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b0ff9700 1 -- 172.21.15.93:0/403477443 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017650 con 0x7fe8b40fec00 2023-07-21T22:25:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.586+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41a2d00 con 0x7fe8b40fec00 2023-07-21T22:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.589+0000 7fe8b0ff9700 1 -- 172.21.15.93:0/403477443 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe8a8022890 con 0x7fe8b40fec00 2023-07-21T22:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.589+0000 7fe8b0ff9700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 0x7fe89c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.589+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 0x7fe89c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.589+0000 7fe8b0ff9700 1 -- 172.21.15.93:0/403477443 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8a80ce0e0 con 0x7fe8b40fec00 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 0x7fe89c0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009c60 tx=0x7fe8a4009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 Processor -- start 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 -- start start 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0438f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a033ec50 con 0x5573a02956b0 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a03919e0 con 0x5573a043f210 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a038ae70 con 0x5573a043f660 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8b37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8b37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47338/0 (socket says 172.21.15.93:47338) 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8b37fe700 1 -- 172.21.15.93:0/4213904201 learned_addr learned my addr 172.21.15.93:0/4213904201 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.590+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0438f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b3fff700 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 -- 172.21.15.93:0/4213904201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 msgr2=0x5573a04394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04394a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 -- 172.21.15.93:0/4213904201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 msgr2=0x5573a0438f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0438f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 -- 172.21.15.93:0/4213904201 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a01b8d80 con 0x5573a02956b0 2023-07-21T22:25:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0438f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e4e0 tx=0x7fe8a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965127117a75a54a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe899ffb700 1 -- 172.21.15.93:0/4213904201 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400ee00 con 0x5573a02956b0 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe899ffb700 1 -- 172.21.15.93:0/4213904201 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a4021070 con 0x5573a02956b0 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.591+0000 7fe899ffb700 1 -- 172.21.15.93:0/4213904201 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401d7b0 con 0x5573a02956b0 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/4213904201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 msgr2=0x5573a043ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e4e0 tx=0x7fe8a4014000 comp rx=0 tx=0).stop 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/4213904201 shutdown_connections 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a02956b0 0x5573a043ecd0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/4213904201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0438f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/4213904201 >> 172.21.15.93:0/4213904201 conn(0x5573a042eae0 msgr2=0x5573a04340e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/4213904201 shutdown_connections 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/4213904201 wait complete. 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 -- start start 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 Processor -- start 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.592+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0445620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0442830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 0x5573a0442f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a032aa60 con 0x5573a043f660 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a032a760 con 0x5573a043f210 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a020e8c0 con 0x5573a02956b0 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 10 client.0 _ll_register_callbacks cb 0x5573a0328700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 0x5573a0442f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0442830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 0x5573a0442f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47348/0 (socket says 172.21.15.93:47348) 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b3fff700 1 -- 172.21.15.93:0/2542514377 learned_addr learned my addr 172.21.15.93:0/2542514377 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0445620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 msgr2=0x5573a0445620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0445620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/2542514377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 msgr2=0x5573a0442f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 0x5573a0442f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/2542514377 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41a2d00 con 0x5573a043f210 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0442830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cb8d0 tx=0x7fe8a809ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8760d443ae9e3c6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe892ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8011110 con 0x5573a043f210 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8a809f0d0 con 0x5573a043f210 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573a02e4490 con 0x5573a043f210 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a809b2e0 con 0x5573a043f210 2023-07-21T22:25:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 10 client.26213 fetch_fsmap learned FSMap version 53 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.593+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fe8a4017070 con 0x5573a043f210 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573a02e4490 con 0x5573a043f210 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe8a809b690 con 0x5573a043f210 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 20 client.26213 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a0452830 0x5573a0454c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 4 client.26213 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573a0383e50 con 0x5573a0452830 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a0452830 0x5573a0454c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe892ffd700 10 client.26213 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.594+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a0452830 0x5573a0454c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b902ff7206569d server_cookie=d1e91d6b7773eb61 in_seq=0 out_seq=0 2023-07-21T22:25:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe892ffd700 1 -- 172.21.15.93:0/2542514377 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573a0383e50 con 0x5573a0452830 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe892ffd700 10 client.26213 handle_command_reply: tid=0 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 client.26213 shutdown 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 msgr2=0x5573a0442830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0442830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80cb8d0 tx=0x7fe8a809ae20 comp rx=0 tx=0).stop 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 shutdown_connections 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a0452830 0x5573a0454c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f660 0x5573a0442f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe892ffd700 0 client.26213 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0445620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/2542514377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f210 0x5573a0442830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 >> 172.21.15.93:0/2542514377 conn(0x5573a042eae0 msgr2=0x5573a04333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 shutdown_connections 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.595+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/2542514377 wait complete. 2023-07-21T22:25:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.596+0000 7fe8c5cd3b80 20 client.26213 trim_cache size 0 max 16384 2023-07-21T22:25:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 Processor -- start 2023-07-21T22:25:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 -- start start 2023-07-21T22:25:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0445150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f660 0x5573a043c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a043e920 con 0x5573a043f210 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0439ff0 con 0x5573a043f660 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.611+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a043d500 con 0x5573a02956b0 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34432/0 (socket says 172.21.15.93:34432) 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 -- 172.21.15.93:0/1518121978 learned_addr learned my addr 172.21.15.93:0/1518121978 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0445150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b3fff700 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f660 0x5573a043c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 -- 172.21.15.93:0/1518121978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f660 msgr2=0x5573a043c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f660 0x5573a043c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 -- 172.21.15.93:0/1518121978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 msgr2=0x5573a0445150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0445150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 -- 172.21.15.93:0/1518121978 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a01a1c00 con 0x5573a02956b0 2023-07-21T22:25:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e4b0 tx=0x7fe8a401ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3621d7727708996c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe89a7fc700 1 -- 172.21.15.93:0/1518121978 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400ec00 con 0x5573a02956b0 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe89a7fc700 1 -- 172.21.15.93:0/1518121978 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a4033170 con 0x5573a02956b0 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.612+0000 7fe89a7fc700 1 -- 172.21.15.93:0/1518121978 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40055c0 con 0x5573a02956b0 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/1518121978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 msgr2=0x5573a0449c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400e4b0 tx=0x7fe8a401ced0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/1518121978 shutdown_connections 2023-07-21T22:25:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a043f660 0x5573a043c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a02956b0 0x5573a0449c40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/1518121978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0445150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/1518121978 >> 172.21.15.93:0/1518121978 conn(0x5573a042eae0 msgr2=0x5573a04595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/1518121978 shutdown_connections 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.613+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/1518121978 wait complete. 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 -- start start 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 Processor -- start 2023-07-21T22:25:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a02956b0 0x5573a0447e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a0448ae0 con 0x5573a043f210 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a04a5c60 con 0x5573a02956b0 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a04a5e20 con 0x5573a043f660 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8c5cd3b80 10 client.0 _ll_register_callbacks cb 0x7fe8a809b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47360/0 (socket says 172.21.15.93:47360) 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/3880160430 learned_addr learned my addr 172.21.15.93:0/3880160430 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a02956b0 0x5573a0447e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 msgr2=0x5573a04a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.614+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/3880160430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a02956b0 msgr2=0x5573a0447e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a02956b0 0x5573a0447e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8b2ffd700 1 -- 172.21.15.93:0/3880160430 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a402a040 con 0x5573a043f210 2023-07-21T22:25:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8b2ffd700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80098e0 tx=0x7fe8a8022f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab496dec1c6c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8011110 con 0x5573a043f210 2023-07-21T22:25:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8a80a36a0 con 0x5573a043f210 2023-07-21T22:25:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a4021070 con 0x5573a043f210 2023-07-21T22:25:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a809aa10 con 0x5573a043f210 2023-07-21T22:25:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 10 client.16815 fetch_fsmap learned FSMap version 53 2023-07-21T22:25:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x5573a01df1f0 con 0x5573a043f210 2023-07-21T22:25:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a80c9080 con 0x5573a043f210 2023-07-21T22:25:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe8a80970d0 con 0x5573a043f210 2023-07-21T22:25:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:25:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 20 client.16815 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a04a7310 0x5573a04a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 4 client.16815 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:25:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573a0345ff0 con 0x5573a04a7310 2023-07-21T22:25:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.615+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a04a7310 0x5573a04a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.616+0000 7fe892ffd700 10 client.16815 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.616+0000 7fe8b37fe700 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a04a7310 0x5573a04a9720 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76257a28d45a0c49 server_cookie=e8691727f53dea18 in_seq=0 out_seq=0 2023-07-21T22:25:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe892ffd700 1 -- 172.21.15.93:0/3880160430 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573a0345ff0 con 0x5573a04a7310 2023-07-21T22:25:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe892ffd700 10 client.16815 handle_command_reply: tid=0 2023-07-21T22:25:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 client.16815 shutdown 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 msgr2=0x5573a0448350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80098e0 tx=0x7fe8a8022f80 comp rx=0 tx=0).stop 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 shutdown_connections 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5573a04a7310 0x5573a04a9720 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5573a043f660 0x5573a04a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573a02956b0 0x5573a0447e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe892ffd700 0 client.16815 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 --2- 172.21.15.93:0/3880160430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5573a043f210 0x5573a0448350 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 >> 172.21.15.93:0/3880160430 conn(0x5573a042eae0 msgr2=0x5573a043c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 shutdown_connections 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 1 -- 172.21.15.93:0/3880160430 wait complete. 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.655+0000 7fe8c5cd3b80 20 client.16815 trim_cache size 0 max 16384 2023-07-21T22:25:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:25:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:25:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:25:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 msgr2=0x7fe89c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 0x7fe89c0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009c60 tx=0x7fe8a4009260 comp rx=0 tx=0).stop 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 msgr2=0x7fe8b41962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a8005740 comp rx=0 tx=0).stop 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 shutdown_connections 2023-07-21T22:25:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b4104db0 0x7fe8b419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b40fe260 0x7fe8b4195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe89c0a12c0 0x7fe89c0a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 --2- 172.21.15.93:0/403477443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b40fec00 0x7fe8b41962a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 >> 172.21.15.93:0/403477443 conn(0x7fe8b4075f90 msgr2=0x7fe8b40fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 shutdown_connections 2023-07-21T22:25:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:02.656+0000 7fe8b9b4e700 1 -- 172.21.15.93:0/403477443 wait complete. 2023-07-21T22:25:03.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 416546 5.30 MB/sec execute 340 sec latency 25.485 ms 2023-07-21T22:25:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:03 smithi134 ceph-mon[97873]: pgmap v268: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2023-07-21T22:25:03.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:03 smithi093 ceph-mon[96579]: pgmap v268: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2023-07-21T22:25:03.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:03 smithi194 ceph-mon[98288]: pgmap v268: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2023-07-21T22:25:04.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 418247 5.31 MB/sec execute 341 sec latency 13.937 ms 2023-07-21T22:25:05.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 419389 5.31 MB/sec execute 342 sec latency 64.992 ms 2023-07-21T22:25:05.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:05 smithi134 ceph-mon[97873]: pgmap v269: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-07-21T22:25:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:05 smithi134 ceph-mon[97873]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:25:05.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:05 smithi093 ceph-mon[96579]: pgmap v269: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-07-21T22:25:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:05 smithi093 ceph-mon[96579]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:25:05.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:05 smithi194 ceph-mon[98288]: pgmap v269: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 255 op/s 2023-07-21T22:25:05.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:05 smithi194 ceph-mon[98288]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:25:06.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 421345 5.33 MB/sec execute 343 sec latency 41.616 ms 2023-07-21T22:25:07.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 422751 5.34 MB/sec execute 344 sec latency 38.577 ms 2023-07-21T22:25:07.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:07 smithi134 ceph-mon[97873]: pgmap v271: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 306 op/s 2023-07-21T22:25:07.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:07 smithi093 ceph-mon[96579]: pgmap v271: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 306 op/s 2023-07-21T22:25:07.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:07 smithi194 ceph-mon[98288]: pgmap v271: 97 pgs: 97 active+clean; 178 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 306 op/s 2023-07-21T22:25:08.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 424048 5.34 MB/sec execute 345 sec latency 16.665 ms 2023-07-21T22:25:09.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 425977 5.36 MB/sec execute 346 sec latency 13.478 ms 2023-07-21T22:25:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:09 smithi134 ceph-mon[97873]: pgmap v272: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:09 smithi093 ceph-mon[96579]: pgmap v272: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:09.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:09 smithi194 ceph-mon[98288]: pgmap v272: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:10.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 427295 5.36 MB/sec execute 347 sec latency 18.839 ms 2023-07-21T22:25:11.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 429383 5.38 MB/sec execute 348 sec latency 22.346 ms 2023-07-21T22:25:11.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:11 smithi134 ceph-mon[97873]: pgmap v273: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:11 smithi093 ceph-mon[96579]: pgmap v273: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:11.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:11 smithi194 ceph-mon[98288]: pgmap v273: 97 pgs: 97 active+clean; 175 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 323 op/s 2023-07-21T22:25:12.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 430443 5.38 MB/sec execute 349 sec latency 32.713 ms 2023-07-21T22:25:13.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 432711 5.40 MB/sec execute 350 sec latency 17.727 ms 2023-07-21T22:25:13.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:13 smithi093 ceph-mon[96579]: pgmap v274: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:13.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:13 smithi093 ceph-mon[96579]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:25:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:13 smithi194 ceph-mon[98288]: pgmap v274: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:13 smithi194 ceph-mon[98288]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:25:13.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:13 smithi134 ceph-mon[97873]: pgmap v274: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:13 smithi134 ceph-mon[97873]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:25:14.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 433744 5.40 MB/sec execute 351 sec latency 14.472 ms 2023-07-21T22:25:15.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 435664 5.41 MB/sec execute 352 sec latency 47.736 ms 2023-07-21T22:25:15.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:15 smithi093 ceph-mon[96579]: pgmap v276: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2023-07-21T22:25:15.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:15 smithi194 ceph-mon[98288]: pgmap v276: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2023-07-21T22:25:15.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:15 smithi134 ceph-mon[97873]: pgmap v276: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2023-07-21T22:25:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 -- 172.21.15.93:0/3371683516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0fea30 msgr2=0x7fa97c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 --2- 172.21.15.93:0/3371683516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0fea30 0x7fa97c0feea0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009990 tx=0x7fa96c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 -- 172.21.15.93:0/3371683516 shutdown_connections 2023-07-21T22:25:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 --2- 172.21.15.93:0/3371683516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 --2- 172.21.15.93:0/3371683516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 0x7fa97c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 --2- 172.21.15.93:0/3371683516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0fea30 0x7fa97c0feea0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 -- 172.21.15.93:0/3371683516 >> 172.21.15.93:0/3371683516 conn(0x7fa97c074760 msgr2=0x7fa97c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 -- 172.21.15.93:0/3371683516 shutdown_connections 2023-07-21T22:25:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.772+0000 7fa980904700 1 -- 172.21.15.93:0/3371683516 wait complete. 2023-07-21T22:25:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 Processor -- start 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 -- start start 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0fea30 0x7fa97c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 0x7fa97c1948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c194f20 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c195090 con 0x7fa97c104a30 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa980904700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c195250 con 0x7fa97c0fea30 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47374/0 (socket says 172.21.15.93:47374) 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 -- 172.21.15.93:0/646950818 learned_addr learned my addr 172.21.15.93:0/646950818 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 0x7fa97c1948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa97a59c700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0fea30 0x7fa97c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 -- 172.21.15.93:0/646950818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0fea30 msgr2=0x7fa97c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0fea30 0x7fa97c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.773+0000 7fa979d9b700 1 -- 172.21.15.93:0/646950818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 msgr2=0x7fa97c1948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa979d9b700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 0x7fa97c1948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa979d9b700 1 -- 172.21.15.93:0/646950818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0095f0 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa979d9b700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa9737fe700 1 -- 172.21.15.93:0/646950818 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c01d070 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa9737fe700 1 -- 172.21.15.93:0/646950818 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa96c021410 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa9737fe700 1 -- 172.21.15.93:0/646950818 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c017650 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c1954d0 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.774+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c1a1a40 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.776+0000 7fa9737fe700 1 -- 172.21.15.93:0/646950818 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa96c021580 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.777+0000 7fa9737fe700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 0x7fa9680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.777+0000 7fa97a59c700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 0x7fa9680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.777+0000 7fa9737fe700 1 -- 172.21.15.93:0/646950818 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa96c0cdf20 con 0x7fa97c0ff3e0 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.778+0000 7fa97a59c700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 0x7fa9680a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0ffba0 tx=0x7fa964009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.778+0000 7fa98ca89b80 1 Processor -- start 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.778+0000 7fa98ca89b80 1 -- start start 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 0x55d7b8b1cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8a22c50 con 0x55d7b8b23660 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8a759e0 con 0x55d7b89796b0 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8a6ee70 con 0x55d7b8b23210 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47384/0 (socket says 172.21.15.93:47384) 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58542/0 (socket says 172.21.15.93:58542) 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97ad9d700 1 -- 172.21.15.93:0/1059046501 learned_addr learned my addr 172.21.15.93:0/1059046501 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa979d9b700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 0x55d7b8b1cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 -- 172.21.15.93:0/1059046501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 msgr2=0x55d7b8b1cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 0x55d7b8b1cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 -- 172.21.15.93:0/1059046501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 msgr2=0x55d7b8b1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 -- 172.21.15.93:0/1059046501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b889cd80 con 0x55d7b89796b0 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa979d9b700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 0x55d7b8b1cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa97a59c700 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa964011c80 tx=0x7fa964014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdaca522ac6b74de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa970ff9700 1 -- 172.21.15.93:0/1059046501 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964004220 con 0x55d7b89796b0 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa970ff9700 1 -- 172.21.15.93:0/1059046501 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa964021070 con 0x55d7b89796b0 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.779+0000 7fa970ff9700 1 -- 172.21.15.93:0/1059046501 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964003b80 con 0x55d7b89796b0 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 -- 172.21.15.93:0/1059046501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 msgr2=0x55d7b8b22cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa964011c80 tx=0x7fa964014000 comp rx=0 tx=0).stop 2023-07-21T22:25:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 -- 172.21.15.93:0/1059046501 shutdown_connections 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b89796b0 0x55d7b8b22cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/1059046501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23210 0x55d7b8b1cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 -- 172.21.15.93:0/1059046501 >> 172.21.15.93:0/1059046501 conn(0x55d7b8b12ae0 msgr2=0x55d7b8b180e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 -- 172.21.15.93:0/1059046501 shutdown_connections 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.780+0000 7fa98ca89b80 1 -- 172.21.15.93:0/1059046501 wait complete. 2023-07-21T22:25:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 -- start start 2023-07-21T22:25:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 Processor -- start 2023-07-21T22:25:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b29540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b26830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8a0ea60 con 0x55d7b8b23660 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8a0e760 con 0x55d7b8b23210 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b88f28c0 con 0x55d7b89796b0 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa98ca89b80 10 client.0 _ll_register_callbacks cb 0x55d7b8a0c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47390/0 (socket says 172.21.15.93:47390) 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 -- 172.21.15.93:0/110755097 learned_addr learned my addr 172.21.15.93:0/110755097 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b29540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa979d9b700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b26830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 -- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 msgr2=0x55d7b8b29540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b29540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 -- 172.21.15.93:0/110755097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 msgr2=0x55d7b8b26830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b26830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 -- 172.21.15.93:0/110755097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9740112b0 con 0x55d7b8b23660 2023-07-21T22:25:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa9740143a0 tx=0x7fa9740146b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ab6c5105fc8462 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.781+0000 7fa961ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400e040 con 0x55d7b8b23660 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7b89c8490 con 0x55d7b8b23660 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa97401a070 con 0x55d7b8b23660 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9740092d0 con 0x55d7b8b23660 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 10 client.16827 fetch_fsmap learned FSMap version 53 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7fa964017070 con 0x55d7b8b23660 2023-07-21T22:25:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7b89c8490 con 0x55d7b8b23660 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa974004590 con 0x55d7b8b23660 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 20 client.16827 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b36ae0 0x55d7b8b38ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 4 client.16827 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.782+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7b8a67e50 con 0x55d7b8b36ae0 2023-07-21T22:25:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.783+0000 7fa97a59c700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b36ae0 0x55d7b8b38ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.783+0000 7fa961ffb700 10 client.16827 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.783+0000 7fa97a59c700 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b36ae0 0x55d7b8b38ef0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff628df7413fa9b server_cookie=edca09041f076d34 in_seq=0 out_seq=0 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa961ffb700 1 -- 172.21.15.93:0/110755097 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d7b8a67e50 con 0x55d7b8b36ae0 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa961ffb700 10 client.16827 handle_command_reply: tid=0 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 client.16827 shutdown 2023-07-21T22:25:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 msgr2=0x55d7b8b26eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa9740143a0 tx=0x7fa9740146b0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 shutdown_connections 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b36ae0 0x55d7b8b38ef0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b26eb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa961ffb700 0 client.16827 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b29540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/110755097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b26830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 >> 172.21.15.93:0/110755097 conn(0x55d7b8b12ae0 msgr2=0x55d7b8b173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 shutdown_connections 2023-07-21T22:25:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 1 -- 172.21.15.93:0/110755097 wait complete. 2023-07-21T22:25:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.784+0000 7fa98ca89b80 20 client.16827 trim_cache size 0 max 16384 2023-07-21T22:25:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 Processor -- start 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 -- start start 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b1cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b22900 con 0x55d7b8b23660 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b144f0 con 0x55d7b8b23210 2023-07-21T22:25:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b24dd0 con 0x55d7b89796b0 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa97ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47398/0 (socket says 172.21.15.93:47398) 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa97ad9d700 1 -- 172.21.15.93:0/2046507376 learned_addr learned my addr 172.21.15.93:0/2046507376 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b1cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.799+0000 7fa979d9b700 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 -- 172.21.15.93:0/2046507376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 msgr2=0x55d7b8b1cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b1cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 -- 172.21.15.93:0/2046507376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 msgr2=0x55d7b8b221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 -- 172.21.15.93:0/2046507376 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b8885c00 con 0x55d7b8b23660 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa974015010 tx=0x7fa974009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49256f12cca9e479 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa9717fa700 1 -- 172.21.15.93:0/2046507376 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400e070 con 0x55d7b8b23660 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa9717fa700 1 -- 172.21.15.93:0/2046507376 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa97400f040 con 0x55d7b8b23660 2023-07-21T22:25:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.800+0000 7fa9717fa700 1 -- 172.21.15.93:0/2046507376 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974027430 con 0x55d7b8b23660 2023-07-21T22:25:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- 172.21.15.93:0/2046507376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 msgr2=0x55d7b8b2db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa974015010 tx=0x7fa974009dd0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- 172.21.15.93:0/2046507376 shutdown_connections 2023-07-21T22:25:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b8b23660 0x55d7b8b2db70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b89796b0 0x55d7b8b1cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/2046507376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- 172.21.15.93:0/2046507376 >> 172.21.15.93:0/2046507376 conn(0x55d7b8b12ae0 msgr2=0x55d7b8b23a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- 172.21.15.93:0/2046507376 shutdown_connections 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- 172.21.15.93:0/2046507376 wait complete. 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- start start 2023-07-21T22:25:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 Processor -- start 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 0x55d7b8b2c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b2e150 con 0x55d7b89796b0 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b2c5f0 con 0x55d7b8b23210 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.801+0000 7fa98ca89b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b8b2c780 con 0x55d7b8b23660 2023-07-21T22:25:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa98ca89b80 10 client.0 _ll_register_callbacks cb 0x7fa9740119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47400/0 (socket says 172.21.15.93:47400) 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 -- 172.21.15.93:0/4288582940 learned_addr learned my addr 172.21.15.93:0/4288582940 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 0x55d7b8b2c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa979d9b700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 -- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 msgr2=0x55d7b8b2c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 0x55d7b8b2c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 -- 172.21.15.93:0/4288582940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 msgr2=0x55d7b8b2bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b2bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 -- 172.21.15.93:0/4288582940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa964017070 con 0x55d7b89796b0 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97ad9d700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 0x55d7b8b2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa97a59c700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa96400e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c060b129169e86ac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa961ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96402c040 con 0x55d7b89796b0 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa96400ece0 con 0x55d7b89796b0 2023-07-21T22:25:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa964021070 con 0x55d7b89796b0 2023-07-21T22:25:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa964031600 con 0x55d7b89796b0 2023-07-21T22:25:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa98ca89b80 10 client.16833 fetch_fsmap learned FSMap version 53 2023-07-21T22:25:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.802+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55d7b88c31f0 con 0x55d7b89796b0 2023-07-21T22:25:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa964021070 con 0x55d7b89796b0 2023-07-21T22:25:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 53) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa9640054d0 con 0x55d7b89796b0 2023-07-21T22:25:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T22:25:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 20 client.16833 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b8bc40 0x55d7b8b8e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 4 client.16833 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:25:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d7b8a29ff0 con 0x55d7b8b8bc40 2023-07-21T22:25:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa979d9b700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b8bc40 0x55d7b8b8e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa961ffb700 10 client.16833 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.803+0000 7fa979d9b700 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b8bc40 0x55d7b8b8e050 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af6aa6624812972 server_cookie=c7cb49c3478578cb in_seq=0 out_seq=0 2023-07-21T22:25:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.848+0000 7fa961ffb700 1 -- 172.21.15.93:0/4288582940 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d7b8a29ff0 con 0x55d7b8b8bc40 2023-07-21T22:25:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.848+0000 7fa961ffb700 10 client.16833 handle_command_reply: tid=0 2023-07-21T22:25:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.848+0000 7fa98ca89b80 1 client.16833 shutdown 2023-07-21T22:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.848+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 msgr2=0x55d7b8b2d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.848+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa96400e7b0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 shutdown_connections 2023-07-21T22:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d7b8b8bc40 0x55d7b8b8e050 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d7b8b23660 0x55d7b8b2c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d7b89796b0 0x55d7b8b2d880 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa961ffb700 0 client.16833 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 --2- 172.21.15.93:0/4288582940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d7b8b23210 0x55d7b8b2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 >> 172.21.15.93:0/4288582940 conn(0x55d7b8b12ae0 msgr2=0x55d7b8b22050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 shutdown_connections 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 1 -- 172.21.15.93:0/4288582940 wait complete. 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.849+0000 7fa98ca89b80 20 client.16833 trim_cache size 0 max 16384 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:25:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:25:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:25:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 msgr2=0x7fa9680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 0x7fa9680a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0ffba0 tx=0x7fa964009260 comp rx=0 tx=0).stop 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 msgr2=0x7fa97c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c004790 comp rx=0 tx=0).stop 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 shutdown_connections 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa97c104a30 0x7fa97c1948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa97c0fea30 0x7fa97c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa9680a1340 0x7fa9680a37f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 --2- 172.21.15.93:0/646950818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97c0ff3e0 0x7fa97c19af90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 >> 172.21.15.93:0/646950818 conn(0x7fa97c074760 msgr2=0x7fa97c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 shutdown_connections 2023-07-21T22:25:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:15.850+0000 7fa980904700 1 -- 172.21.15.93:0/646950818 wait complete. 2023-07-21T22:25:16.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 436905 5.42 MB/sec execute 353 sec latency 37.749 ms 2023-07-21T22:25:16.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:25:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:25:16.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:25:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:25:16.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:25:16 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/3785132433 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55deaa39d1e0 con 0x55deaa372c00 2023-07-21T22:25:16.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:25:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:25:17.336 INFO:tasks.workunit.client.0.smithi093.stdout: 1 438872 5.42 MB/sec execute 354 sec latency 22.287 ms 2023-07-21T22:25:17.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: pgmap v277: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:17.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:25:17.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:25:17.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/544935120,v1:172.21.15.134:6837/544935120] up:boot 2023-07-21T22:25:17.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:17.443 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:17 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:25:17.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: pgmap v277: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:25:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:25:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/544935120,v1:172.21.15.134:6837/544935120] up:boot 2023-07-21T22:25:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:17 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:25:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: pgmap v277: 97 pgs: 97 active+clean; 182 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2023-07-21T22:25:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:25:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:25:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/544935120,v1:172.21.15.134:6837/544935120] up:boot 2023-07-21T22:25:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:17 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:25:18.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 440429 5.44 MB/sec execute 355 sec latency 10.435 ms 2023-07-21T22:25:19.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 441716 5.44 MB/sec execute 356 sec latency 18.938 ms 2023-07-21T22:25:19.442 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:19 smithi093 ceph-mon[96579]: pgmap v279: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 363 op/s 2023-07-21T22:25:19.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:19 smithi194 ceph-mon[98288]: pgmap v279: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 363 op/s 2023-07-21T22:25:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:19 smithi134 ceph-mon[97873]: pgmap v279: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 363 op/s 2023-07-21T22:25:20.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 443684 5.46 MB/sec execute 357 sec latency 12.525 ms 2023-07-21T22:25:21.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 444857 5.46 MB/sec execute 358 sec latency 87.759 ms 2023-07-21T22:25:21.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:21 smithi194 ceph-mon[98288]: pgmap v280: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:25:21.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:21 smithi134 ceph-mon[97873]: pgmap v280: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:25:21.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:21 smithi093 ceph-mon[96579]: pgmap v280: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:25:22.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 447055 5.48 MB/sec execute 359 sec latency 20.711 ms 2023-07-21T22:25:23.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 448130 5.48 MB/sec execute 360 sec latency 65.052 ms 2023-07-21T22:25:23.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:23 smithi194 ceph-mon[98288]: pgmap v281: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:23.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:23 smithi134 ceph-mon[97873]: pgmap v281: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:23.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:23 smithi093 ceph-mon[96579]: pgmap v281: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:24.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 449438 5.48 MB/sec execute 361 sec latency 18.056 ms 2023-07-21T22:25:25.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 450971 5.49 MB/sec execute 362 sec latency 13.339 ms 2023-07-21T22:25:25.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:25 smithi194 ceph-mon[98288]: pgmap v282: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:25 smithi134 ceph-mon[97873]: pgmap v282: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:25 smithi093 ceph-mon[96579]: pgmap v282: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:26.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 452822 5.50 MB/sec execute 363 sec latency 28.257 ms 2023-07-21T22:25:27.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 454438 5.52 MB/sec execute 364 sec latency 14.728 ms 2023-07-21T22:25:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:27 smithi194 ceph-mon[98288]: pgmap v283: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:27.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:27 smithi134 ceph-mon[97873]: pgmap v283: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:27.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:27 smithi093 ceph-mon[96579]: pgmap v283: 97 pgs: 97 active+clean; 187 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 291 op/s 2023-07-21T22:25:28.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 456180 5.52 MB/sec execute 365 sec latency 15.205 ms 2023-07-21T22:25:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 -- 172.21.15.93:0/953363611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 msgr2=0x7f69dc10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 --2- 172.21.15.93:0/953363611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc10ca00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f69d400b0d0 tx=0x7f69d400b3e0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 -- 172.21.15.93:0/953363611 shutdown_connections 2023-07-21T22:25:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 --2- 172.21.15.93:0/953363611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc10ca00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 --2- 172.21.15.93:0/953363611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 0x7f69dc067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 --2- 172.21.15.93:0/953363611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 0x7f69dc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 -- 172.21.15.93:0/953363611 >> 172.21.15.93:0/953363611 conn(0x7f69dc074760 msgr2=0x7f69dc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 -- 172.21.15.93:0/953363611 shutdown_connections 2023-07-21T22:25:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.966+0000 7f69e466e700 1 -- 172.21.15.93:0/953363611 wait complete. 2023-07-21T22:25:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 Processor -- start 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 -- start start 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 0x7f69dc0fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 0x7f69dc0ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1a4400 con 0x7f69dc104e50 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1a4540 con 0x7f69dc067c10 2023-07-21T22:25:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e466e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc1a46b0 con 0x7f69dc0685b0 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47418/0 (socket says 172.21.15.93:47418) 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 -- 172.21.15.93:0/2611824319 learned_addr learned my addr 172.21.15.93:0/2611824319 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e1c09700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 0x7f69dc0ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e240a700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 0x7f69dc0fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 -- 172.21.15.93:0/2611824319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 msgr2=0x7f69dc0ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 0x7f69dc0ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 -- 172.21.15.93:0/2611824319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 msgr2=0x7f69dc0fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 0x7f69dc0fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 -- 172.21.15.93:0/2611824319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009d00 con 0x7f69dc104e50 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f69d4007870 tx=0x7f69d40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.967+0000 7f69d37fe700 1 -- 172.21.15.93:0/2611824319 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d400e040 con 0x7f69dc104e50 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.968+0000 7f69d37fe700 1 -- 172.21.15.93:0/2611824319 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d4005070 con 0x7f69dc104e50 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.968+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc1a4930 con 0x7f69dc104e50 2023-07-21T22:25:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.968+0000 7f69d37fe700 1 -- 172.21.15.93:0/2611824319 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d401b6c0 con 0x7f69dc104e50 2023-07-21T22:25:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.968+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc1a4d50 con 0x7f69dc104e50 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.970+0000 7f69d37fe700 1 -- 172.21.15.93:0/2611824319 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f69d40051e0 con 0x7f69dc104e50 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.971+0000 7f69d37fe700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 0x7f69c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.971+0000 7f69e240a700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 0x7f69c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.971+0000 7f69d37fe700 1 -- 172.21.15.93:0/2611824319 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f69d40cde30 con 0x7f69dc104e50 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.971+0000 7f69f07f3b80 1 Processor -- start 2023-07-21T22:25:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.971+0000 7f69e240a700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 0x7f69c80a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005950 tx=0x7f69cc00b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 -- start start 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cd7fc50 con 0x56205ce80660 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cdd29e0 con 0x56205ccd66b0 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cdcbe70 con 0x56205ce80210 2023-07-21T22:25:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e1c09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47426/0 (socket says 172.21.15.93:47426) 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 -- 172.21.15.93:0/354591826 learned_addr learned my addr 172.21.15.93:0/354591826 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e240a700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 -- 172.21.15.93:0/354591826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 msgr2=0x56205ce79f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce79f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 -- 172.21.15.93:0/354591826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 msgr2=0x56205ce7fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce7fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 -- 172.21.15.93:0/354591826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56205cbf9d80 con 0x56205ce80660 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e1c09700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e240a700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.972+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f69d40974b0 tx=0x7f69d409ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5f4223a864fbeb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69d0ff9700 1 -- 172.21.15.93:0/354591826 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40a3370 con 0x56205ce80660 2023-07-21T22:25:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69d0ff9700 1 -- 172.21.15.93:0/354591826 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d40054c0 con 0x56205ce80660 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69d0ff9700 1 -- 172.21.15.93:0/354591826 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d409a4a0 con 0x56205ce80660 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 -- 172.21.15.93:0/354591826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 msgr2=0x56205ce7a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f69d40974b0 tx=0x7f69d409ae90 comp rx=0 tx=0).stop 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 -- 172.21.15.93:0/354591826 shutdown_connections 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7a4a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce7fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.973+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/354591826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- 172.21.15.93:0/354591826 >> 172.21.15.93:0/354591826 conn(0x56205ce6fae0 msgr2=0x56205ce750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- 172.21.15.93:0/354591826 shutdown_connections 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- 172.21.15.93:0/354591826 wait complete. 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- start start 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 Processor -- start 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cd6ba60 con 0x56205ce80660 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cd6b760 con 0x56205ccd66b0 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cc4f8c0 con 0x56205ce80210 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69f07f3b80 10 client.0 _ll_register_callbacks cb 0x56205cd69700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69e240a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47432/0 (socket says 172.21.15.93:47432) 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69e1c09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.974+0000 7f69e2c0b700 1 -- 172.21.15.93:0/3481033449 learned_addr learned my addr 172.21.15.93:0/3481033449 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 -- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 msgr2=0x56205ce89820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce89820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 -- 172.21.15.93:0/3481033449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 msgr2=0x56205ce892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 -- 172.21.15.93:0/3481033449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d80098e0 con 0x56205ce80660 2023-07-21T22:25:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e240a700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e1c09700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce89820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f69d40cbb20 tx=0x7f69d4097970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c12fec0d5e5cc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40a3370 con 0x56205ce80660 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f69d409a6b0 con 0x56205ce80660 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56205cd25490 con 0x56205ce80660 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d40b91b0 con 0x56205ce80660 2023-07-21T22:25:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69f07f3b80 10 client.16851 fetch_fsmap learned FSMap version 54 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f69dc1a4d50 con 0x56205ce80660 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.975+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56205cd25490 con 0x56205ce80660 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 54) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f69d40a3000 con 0x56205ce80660 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 10 client.16851 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 20 client.16851 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205ce93430 0x56205ce95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 4 client.16851 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56205cdc4e50 con 0x56205ce93430 2023-07-21T22:25:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69e240a700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205ce93430 0x56205ce95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69c5ffb700 10 client.16851 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.976+0000 7f69e240a700 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205ce93430 0x56205ce95840 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92dedd14ff900944 server_cookie=c5d3144832681038 in_seq=0 out_seq=0 2023-07-21T22:25:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69c5ffb700 1 -- 172.21.15.93:0/3481033449 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56205cdc4e50 con 0x56205ce93430 2023-07-21T22:25:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69c5ffb700 10 client.16851 handle_command_reply: tid=0 2023-07-21T22:25:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 client.16851 shutdown 2023-07-21T22:25:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 msgr2=0x56205ce83570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f69d40cbb20 tx=0x7f69d4097970 comp rx=0 tx=0).stop 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 shutdown_connections 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205ce93430 0x56205ce95840 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce83570 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69c5ffb700 0 client.16851 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/3481033449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce89820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 >> 172.21.15.93:0/3481033449 conn(0x56205ce6fae0 msgr2=0x56205ce743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 shutdown_connections 2023-07-21T22:25:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 1 -- 172.21.15.93:0/3481033449 wait complete. 2023-07-21T22:25:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.977+0000 7f69f07f3b80 20 client.16851 trim_cache size 0 max 16384 2023-07-21T22:25:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.992+0000 7f69f07f3b80 1 Processor -- start 2023-07-21T22:25:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.992+0000 7f69f07f3b80 1 -- start start 2023-07-21T22:25:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.992+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce82450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cd25490 con 0x56205ce80660 2023-07-21T22:25:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ce75f80 con 0x56205ccd66b0 2023-07-21T22:25:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ce843a0 con 0x56205ce80210 2023-07-21T22:25:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47440/0 (socket says 172.21.15.93:47440) 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e1c09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce82450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 -- 172.21.15.93:0/1580695930 learned_addr learned my addr 172.21.15.93:0/1580695930 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e240a700 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 -- 172.21.15.93:0/1580695930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 msgr2=0x56205ce82450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce82450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 -- 172.21.15.93:0/1580695930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 msgr2=0x56205ce810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 -- 172.21.15.93:0/1580695930 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56205cbe2c00 con 0x56205ce80660 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e240a700 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69d4097a20 tx=0x7f69d4097420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f09d61e01a76d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69d17fa700 1 -- 172.21.15.93:0/1580695930 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40a3370 con 0x56205ce80660 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69d17fa700 1 -- 172.21.15.93:0/1580695930 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d4099ca0 con 0x56205ce80660 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.993+0000 7f69d17fa700 1 -- 172.21.15.93:0/1580695930 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4099470 con 0x56205ce80660 2023-07-21T22:25:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- 172.21.15.93:0/1580695930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 msgr2=0x56205ce7c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69d4097a20 tx=0x7f69d4097420 comp rx=0 tx=0).stop 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- 172.21.15.93:0/1580695930 shutdown_connections 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80660 0x56205ce7c740 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/1580695930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80210 0x56205ce82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- 172.21.15.93:0/1580695930 >> 172.21.15.93:0/1580695930 conn(0x56205ce6fae0 msgr2=0x56205ce81d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- 172.21.15.93:0/1580695930 shutdown_connections 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- 172.21.15.93:0/1580695930 wait complete. 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 -- start start 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.994+0000 7f69f07f3b80 1 Processor -- start 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce86f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80660 0x56205cee62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ce87c80 con 0x56205ce80210 2023-07-21T22:25:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ce87df0 con 0x56205ccd66b0 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205cee6860 con 0x56205ce80660 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69f07f3b80 10 client.0 _ll_register_callbacks cb 0x7f69d8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47444/0 (socket says 172.21.15.93:47444) 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 -- 172.21.15.93:0/2032369414 learned_addr learned my addr 172.21.15.93:0/2032369414 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e2c0b700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80660 0x56205cee62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e240a700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce86f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 -- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80660 msgr2=0x56205cee62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80660 0x56205cee62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 -- 172.21.15.93:0/2032369414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 msgr2=0x56205ce86f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce86f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 -- 172.21.15.93:0/2032369414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc1a4d50 con 0x56205ce80210 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e240a700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce86f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69e1c09700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0733d0 tx=0x7f69d800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9b089bb0396399 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.995+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8017070 con 0x56205ce80210 2023-07-21T22:25:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f69d800fe00 con 0x56205ce80210 2023-07-21T22:25:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69d40054c0 con 0x56205ce80210 2023-07-21T22:25:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d8004050 con 0x56205ce80210 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 10 client.16857 fetch_fsmap learned FSMap version 54 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x56205cc201f0 con 0x56205ce80210 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d40054c0 con 0x56205ce80210 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 54) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f69d801d440 con 0x56205ce80210 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 10 client.16857 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 20 client.16857 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205cee7ff0 0x56205ceea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 4 client.16857 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56205cd86ff0 con 0x56205cee7ff0 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.996+0000 7f69e240a700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205cee7ff0 0x56205ceea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.997+0000 7f69c5ffb700 10 client.16857 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:28.997+0000 7f69e240a700 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205cee7ff0 0x56205ceea400 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f346853f6c307a server_cookie=fa151f62192e4ca2 in_seq=0 out_seq=0 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69c5ffb700 1 -- 172.21.15.93:0/2032369414 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56205cd86ff0 con 0x56205cee7ff0 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69c5ffb700 10 client.16857 handle_command_reply: tid=0 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 client.16857 shutdown 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 msgr2=0x56205ce874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0733d0 tx=0x7f69d800f6a0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 shutdown_connections 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56205cee7ff0 0x56205ceea400 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56205ce80660 0x56205cee62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56205ccd66b0 0x56205ce86f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69c5ffb700 0 client.16857 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 --2- 172.21.15.93:0/2032369414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56205ce80210 0x56205ce874d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 >> 172.21.15.93:0/2032369414 conn(0x56205ce6fae0 msgr2=0x56205ce73b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 shutdown_connections 2023-07-21T22:25:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 1 -- 172.21.15.93:0/2032369414 wait complete. 2023-07-21T22:25:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.035+0000 7f69f07f3b80 20 client.16857 trim_cache size 0 max 16384 2023-07-21T22:25:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:25:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:25:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:25:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:25:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 msgr2=0x7f69c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 0x7f69c80a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005950 tx=0x7f69cc00b410 comp rx=0 tx=0).stop 2023-07-21T22:25:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 msgr2=0x7f69dc1a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f69d4007870 tx=0x7f69d40078a0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 shutdown_connections 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f69c80a1340 0x7f69c80a37f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69dc104e50 0x7f69dc1a3ec0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69dc067c10 0x7f69dc0fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 --2- 172.21.15.93:0/2611824319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f69dc0685b0 0x7f69dc0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 >> 172.21.15.93:0/2611824319 conn(0x7f69dc074760 msgr2=0x7f69dc1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 shutdown_connections 2023-07-21T22:25:29.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:29.036+0000 7f69e466e700 1 -- 172.21.15.93:0/2611824319 wait complete. 2023-07-21T22:25:29.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 457941 5.54 MB/sec execute 366 sec latency 25.872 ms 2023-07-21T22:25:29.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:29 smithi194 ceph-mon[98288]: pgmap v284: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2023-07-21T22:25:29.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:29 smithi134 ceph-mon[97873]: pgmap v284: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2023-07-21T22:25:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:29 smithi093 ceph-mon[96579]: pgmap v284: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 232 op/s 2023-07-21T22:25:30.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 458749 5.56 MB/sec execute 367 sec latency 59.570 ms 2023-07-21T22:25:31.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 460482 5.58 MB/sec execute 368 sec latency 38.534 ms 2023-07-21T22:25:31.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:31 smithi194 ceph-mon[98288]: pgmap v285: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2023-07-21T22:25:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:31 smithi134 ceph-mon[97873]: pgmap v285: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2023-07-21T22:25:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:31 smithi093 ceph-mon[96579]: pgmap v285: 97 pgs: 97 active+clean; 193 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2023-07-21T22:25:32.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 462192 5.60 MB/sec execute 369 sec latency 60.391 ms 2023-07-21T22:25:33.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 463598 5.60 MB/sec execute 370 sec latency 30.396 ms 2023-07-21T22:25:33.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:33 smithi194 ceph-mon[98288]: pgmap v286: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-21T22:25:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:33 smithi134 ceph-mon[97873]: pgmap v286: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-21T22:25:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:33 smithi093 ceph-mon[96579]: pgmap v286: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2023-07-21T22:25:34.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 465724 5.62 MB/sec execute 371 sec latency 13.619 ms 2023-07-21T22:25:35.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 467150 5.62 MB/sec execute 372 sec latency 16.739 ms 2023-07-21T22:25:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:35 smithi194 ceph-mon[98288]: pgmap v287: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:35.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:35 smithi134 ceph-mon[97873]: pgmap v287: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:35 smithi093 ceph-mon[96579]: pgmap v287: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:36.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 469262 5.64 MB/sec execute 373 sec latency 36.999 ms 2023-07-21T22:25:37.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 470552 5.64 MB/sec execute 374 sec latency 27.709 ms 2023-07-21T22:25:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:37 smithi194 ceph-mon[98288]: pgmap v288: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:37.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:37 smithi134 ceph-mon[97873]: pgmap v288: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:37.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:37 smithi093 ceph-mon[96579]: pgmap v288: 97 pgs: 97 active+clean; 193 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T22:25:38.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 472624 5.66 MB/sec execute 375 sec latency 40.465 ms 2023-07-21T22:25:38.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:38.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:38.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:25:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:25:38.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:25:39.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 473974 5.66 MB/sec execute 376 sec latency 20.355 ms 2023-07-21T22:25:39.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:39 smithi194 ceph-mon[98288]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2023-07-21T22:25:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:39 smithi134 ceph-mon[97873]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2023-07-21T22:25:39.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:39 smithi093 ceph-mon[96579]: pgmap v289: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2023-07-21T22:25:40.337 INFO:tasks.workunit.client.0.smithi093.stdout: 1 476194 5.68 MB/sec execute 377 sec latency 14.250 ms 2023-07-21T22:25:41.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 477345 5.68 MB/sec execute 378 sec latency 46.857 ms 2023-07-21T22:25:41.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:41 smithi194 ceph-mon[98288]: pgmap v290: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2023-07-21T22:25:41.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:41 smithi134 ceph-mon[97873]: pgmap v290: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2023-07-21T22:25:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:41 smithi093 ceph-mon[96579]: pgmap v290: 97 pgs: 97 active+clean; 204 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2023-07-21T22:25:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 -- 172.21.15.93:0/961701733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c104ad0 msgr2=0x7fce4c104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 --2- 172.21.15.93:0/961701733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c104ad0 0x7fce4c104ea0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 -- 172.21.15.93:0/961701733 shutdown_connections 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 --2- 172.21.15.93:0/961701733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0ff470 0x7fce4c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 --2- 172.21.15.93:0/961701733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c104ad0 0x7fce4c104ea0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 --2- 172.21.15.93:0/961701733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c0feac0 0x7fce4c0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 -- 172.21.15.93:0/961701733 >> 172.21.15.93:0/961701733 conn(0x7fce4c075bd0 msgr2=0x7fce4c075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 -- 172.21.15.93:0/961701733 shutdown_connections 2023-07-21T22:25:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.160+0000 7fce51930700 1 -- 172.21.15.93:0/961701733 wait complete. 2023-07-21T22:25:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.161+0000 7fce51930700 1 Processor -- start 2023-07-21T22:25:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.161+0000 7fce51930700 1 -- start start 2023-07-21T22:25:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0feac0 0x7fce4c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c104ad0 0x7fce4c1948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c194f50 con 0x7fce4c0ff470 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1950c0 con 0x7fce4c0feac0 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce51930700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c195280 con 0x7fce4c104ad0 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47456/0 (socket says 172.21.15.93:47456) 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0feac0 0x7fce4c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 -- 172.21.15.93:0/1137628527 learned_addr learned my addr 172.21.15.93:0/1137628527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c104ad0 0x7fce4c1948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 -- 172.21.15.93:0/1137628527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c104ad0 msgr2=0x7fce4c1948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c104ad0 0x7fce4c1948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 -- 172.21.15.93:0/1137628527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0feac0 msgr2=0x7fce4c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0feac0 0x7fce4c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 -- 172.21.15.93:0/1137628527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0095f0 con 0x7fce4c0ff470 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.162+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fce4000d950 tx=0x7fce4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.163+0000 7fce5092e700 1 -- 172.21.15.93:0/1137628527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000f890 con 0x7fce4c0ff470 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.163+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c195560 con 0x7fce4c0ff470 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.163+0000 7fce5092e700 1 -- 172.21.15.93:0/1137628527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce40010460 con 0x7fce4c0ff470 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.163+0000 7fce5092e700 1 -- 172.21.15.93:0/1137628527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000e610 con 0x7fce4c0ff470 2023-07-21T22:25:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.163+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c1a1a70 con 0x7fce4c0ff470 2023-07-21T22:25:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.165+0000 7fce5092e700 1 -- 172.21.15.93:0/1137628527 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fce400108d0 con 0x7fce4c0ff470 2023-07-21T22:25:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5092e700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 0x7fce340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5092e700 1 -- 172.21.15.93:0/1137628527 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce400ccae0 con 0x7fce4c0ff470 2023-07-21T22:25:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce4affd700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 0x7fce340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5dab5b80 1 Processor -- start 2023-07-21T22:25:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5dab5b80 1 -- start start 2023-07-21T22:25:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce4affd700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 0x7fce340a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b5c0 tx=0x7fce3c005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.166+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 0x5586f0f436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0e46c50 con 0x5586f0f49420 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0e999e0 con 0x5586f0f49870 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0e92e70 con 0x5586f0d9d6b0 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34542/0 (socket says 172.21.15.93:34542) 2023-07-21T22:25:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47464/0 (socket says 172.21.15.93:47464) 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 -- 172.21.15.93:0/3169508584 learned_addr learned my addr 172.21.15.93:0/3169508584 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 0x5586f0f436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 -- 172.21.15.93:0/3169508584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 msgr2=0x5586f0f436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 0x5586f0f436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 -- 172.21.15.93:0/3169508584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 msgr2=0x5586f0f43150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 -- 172.21.15.93:0/3169508584 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586f0cc0d80 con 0x5586f0d9d6b0 2023-07-21T22:25:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 0x5586f0f436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.167+0000 7fce4affd700 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce3c003730 tx=0x7fce3c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aca24d46f49291f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.170+0000 7fce31ffb700 1 -- 172.21.15.93:0/3169508584 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c02a070 con 0x5586f0d9d6b0 2023-07-21T22:25:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.170+0000 7fce31ffb700 1 -- 172.21.15.93:0/3169508584 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c01e040 con 0x5586f0d9d6b0 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.170+0000 7fce31ffb700 1 -- 172.21.15.93:0/3169508584 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c023800 con 0x5586f0d9d6b0 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3169508584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 msgr2=0x5586f0f48ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce3c003730 tx=0x7fce3c020000 comp rx=0 tx=0).stop 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3169508584 shutdown_connections 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49870 0x5586f0f436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0d9d6b0 0x5586f0f48ee0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3169508584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f43150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3169508584 >> 172.21.15.93:0/3169508584 conn(0x5586f0f36ae0 msgr2=0x5586f0f3c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3169508584 shutdown_connections 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3169508584 wait complete. 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.171+0000 7fce5dab5b80 1 -- start start 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 Processor -- start 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f4c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0e32a60 con 0x5586f0f49420 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0e32760 con 0x5586f0d9d6b0 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0d168c0 con 0x5586f0f49870 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce5dab5b80 10 client.0 _ll_register_callbacks cb 0x5586f0e30700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f4c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f52730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58630/0 (socket says 172.21.15.93:58630) 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4affd700 1 -- 172.21.15.93:0/1125294919 learned_addr learned my addr 172.21.15.93:0/1125294919 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47470/0 (socket says 172.21.15.93:47470) 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 -- 172.21.15.93:0/1125294919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 msgr2=0x5586f0f52730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f52730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 -- 172.21.15.93:0/1125294919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 msgr2=0x5586f0f52c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 -- 172.21.15.93:0/1125294919 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c1a1a70 con 0x5586f0f49870 2023-07-21T22:25:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f4c9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce4c19b780 tx=0x7fce44004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199b4e055fd1ba02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.172+0000 7fce2affd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:25:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44007910 con 0x5586f0f49870 2023-07-21T22:25:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce44007a70 con 0x5586f0f49870 2023-07-21T22:25:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586f0dec490 con 0x5586f0f49870 2023-07-21T22:25:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce44007c30 con 0x5586f0f49870 2023-07-21T22:25:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 10 client.26218 fetch_fsmap learned FSMap version 54 2023-07-21T22:25:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7fce44003b50 con 0x5586f0f49870 2023-07-21T22:25:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586f0dec490 con 0x5586f0f49870 2023-07-21T22:25:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 54) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fce4401e3b0 con 0x5586f0f49870 2023-07-21T22:25:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 10 client.26218 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T22:25:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 10 client.26218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 20 client.26218 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0f5cca0 0x5586f0f5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 4 client.26218 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586f0e8be50 con 0x5586f0f5cca0 2023-07-21T22:25:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.173+0000 7fce4affd700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0f5cca0 0x5586f0f5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.174+0000 7fce2affd700 10 client.26218 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.174+0000 7fce4affd700 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0f5cca0 0x5586f0f5f0b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38a960510da69c server_cookie=5d50329c0315db68 in_seq=0 out_seq=0 2023-07-21T22:25:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.174+0000 7fce2affd700 1 -- 172.21.15.93:0/1125294919 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5586f0e8be50 con 0x5586f0f5cca0 2023-07-21T22:25:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.174+0000 7fce2affd700 10 client.26218 handle_command_reply: tid=0 2023-07-21T22:25:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.174+0000 7fce5dab5b80 1 client.26218 shutdown 2023-07-21T22:25:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 msgr2=0x5586f0f4c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f4c9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce4c19b780 tx=0x7fce44004270 comp rx=0 tx=0).stop 2023-07-21T22:25:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 shutdown_connections 2023-07-21T22:25:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0f5cca0 0x5586f0f5f0b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f4c9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f52730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce2affd700 0 client.26218 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/1125294919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49420 0x5586f0f52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 >> 172.21.15.93:0/1125294919 conn(0x5586f0f36ae0 msgr2=0x5586f0f3b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 shutdown_connections 2023-07-21T22:25:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 1 -- 172.21.15.93:0/1125294919 wait complete. 2023-07-21T22:25:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.175+0000 7fce5dab5b80 20 client.26218 trim_cache size 0 max 16384 2023-07-21T22:25:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 Processor -- start 2023-07-21T22:25:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 -- start start 2023-07-21T22:25:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49420 0x5586f0f3bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f3c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0f49290 con 0x5586f0d9d6b0 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0f33b20 con 0x5586f0f49420 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0f44370 con 0x5586f0f49870 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47476/0 (socket says 172.21.15.93:47476) 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 -- 172.21.15.93:0/171419653 learned_addr learned my addr 172.21.15.93:0/171419653 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f3c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49420 0x5586f0f3bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 -- 172.21.15.93:0/171419653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 msgr2=0x5586f0f3c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f3c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 -- 172.21.15.93:0/171419653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49420 msgr2=0x5586f0f3bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49420 0x5586f0f3bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.193+0000 7fce4affd700 1 -- 172.21.15.93:0/171419653 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c10c200 con 0x5586f0d9d6b0 2023-07-21T22:25:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.194+0000 7fce4affd700 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fce3c005b00 tx=0x7fce3c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebac080f27ddc536 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.194+0000 7fce327fc700 1 -- 172.21.15.93:0/171419653 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c02a040 con 0x5586f0d9d6b0 2023-07-21T22:25:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.194+0000 7fce327fc700 1 -- 172.21.15.93:0/171419653 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c023920 con 0x5586f0d9d6b0 2023-07-21T22:25:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.194+0000 7fce327fc700 1 -- 172.21.15.93:0/171419653 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c005320 con 0x5586f0d9d6b0 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.194+0000 7fce5dab5b80 1 -- 172.21.15.93:0/171419653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 msgr2=0x5586f0f4a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fce3c005b00 tx=0x7fce3c003ed0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- 172.21.15.93:0/171419653 shutdown_connections 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49870 0x5586f0f3c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0d9d6b0 0x5586f0f4a4d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/171419653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0f49420 0x5586f0f3bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- 172.21.15.93:0/171419653 >> 172.21.15.93:0/171419653 conn(0x5586f0f36ae0 msgr2=0x5586f0f4c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- 172.21.15.93:0/171419653 shutdown_connections 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- 172.21.15.93:0/171419653 wait complete. 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- start start 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 Processor -- start 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 0x5586f0f50820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0fafc90 con 0x5586f0f49870 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0fafe00 con 0x5586f0d9d6b0 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.195+0000 7fce5dab5b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586f0faffc0 con 0x5586f0f49420 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce5dab5b80 10 client.0 _ll_register_callbacks cb 0x7fce44009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47484/0 (socket says 172.21.15.93:47484) 2023-07-21T22:25:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4a7fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 0x5586f0f50820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 -- 172.21.15.93:0/3488404977 learned_addr learned my addr 172.21.15.93:0/3488404977 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4affd700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 -- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 msgr2=0x5586f0f50820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 0x5586f0f50820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 -- 172.21.15.93:0/3488404977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 msgr2=0x5586f0f502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 -- 172.21.15.93:0/3488404977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586f0cc0d80 con 0x5586f0f49870 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4a7fc700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 0x5586f0f50820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4affd700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce4b7fe700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fce4c19b780 tx=0x7fce44005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e052711d07ffc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce2affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4401e5c0 con 0x5586f0f49870 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fce44009230 con 0x5586f0f49870 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce3c01e040 con 0x5586f0f49870 2023-07-21T22:25:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4401ec20 con 0x5586f0f49870 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce5dab5b80 10 client.16875 fetch_fsmap learned FSMap version 54 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.196+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5586f0ca9c00 con 0x5586f0f49870 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce3c01e040 con 0x5586f0f49870 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 54) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fce44033560 con 0x5586f0f49870 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 20 client.16875 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0fb1580 0x5586f0fb3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 4 client.16875 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586f0e4dff0 con 0x5586f0fb1580 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.197+0000 7fce4affd700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0fb1580 0x5586f0fb3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.198+0000 7fce2affd700 10 client.16875 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.198+0000 7fce4affd700 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0fb1580 0x5586f0fb3990 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c8abfd88082b90 server_cookie=30711c8de24fa009 in_seq=0 out_seq=0 2023-07-21T22:25:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.239+0000 7fce2affd700 1 -- 172.21.15.93:0/3488404977 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586f0e4dff0 con 0x5586f0fb1580 2023-07-21T22:25:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.239+0000 7fce2affd700 10 client.16875 handle_command_reply: tid=0 2023-07-21T22:25:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.239+0000 7fce5dab5b80 1 client.16875 shutdown 2023-07-21T22:25:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 msgr2=0x5586f0f50d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fce4c19b780 tx=0x7fce44005b90 comp rx=0 tx=0).stop 2023-07-21T22:25:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 shutdown_connections 2023-07-21T22:25:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5586f0fb1580 0x5586f0fb3990 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5586f0f49870 0x5586f0f50d60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586f0d9d6b0 0x5586f0f502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 --2- 172.21.15.93:0/3488404977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5586f0f49420 0x5586f0f50820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 >> 172.21.15.93:0/3488404977 conn(0x5586f0f36ae0 msgr2=0x5586f0f4b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.240+0000 7fce2affd700 0 client.16875 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.241+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 shutdown_connections 2023-07-21T22:25:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.241+0000 7fce5dab5b80 1 -- 172.21.15.93:0/3488404977 wait complete. 2023-07-21T22:25:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.241+0000 7fce5dab5b80 20 client.16875 trim_cache size 0 max 16384 2023-07-21T22:25:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:25:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:25:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:25:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:25:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 msgr2=0x7fce340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 0x7fce340a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b5c0 tx=0x7fce3c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 msgr2=0x7fce4c19afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fce4000d950 tx=0x7fce4000dc60 comp rx=0 tx=0).stop 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 shutdown_connections 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce4c104ad0 0x7fce4c1948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce4c0feac0 0x7fce4c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce340a12c0 0x7fce340a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 --2- 172.21.15.93:0/1137628527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce4c0ff470 0x7fce4c19afc0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 >> 172.21.15.93:0/1137628527 conn(0x7fce4c075bd0 msgr2=0x7fce4c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 shutdown_connections 2023-07-21T22:25:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:42.242+0000 7fce51930700 1 -- 172.21.15.93:0/1137628527 wait complete. 2023-07-21T22:25:42.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 479336 5.69 MB/sec execute 379 sec latency 31.698 ms 2023-07-21T22:25:43.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 480741 5.70 MB/sec execute 380 sec latency 15.859 ms 2023-07-21T22:25:43.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:43 smithi194 ceph-mon[98288]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-21T22:25:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:43 smithi134 ceph-mon[97873]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-21T22:25:43.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:43 smithi093 ceph-mon[96579]: pgmap v291: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-21T22:25:44.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 482856 5.71 MB/sec execute 381 sec latency 17.381 ms 2023-07-21T22:25:45.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 484311 5.72 MB/sec execute 382 sec latency 12.157 ms 2023-07-21T22:25:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:45 smithi194 ceph-mon[98288]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:45.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:45 smithi134 ceph-mon[97873]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:45.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:45 smithi093 ceph-mon[96579]: pgmap v292: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:46.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 486424 5.73 MB/sec execute 383 sec latency 21.535 ms 2023-07-21T22:25:47.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 487740 5.73 MB/sec execute 384 sec latency 23.946 ms 2023-07-21T22:25:47.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:47 smithi194 ceph-mon[98288]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:47.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:47 smithi134 ceph-mon[97873]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:47.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:47 smithi093 ceph-mon[96579]: pgmap v293: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2023-07-21T22:25:48.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 489734 5.75 MB/sec execute 385 sec latency 41.143 ms 2023-07-21T22:25:48.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:48 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:48.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:48 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:48.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:48 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:25:49.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 491098 5.75 MB/sec execute 386 sec latency 32.196 ms 2023-07-21T22:25:49.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:49 smithi194 ceph-mon[98288]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2023-07-21T22:25:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:49 smithi134 ceph-mon[97873]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2023-07-21T22:25:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:49 smithi093 ceph-mon[96579]: pgmap v294: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 351 op/s 2023-07-21T22:25:50.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 492860 5.76 MB/sec execute 387 sec latency 115.934 ms 2023-07-21T22:25:51.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 494471 5.77 MB/sec execute 388 sec latency 10.603 ms 2023-07-21T22:25:51.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:51 smithi194 ceph-mon[98288]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-21T22:25:51.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:51 smithi134 ceph-mon[97873]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-21T22:25:51.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:51 smithi093 ceph-mon[96579]: pgmap v295: 97 pgs: 97 active+clean; 211 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2023-07-21T22:25:52.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 496282 5.77 MB/sec execute 389 sec latency 37.348 ms 2023-07-21T22:25:53.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 497861 5.79 MB/sec execute 390 sec latency 16.960 ms 2023-07-21T22:25:53.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:53 smithi194 ceph-mon[98288]: pgmap v296: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 352 op/s 2023-07-21T22:25:53.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:53 smithi134 ceph-mon[97873]: pgmap v296: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 352 op/s 2023-07-21T22:25:53.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:53 smithi093 ceph-mon[96579]: pgmap v296: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 352 op/s 2023-07-21T22:25:54.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 499791 5.79 MB/sec execute 391 sec latency 27.609 ms 2023-07-21T22:25:55.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 501427 5.81 MB/sec execute 392 sec latency 13.119 ms 2023-07-21T22:25:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 -- 172.21.15.93:0/3006206539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88441016d0 msgr2=0x7f8844109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 --2- 172.21.15.93:0/3006206539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88441016d0 0x7f8844109bc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8838009990 tx=0x7f8838009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 -- 172.21.15.93:0/3006206539 shutdown_connections 2023-07-21T22:25:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 --2- 172.21.15.93:0/3006206539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f884410c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 --2- 172.21.15.93:0/3006206539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8844100dc0 0x7f8844101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 --2- 172.21.15.93:0/3006206539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88441016d0 0x7f8844109bc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 -- 172.21.15.93:0/3006206539 >> 172.21.15.93:0/3006206539 conn(0x7f88440fa590 msgr2=0x7f88440fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 -- 172.21.15.93:0/3006206539 shutdown_connections 2023-07-21T22:25:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.363+0000 7f88496b8700 1 -- 172.21.15.93:0/3006206539 wait complete. 2023-07-21T22:25:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 Processor -- start 2023-07-21T22:25:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 -- start start 2023-07-21T22:25:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 0x7f884419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f8844195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88441960a0 con 0x7f8844100dc0 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844196210 con 0x7f88441016d0 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.364+0000 7f88496b8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88441963d0 con 0x7f884410a100 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47494/0 (socket says 172.21.15.93:47494) 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 -- 172.21.15.93:0/2569836704 learned_addr learned my addr 172.21.15.93:0/2569836704 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88427fc700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 0x7f884419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 -- 172.21.15.93:0/2569836704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 msgr2=0x7f8844195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88437fe700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f8844195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f8844195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 -- 172.21.15.93:0/2569836704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 msgr2=0x7f884419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 0x7f884419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 -- 172.21.15.93:0/2569836704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380095f0 con 0x7f8844100dc0 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88427fc700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 0x7f884419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88437fe700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f8844195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f8842ffd700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f883400e9a0 tx=0x7f883400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f882bfff700 1 -- 172.21.15.93:0/2569836704 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834009790 con 0x7f8844100dc0 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f882bfff700 1 -- 172.21.15.93:0/2569836704 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883400cbd0 con 0x7f8844100dc0 2023-07-21T22:25:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88441966b0 con 0x7f8844100dc0 2023-07-21T22:25:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f882bfff700 1 -- 172.21.15.93:0/2569836704 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834018680 con 0x7f8844100dc0 2023-07-21T22:25:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.365+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88441a2bc0 con 0x7f8844100dc0 2023-07-21T22:25:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.367+0000 7f882bfff700 1 -- 172.21.15.93:0/2569836704 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f883400cd40 con 0x7f8844100dc0 2023-07-21T22:25:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.368+0000 7f882bfff700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 0x7f882c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.368+0000 7f88427fc700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 0x7f882c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.368+0000 7f882bfff700 1 -- 172.21.15.93:0/2569836704 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f8834014070 con 0x7f8844100dc0 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.368+0000 7f88427fc700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 0x7f882c0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8838006010 tx=0x7f883800b260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 Processor -- start 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 -- start start 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfa176b0 0x5628bfbc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1660 0x5628bfbbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfac0c50 con 0x5628bfbc1210 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfb139e0 con 0x5628bfbc1660 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f885583db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfb0ce70 con 0x5628bfa176b0 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47504/0 (socket says 172.21.15.93:47504) 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 -- 172.21.15.93:0/1559521313 learned_addr learned my addr 172.21.15.93:0/1559521313 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfa176b0 0x5628bfbc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88437fe700 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1660 0x5628bfbbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 -- 172.21.15.93:0/1559521313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfa176b0 msgr2=0x5628bfbc0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfa176b0 0x5628bfbc0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 -- 172.21.15.93:0/1559521313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1660 msgr2=0x5628bfbbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.369+0000 7f88427fc700 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1660 0x5628bfbbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.370+0000 7f88427fc700 1 -- 172.21.15.93:0/1559521313 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bf93ad80 con 0x5628bfbc1210 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.370+0000 7f88427fc700 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8838015000 tx=0x7f883801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bf904a0637f783 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.370+0000 7f88297fa700 1 -- 172.21.15.93:0/1559521313 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883802b070 con 0x5628bfbc1210 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.370+0000 7f88297fa700 1 -- 172.21.15.93:0/1559521313 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883801e040 con 0x5628bfbc1210 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.370+0000 7f88297fa700 1 -- 172.21.15.93:0/1559521313 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800f460 con 0x5628bfbc1210 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- 172.21.15.93:0/1559521313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 msgr2=0x5628bfbbaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8838015000 tx=0x7f883801f000 comp rx=0 tx=0).stop 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- 172.21.15.93:0/1559521313 shutdown_connections 2023-07-21T22:25:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1660 0x5628bfbbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfa176b0 0x5628bfbc0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 --2- 172.21.15.93:0/1559521313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1210 0x5628bfbbaf40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- 172.21.15.93:0/1559521313 >> 172.21.15.93:0/1559521313 conn(0x5628bfbb0ae0 msgr2=0x5628bfbb60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- 172.21.15.93:0/1559521313 shutdown_connections 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- 172.21.15.93:0/1559521313 wait complete. 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.371+0000 7f885583db80 1 -- start start 2023-07-21T22:25:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 Processor -- start 2023-07-21T22:25:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1210 0x5628bfbcaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1660 0x5628bfbc4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfaaca60 con 0x5628bfbc1660 2023-07-21T22:25:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfaac760 con 0x5628bfa176b0 2023-07-21T22:25:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bf9908c0 con 0x5628bfbc1210 2023-07-21T22:25:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f885583db80 10 client.0 _ll_register_callbacks cb 0x5628bfaaa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58670/0 (socket says 172.21.15.93:58670) 2023-07-21T22:25:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 -- 172.21.15.93:0/1338597513 learned_addr learned my addr 172.21.15.93:0/1338597513 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f88427fc700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1210 0x5628bfbcaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f88437fe700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1660 0x5628bfbc4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 -- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1210 msgr2=0x5628bfbcaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1210 0x5628bfbcaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 -- 172.21.15.93:0/1338597513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1660 msgr2=0x5628bfbc4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1660 0x5628bfbc4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.372+0000 7f8842ffd700 1 -- 172.21.15.93:0/1338597513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884410d250 con 0x5628bfa176b0 2023-07-21T22:25:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f883400e970 tx=0x7f883409e290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f503ed3e558d5d07 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:25:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340a2130 con 0x5628bfa176b0 2023-07-21T22:25:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f88340a2290 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628bfa66490 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8834095f60 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 10 client.26288 fetch_fsmap learned FSMap version 55 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5628bf923c00 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628bfa66490 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f88340b8420 con 0x5628bfa176b0 2023-07-21T22:25:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 10 client.26288 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 20 client.26288 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfbd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 4 client.26288 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.373+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628bfb05e50 con 0x5628bfbd49c0 2023-07-21T22:25:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f88427fc700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfbd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f8826ff5700 10 client.26288 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f88427fc700 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfbd6da0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0e4fe70d075366 server_cookie=7de4a1bfe2ddcae1 in_seq=0 out_seq=0 2023-07-21T22:25:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f8826ff5700 1 -- 172.21.15.93:0/1338597513 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5628bfb05e50 con 0x5628bfbd49c0 2023-07-21T22:25:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f8826ff5700 10 client.26288 handle_command_reply: tid=0 2023-07-21T22:25:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.374+0000 7f885583db80 1 client.26288 shutdown 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 msgr2=0x5628bfbca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f883400e970 tx=0x7f883409e290 comp rx=0 tx=0).stop 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 shutdown_connections 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfbd6da0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfbc1660 0x5628bfbc4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f8826ff5700 0 client.26288 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfa176b0 0x5628bfbca540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 --2- 172.21.15.93:0/1338597513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1210 0x5628bfbcaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 >> 172.21.15.93:0/1338597513 conn(0x5628bfbb0ae0 msgr2=0x5628bfbb53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 shutdown_connections 2023-07-21T22:25:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 1 -- 172.21.15.93:0/1338597513 wait complete. 2023-07-21T22:25:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.375+0000 7f885583db80 20 client.26288 trim_cache size 0 max 16384 2023-07-21T22:25:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 Processor -- start 2023-07-21T22:25:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 -- start start 2023-07-21T22:25:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbb5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfbb60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfbbbc40 con 0x5628bfa176b0 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfbc0cd0 con 0x5628bfbc1210 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f885583db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfbc2d30 con 0x5628bfbc1660 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47516/0 (socket says 172.21.15.93:47516) 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f8842ffd700 1 -- 172.21.15.93:0/3889537108 learned_addr learned my addr 172.21.15.93:0/3889537108 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f88427fc700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbb5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.390+0000 7f88437fe700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfbb60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 -- 172.21.15.93:0/3889537108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 msgr2=0x5628bfbb60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfbb60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 -- 172.21.15.93:0/3889537108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 msgr2=0x5628bfbb5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbb5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 -- 172.21.15.93:0/3889537108 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628bf9611f0 con 0x5628bfa176b0 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f88437fe700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfbb60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8842ffd700 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f88340cadf0 tx=0x7f883409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f965dd02378dfc2e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8829ffb700 1 -- 172.21.15.93:0/3889537108 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883409b0a0 con 0x5628bfa176b0 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8829ffb700 1 -- 172.21.15.93:0/3889537108 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883409b290 con 0x5628bfa176b0 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.391+0000 7f8829ffb700 1 -- 172.21.15.93:0/3889537108 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88340b2db0 con 0x5628bfa176b0 2023-07-21T22:25:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- 172.21.15.93:0/3889537108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 msgr2=0x5628bfbc2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f88340cadf0 tx=0x7f883409a070 comp rx=0 tx=0).stop 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- 172.21.15.93:0/3889537108 shutdown_connections 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfbb60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc2330 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- 172.21.15.93:0/3889537108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbb5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- 172.21.15.93:0/3889537108 >> 172.21.15.93:0/3889537108 conn(0x5628bfbb0ae0 msgr2=0x5628bfbc46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- 172.21.15.93:0/3889537108 shutdown_connections 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- 172.21.15.93:0/3889537108 wait complete. 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- start start 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 Processor -- start 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbc8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfc27850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfc27dc0 con 0x5628bfa176b0 2023-07-21T22:25:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfc27f30 con 0x5628bfbc1210 2023-07-21T22:25:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628bfc280f0 con 0x5628bfbc1660 2023-07-21T22:25:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f885583db80 10 client.0 _ll_register_callbacks cb 0x7f883409a170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.392+0000 7f8842ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47522/0 (socket says 172.21.15.93:47522) 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 -- 172.21.15.93:0/1267428949 learned_addr learned my addr 172.21.15.93:0/1267428949 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f88427fc700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbc8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f88437fe700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfc27850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 -- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 msgr2=0x5628bfc27850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfc27850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 -- 172.21.15.93:0/1267428949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 msgr2=0x5628bfbc8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbc8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 -- 172.21.15.93:0/1267428949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628bf923c00 con 0x5628bfa176b0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8842ffd700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f88340b4050 tx=0x7f88340a2700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9921a10c7c8c41 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8826ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883409b400 con 0x5628bfa176b0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f883409b0a0 con 0x5628bfa176b0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f883801e040 con 0x5628bfa176b0 2023-07-21T22:25:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8834062080 con 0x5628bfa176b0 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f885583db80 10 client.16893 fetch_fsmap learned FSMap version 55 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x5628bf939250 con 0x5628bfa176b0 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.393+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883801e040 con 0x5628bfa176b0 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f88340b8380 con 0x5628bfa176b0 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 10 client.16893 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:25:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 20 client.16893 populate_metadata read hostname 'smithi093' 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfc2b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 4 client.16893 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628bfac7ff0 con 0x5628bfbd49c0 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f88427fc700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfc2b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f8826ff5700 10 client.16893 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.394+0000 7f88427fc700 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfc2b320 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69b1a3d8746ccf5 server_cookie=4a56c107e1df4b0 in_seq=0 out_seq=0 2023-07-21T22:25:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f8826ff5700 1 -- 172.21.15.93:0/1267428949 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628bfac7ff0 con 0x5628bfbd49c0 2023-07-21T22:25:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f8826ff5700 10 client.16893 handle_command_reply: tid=0 2023-07-21T22:25:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 client.16893 shutdown 2023-07-21T22:25:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 msgr2=0x5628bfbc86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f88340b4050 tx=0x7f88340a2700 comp rx=0 tx=0).stop 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 shutdown_connections 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5628bfbd49c0 0x5628bfc2b320 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628bfbc1660 0x5628bfc27850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f8826ff5700 0 client.16893 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5628bfa176b0 0x5628bfbc86f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 --2- 172.21.15.93:0/1267428949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628bfbc1210 0x5628bfbc8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 >> 172.21.15.93:0/1267428949 conn(0x5628bfbb0ae0 msgr2=0x5628bfbc2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 shutdown_connections 2023-07-21T22:25:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 1 -- 172.21.15.93:0/1267428949 wait complete. 2023-07-21T22:25:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.437+0000 7f885583db80 20 client.16893 trim_cache size 0 max 16384 2023-07-21T22:25:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:25:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:25:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:25:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 msgr2=0x7f882c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 0x7f882c0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8838006010 tx=0x7f883800b260 comp rx=0 tx=0).stop 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 msgr2=0x7f884419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f883400e9a0 tx=0x7f883400ed60 comp rx=0 tx=0).stop 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 shutdown_connections 2023-07-21T22:25:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884410a100 0x7f8844195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f882c0a12c0 0x7f882c0a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8844100dc0 0x7f884419bbd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 --2- 172.21.15.93:0/2569836704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88441016d0 0x7f884419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 >> 172.21.15.93:0/2569836704 conn(0x7f88440fa590 msgr2=0x7f884410c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 shutdown_connections 2023-07-21T22:25:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:25:55.438+0000 7f88496b8700 1 -- 172.21.15.93:0/2569836704 wait complete. 2023-07-21T22:25:55.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:55 smithi194 ceph-mon[98288]: pgmap v297: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:55 smithi134 ceph-mon[97873]: pgmap v297: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:55.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:55 smithi093 ceph-mon[96579]: pgmap v297: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:56.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 503372 5.81 MB/sec execute 393 sec latency 22.053 ms 2023-07-21T22:25:57.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 504891 5.83 MB/sec execute 394 sec latency 58.462 ms 2023-07-21T22:25:57.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:57 smithi194 ceph-mon[98288]: pgmap v298: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:57 smithi134 ceph-mon[97873]: pgmap v298: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:57.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:57 smithi093 ceph-mon[96579]: pgmap v298: 97 pgs: 97 active+clean; 217 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:25:58.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 506691 5.83 MB/sec execute 395 sec latency 21.723 ms 2023-07-21T22:25:59.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 508414 5.85 MB/sec execute 396 sec latency 36.040 ms 2023-07-21T22:25:59.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:25:59 smithi194 ceph-mon[98288]: pgmap v299: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2023-07-21T22:25:59.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:25:59 smithi134 ceph-mon[97873]: pgmap v299: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2023-07-21T22:25:59.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:25:59 smithi093 ceph-mon[96579]: pgmap v299: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 366 op/s 2023-07-21T22:26:00.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 509991 5.85 MB/sec execute 397 sec latency 18.445 ms 2023-07-21T22:26:01.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 511657 5.86 MB/sec execute 398 sec latency 22.360 ms 2023-07-21T22:26:01.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:01 smithi194 ceph-mon[98288]: pgmap v300: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:26:01.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:01 smithi134 ceph-mon[97873]: pgmap v300: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:26:01.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:01 smithi093 ceph-mon[96579]: pgmap v300: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 240 op/s 2023-07-21T22:26:02.338 INFO:tasks.workunit.client.0.smithi093.stdout: 1 512760 5.86 MB/sec execute 399 sec latency 49.811 ms 2023-07-21T22:26:03.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 514930 5.87 MB/sec execute 400 sec latency 20.610 ms 2023-07-21T22:26:03.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:03 smithi194 ceph-mon[98288]: pgmap v301: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2023-07-21T22:26:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:03 smithi134 ceph-mon[97873]: pgmap v301: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2023-07-21T22:26:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:03 smithi093 ceph-mon[96579]: pgmap v301: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.6 MiB/s wr, 387 op/s 2023-07-21T22:26:04.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 516105 5.87 MB/sec execute 401 sec latency 39.617 ms 2023-07-21T22:26:05.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 518139 5.89 MB/sec execute 402 sec latency 17.101 ms 2023-07-21T22:26:05.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:05 smithi194 ceph-mon[98288]: pgmap v302: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 272 op/s 2023-07-21T22:26:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:05 smithi194 ceph-mon[98288]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:26:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:05 smithi134 ceph-mon[97873]: pgmap v302: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 272 op/s 2023-07-21T22:26:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:05 smithi134 ceph-mon[97873]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:26:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:05 smithi093 ceph-mon[96579]: pgmap v302: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 272 op/s 2023-07-21T22:26:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:05 smithi093 ceph-mon[96579]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:26:06.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 519641 5.89 MB/sec execute 403 sec latency 13.396 ms 2023-07-21T22:26:07.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 521111 5.89 MB/sec execute 404 sec latency 262.025 ms 2023-07-21T22:26:07.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:07 smithi194 ceph-mon[98288]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-07-21T22:26:07.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:07 smithi134 ceph-mon[97873]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-07-21T22:26:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:07 smithi093 ceph-mon[96579]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 327 op/s 2023-07-21T22:26:08.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 522664 5.91 MB/sec execute 405 sec latency 13.895 ms 2023-07-21T22:26:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/3827137249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80fea30 msgr2=0x7ff6d80feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/3827137249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80fea30 0x7ff6d80feea0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009990 tx=0x7ff6c0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/3827137249 shutdown_connections 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/3827137249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 0x7ff6d810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/3827137249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d8104a30 0x7ff6d8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/3827137249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80fea30 0x7ff6d80feea0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/3827137249 >> 172.21.15.93:0/3827137249 conn(0x7ff6d8074760 msgr2=0x7ff6d8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.564+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/3827137249 shutdown_connections 2023-07-21T22:26:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.565+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/3827137249 wait complete. 2023-07-21T22:26:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.565+0000 7ff6ddc1b700 1 Processor -- start 2023-07-21T22:26:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.565+0000 7ff6ddc1b700 1 -- start start 2023-07-21T22:26:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 0x7ff6d8194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 0x7ff6d8194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8195730 con 0x7ff6d8104a30 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81a4480 con 0x7ff6d80fea30 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6ddc1b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81a4610 con 0x7ff6d80ff3e0 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47540/0 (socket says 172.21.15.93:47540) 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 -- 172.21.15.93:0/182109387 learned_addr learned my addr 172.21.15.93:0/182109387 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 0x7ff6d8194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 0x7ff6d8194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 -- 172.21.15.93:0/182109387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 msgr2=0x7ff6d8194fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 0x7ff6d8194fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 -- 172.21.15.93:0/182109387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 msgr2=0x7ff6d8194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 0x7ff6d8194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d7fff700 1 -- 172.21.15.93:0/182109387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00095f0 con 0x7ff6d8104a30 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.566+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 0x7ff6d8194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800ef30 tx=0x7ff6c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6d4ff9700 1 -- 172.21.15.93:0/182109387 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8018070 con 0x7ff6d8104a30 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6d4ff9700 1 -- 172.21.15.93:0/182109387 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c800f040 con 0x7ff6d8104a30 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6d4ff9700 1 -- 172.21.15.93:0/182109387 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800cd00 con 0x7ff6d8104a30 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d81a4890 con 0x7ff6d8104a30 2023-07-21T22:26:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.567+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81a4d50 con 0x7ff6d8104a30 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.569+0000 7ff6d4ff9700 1 -- 172.21.15.93:0/182109387 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff6c8003a00 con 0x7ff6d8104a30 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.570+0000 7ff6d4ff9700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 0x7ff6c40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.570+0000 7ff6d4ff9700 1 -- 172.21.15.93:0/182109387 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ff6c80cc7a0 con 0x7ff6d8104a30 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.570+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 0x7ff6c40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.570+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 0x7ff6c40a38c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80ffba0 tx=0x7ff6d0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 Processor -- start 2023-07-21T22:26:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 -- start start 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faed2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faed34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fadd8c50 con 0x55d9fad2f6b0 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fae2b9e0 con 0x55d9faed9210 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fae24e70 con 0x55d9faed9660 2023-07-21T22:26:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faed2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47544/0 (socket says 172.21.15.93:47544) 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6d77fe700 1 -- 172.21.15.93:0/3481823593 learned_addr learned my addr 172.21.15.93:0/3481823593 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.571+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faed34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 -- 172.21.15.93:0/3481823593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 msgr2=0x55d9faed34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faed34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 -- 172.21.15.93:0/3481823593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 msgr2=0x55d9faed2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faed2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 -- 172.21.15.93:0/3481823593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9fac52d80 con 0x55d9fad2f6b0 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faed2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e720 tx=0x7ff6d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3760a62a7af66d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6cdffb700 1 -- 172.21.15.93:0/3481823593 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0026600 con 0x55d9fad2f6b0 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6cdffb700 1 -- 172.21.15.93:0/3481823593 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d0026c20 con 0x55d9fad2f6b0 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.572+0000 7ff6cdffb700 1 -- 172.21.15.93:0/3481823593 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001ec00 con 0x55d9fad2f6b0 2023-07-21T22:26:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/3481823593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 msgr2=0x55d9faed8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e720 tx=0x7ff6d0014000 comp rx=0 tx=0).stop 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/3481823593 shutdown_connections 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faed34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8cd0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/3481823593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faed2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/3481823593 >> 172.21.15.93:0/3481823593 conn(0x55d9faec8ae0 msgr2=0x55d9faece0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/3481823593 shutdown_connections 2023-07-21T22:26:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/3481823593 wait complete. 2023-07-21T22:26:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.573+0000 7ff6e9da0b80 1 -- start start 2023-07-21T22:26:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 Processor -- start 2023-07-21T22:26:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faedc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faedcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fadc4a60 con 0x55d9faed9660 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fadc4760 con 0x55d9faed9210 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faca88c0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6e9da0b80 10 client.0 _ll_register_callbacks cb 0x55d9fadc2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faedcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faedcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47554/0 (socket says 172.21.15.93:47554) 2023-07-21T22:26:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34630/0 (socket says 172.21.15.93:34630) 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d7fff700 1 -- 172.21.15.93:0/2655489651 learned_addr learned my addr 172.21.15.93:0/2655489651 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faedc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 -- 172.21.15.93:0/2655489651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 msgr2=0x55d9faedc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faedc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 -- 172.21.15.93:0/2655489651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 msgr2=0x55d9faedcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faedcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d77fe700 1 -- 172.21.15.93:0/2655489651 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d810c160 con 0x55d9fad2f6b0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.574+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faedc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ed10 tx=0x7ff6d001fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6781c66156d6f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002c070 con 0x55d9fad2f6b0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9fad7e490 con 0x55d9fad2f6b0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ff6d001e3f0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d00050c0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6e9da0b80 10 client.26251 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7ff6d0017070 con 0x55d9fad2f6b0 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.575+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d003d080 con 0x55d9fad2f6b0 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6d0024020 con 0x55d9fad2f6b0 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 10 client.26251 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 20 client.26251 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faeec9b0 0x55d9faeeed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 4 client.26251 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9fae1de50 con 0x55d9faeec9b0 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faeec9b0 0x55d9faeeed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6b6ffd700 10 client.26251 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.576+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faeec9b0 0x55d9faeeed90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6beea510519f70e server_cookie=387dc23aaebf134d in_seq=0 out_seq=0 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/2655489651 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9fae1de50 con 0x55d9faeec9b0 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6b6ffd700 10 client.26251 handle_command_reply: tid=0 2023-07-21T22:26:08.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 client.26251 shutdown 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 msgr2=0x55d9faedf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000ed10 tx=0x7ff6d001fd00 comp rx=0 tx=0).stop 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 shutdown_connections 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faeec9b0 0x55d9faeeed90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faedcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6b6ffd700 0 client.26251 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9fad2f6b0 0x55d9faedf630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/2655489651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9faed9210 0x55d9faedc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 >> 172.21.15.93:0/2655489651 conn(0x55d9faec8ae0 msgr2=0x55d9faecd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 shutdown_connections 2023-07-21T22:26:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/2655489651 wait complete. 2023-07-21T22:26:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.577+0000 7ff6e9da0b80 20 client.26251 trim_cache size 0 max 16384 2023-07-21T22:26:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 Processor -- start 2023-07-21T22:26:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 -- start start 2023-07-21T22:26:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faece0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faece5e0 con 0x55d9faed9210 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faed7530 con 0x55d9fad2f6b0 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faecf9d0 con 0x55d9faed9660 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47560/0 (socket says 172.21.15.93:47560) 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d6ffd700 1 -- 172.21.15.93:0/1708966219 learned_addr learned my addr 172.21.15.93:0/1708966219 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faece0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 -- 172.21.15.93:0/1708966219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 msgr2=0x55d9faece0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faece0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 -- 172.21.15.93:0/1708966219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 msgr2=0x55d9faecdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d77fe700 1 -- 172.21.15.93:0/1708966219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9fac3bc00 con 0x55d9fad2f6b0 2023-07-21T22:26:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.592+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d001f880 tx=0x7ff6d0009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c52d287d1ae3b15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6ce7fc700 1 -- 172.21.15.93:0/1708966219 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002c020 con 0x55d9fad2f6b0 2023-07-21T22:26:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6ce7fc700 1 -- 172.21.15.93:0/1708966219 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9fad7e490 con 0x55d9fad2f6b0 2023-07-21T22:26:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6ce7fc700 1 -- 172.21.15.93:0/1708966219 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00050b0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/1708966219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 msgr2=0x55d9faed8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6d001f880 tx=0x7ff6d0009370 comp rx=0 tx=0).stop 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/1708966219 shutdown_connections 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9660 0x55d9faece0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faed8050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/1708966219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9210 0x55d9faecdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/1708966219 >> 172.21.15.93:0/1708966219 conn(0x55d9faec8ae0 msgr2=0x55d9faed7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/1708966219 shutdown_connections 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.593+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/1708966219 wait complete. 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 -- start start 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 Processor -- start 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 0x55d9faee18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faf3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faee2120 con 0x55d9faed9660 2023-07-21T22:26:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faf3fd80 con 0x55d9fad2f6b0 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9faf3fec0 con 0x55d9faed9210 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6e9da0b80 10 client.0 _ll_register_callbacks cb 0x7ff6d0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6d6ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 0x55d9faee18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6d77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58726/0 (socket says 172.21.15.93:58726) 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6d77fe700 1 -- 172.21.15.93:0/4234557498 learned_addr learned my addr 172.21.15.93:0/4234557498 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.594+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faf3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 -- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 msgr2=0x55d9faee18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 0x55d9faee18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 -- 172.21.15.93:0/4234557498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 msgr2=0x55d9faf3f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faf3f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 -- 172.21.15.93:0/4234557498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d0017070 con 0x55d9fad2f6b0 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 0x55d9faee18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d7fff700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faf3f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6d77fe700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0026bf0 tx=0x7ff6d0011bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d876339d0e97b0e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002c030 con 0x55d9fad2f6b0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ff6d0003bc0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6d0026c20 con 0x55d9fad2f6b0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d0041070 con 0x55d9fad2f6b0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6e9da0b80 10 client.26312 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55d9fac791f0 con 0x55d9fad2f6b0 2023-07-21T22:26:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.595+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d0026c20 con 0x55d9fad2f6b0 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6d0024070 con 0x55d9fad2f6b0 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 20 client.26312 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faf41060 0x55d9faf43440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 4 client.26312 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:26:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9faddfff0 con 0x55d9faf41060 2023-07-21T22:26:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faf41060 0x55d9faf43440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6b6ffd700 10 client.26312 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.596+0000 7ff6d6ffd700 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faf41060 0x55d9faf43440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cf6695177555fa server_cookie=64ad569acd6dc7ac in_seq=0 out_seq=0 2023-07-21T22:26:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.638+0000 7ff6b6ffd700 1 -- 172.21.15.93:0/4234557498 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9faddfff0 con 0x55d9faf41060 2023-07-21T22:26:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.638+0000 7ff6b6ffd700 10 client.26312 handle_command_reply: tid=0 2023-07-21T22:26:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.638+0000 7ff6e9da0b80 1 client.26312 shutdown 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 msgr2=0x55d9faee13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0026bf0 tx=0x7ff6d0011bd0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 shutdown_connections 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d9faf41060 0x55d9faf43440 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d9faed9660 0x55d9faf3f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6b6ffd700 0 client.26312 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d9fad2f6b0 0x55d9faee13a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 --2- 172.21.15.93:0/4234557498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d9faed9210 0x55d9faee18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 >> 172.21.15.93:0/4234557498 conn(0x55d9faec8ae0 msgr2=0x55d9faed1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 shutdown_connections 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 1 -- 172.21.15.93:0/4234557498 wait complete. 2023-07-21T22:26:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.639+0000 7ff6e9da0b80 20 client.26312 trim_cache size 0 max 16384 2023-07-21T22:26:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:26:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:26:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:26:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 msgr2=0x7ff6c40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 0x7ff6c40a38c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80ffba0 tx=0x7ff6d0009260 comp rx=0 tx=0).stop 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 msgr2=0x7ff6d81a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800ef30 tx=0x7ff6c800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 shutdown_connections 2023-07-21T22:26:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff6c40a1410 0x7ff6c40a38c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d8104a30 0x7ff6d81a3f40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80fea30 0x7ff6d8194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 --2- 172.21.15.93:0/182109387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6d80ff3e0 0x7ff6d8194fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 >> 172.21.15.93:0/182109387 conn(0x7ff6d8074760 msgr2=0x7ff6d81025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 shutdown_connections 2023-07-21T22:26:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:08.640+0000 7ff6ddc1b700 1 -- 172.21.15.93:0/182109387 wait complete. 2023-07-21T22:26:09.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 524417 5.91 MB/sec execute 406 sec latency 29.798 ms 2023-07-21T22:26:09.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:09 smithi194 ceph-mon[98288]: pgmap v305: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:09 smithi134 ceph-mon[97873]: pgmap v305: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:09.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:09 smithi093 ceph-mon[96579]: pgmap v305: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:10.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 526164 5.92 MB/sec execute 407 sec latency 11.338 ms 2023-07-21T22:26:11.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 527625 5.92 MB/sec execute 408 sec latency 30.171 ms 2023-07-21T22:26:11.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:11 smithi194 ceph-mon[98288]: pgmap v306: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:11.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:11 smithi134 ceph-mon[97873]: pgmap v306: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:11 smithi093 ceph-mon[96579]: pgmap v306: 97 pgs: 97 active+clean; 247 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 306 op/s 2023-07-21T22:26:12.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 529581 5.94 MB/sec execute 409 sec latency 27.942 ms 2023-07-21T22:26:13.198 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:13 smithi093 ceph-mon[96579]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:13.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 530916 5.94 MB/sec execute 410 sec latency 19.109 ms 2023-07-21T22:26:13.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:13 smithi194 ceph-mon[98288]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:13 smithi194 ceph-mon[98288]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:26:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:13 smithi134 ceph-mon[97873]: pgmap v307: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:13 smithi134 ceph-mon[97873]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:26:14.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 533101 5.96 MB/sec execute 411 sec latency 16.313 ms 2023-07-21T22:26:15.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 534340 5.96 MB/sec execute 412 sec latency 18.381 ms 2023-07-21T22:26:15.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:15 smithi194 ceph-mon[98288]: pgmap v309: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2023-07-21T22:26:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:15 smithi134 ceph-mon[97873]: pgmap v309: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2023-07-21T22:26:15.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:13 smithi093 ceph-mon[96579]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:26:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:15 smithi093 ceph-mon[96579]: pgmap v309: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2023-07-21T22:26:16.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 536494 5.97 MB/sec execute 413 sec latency 31.881 ms 2023-07-21T22:26:17.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 537489 5.97 MB/sec execute 414 sec latency 30.072 ms 2023-07-21T22:26:17.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:17 smithi194 ceph-mon[98288]: pgmap v310: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:17.630 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:17 smithi134 ceph-mon[97873]: pgmap v310: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:17.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:17 smithi093 ceph-mon[96579]: pgmap v310: 97 pgs: 97 active+clean; 246 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2023-07-21T22:26:18.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 539416 5.98 MB/sec execute 415 sec latency 28.548 ms 2023-07-21T22:26:19.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 540770 5.98 MB/sec execute 416 sec latency 14.947 ms 2023-07-21T22:26:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:19 smithi194 ceph-mon[98288]: pgmap v311: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:19 smithi134 ceph-mon[97873]: pgmap v311: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:19 smithi093 ceph-mon[96579]: pgmap v311: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:20.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 542856 5.99 MB/sec execute 417 sec latency 17.137 ms 2023-07-21T22:26:21.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 544328 6.00 MB/sec execute 418 sec latency 14.365 ms 2023-07-21T22:26:21.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:21 smithi194 ceph-mon[98288]: pgmap v312: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:21.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:21 smithi134 ceph-mon[97873]: pgmap v312: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:21 smithi093 ceph-mon[96579]: pgmap v312: 97 pgs: 97 active+clean; 249 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2023-07-21T22:26:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/4074270136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 msgr2=0x7f4e280fdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/4074270136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e280fdff0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/4074270136 shutdown_connections 2023-07-21T22:26:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/4074270136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e280fec10 0x7f4e2810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/4074270136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e280fdff0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/4074270136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e28100e90 0x7f4e280fe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/4074270136 >> 172.21.15.93:0/4074270136 conn(0x7f4e28075f90 msgr2=0x7f4e28076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/4074270136 shutdown_connections 2023-07-21T22:26:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.755+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/4074270136 wait complete. 2023-07-21T22:26:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 Processor -- start 2023-07-21T22:26:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 -- start start 2023-07-21T22:26:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 0x7f4e28197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e28100e90 0x7f4e2819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819cfb0 con 0x7f4e28100a30 2023-07-21T22:26:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819d120 con 0x7f4e280fec10 2023-07-21T22:26:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.756+0000 7f4e2e6a6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819d2e0 con 0x7f4e28100e90 2023-07-21T22:26:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 0x7f4e28197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e2cc43700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e28100e90 0x7f4e2819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 0x7f4e28197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58738/0 (socket says 172.21.15.93:58738) 2023-07-21T22:26:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47578/0 (socket says 172.21.15.93:47578) 2023-07-21T22:26:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e27fff700 1 -- 172.21.15.93:0/1047546858 learned_addr learned my addr 172.21.15.93:0/1047546858 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 -- 172.21.15.93:0/1047546858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e28100e90 msgr2=0x7f4e2819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e28100e90 0x7f4e2819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 -- 172.21.15.93:0/1047546858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 msgr2=0x7f4e28197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 0x7f4e28197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 -- 172.21.15.93:0/1047546858 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180095f0 con 0x7f4e28100a30 2023-07-21T22:26:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e277fe700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4e281012e0 tx=0x7f4e1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e257fa700 1 -- 172.21.15.93:0/1047546858 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00f890 con 0x7f4e28100a30 2023-07-21T22:26:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e257fa700 1 -- 172.21.15.93:0/1047546858 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c010460 con 0x7f4e28100a30 2023-07-21T22:26:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2819d5c0 con 0x7f4e28100a30 2023-07-21T22:26:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e257fa700 1 -- 172.21.15.93:0/1047546858 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00e610 con 0x7f4e28100a30 2023-07-21T22:26:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.757+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e281a4e00 con 0x7f4e28100a30 2023-07-21T22:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.759+0000 7f4e257fa700 1 -- 172.21.15.93:0/1047546858 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e1c010890 con 0x7f4e28100a30 2023-07-21T22:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.760+0000 7f4e257fa700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 0x7f4e100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.760+0000 7f4e27fff700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 0x7f4e100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.760+0000 7f4e257fa700 1 -- 172.21.15.93:0/1047546858 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e1c0cc8a0 con 0x7f4e28100a30 2023-07-21T22:26:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e27fff700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 0x7f4e100a3870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800b5c0 tx=0x7f4e18009ee0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 Processor -- start 2023-07-21T22:26:21.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 -- start start 2023-07-21T22:26:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4de9fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de905c50 con 0x55e4dea06660 2023-07-21T22:26:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de9589e0 con 0x55e4de85c6b0 2023-07-21T22:26:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de951e70 con 0x55e4dea06210 2023-07-21T22:26:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e2cc43700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e2cc43700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47586/0 (socket says 172.21.15.93:47586) 2023-07-21T22:26:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4de9fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.761+0000 7f4e2cc43700 1 -- 172.21.15.93:0/3214059183 learned_addr learned my addr 172.21.15.93:0/3214059183 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 -- 172.21.15.93:0/3214059183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 msgr2=0x55e4de9fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4de9fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 -- 172.21.15.93:0/3214059183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 msgr2=0x55e4dea05cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea05cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 -- 172.21.15.93:0/3214059183 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4de77fd80 con 0x55e4dea06660 2023-07-21T22:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e27fff700 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4e20003fb0 tx=0x7f4e20009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9651d6cdd495e748 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e0e7fc700 1 -- 172.21.15.93:0/3214059183 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200046f0 con 0x55e4dea06660 2023-07-21T22:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e0e7fc700 1 -- 172.21.15.93:0/3214059183 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e2000f040 con 0x55e4dea06660 2023-07-21T22:26:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.762+0000 7f4e0e7fc700 1 -- 172.21.15.93:0/3214059183 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20015710 con 0x55e4dea06660 2023-07-21T22:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/3214059183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 msgr2=0x55e4dea004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4e20003fb0 tx=0x7f4e20009c20 comp rx=0 tx=0).stop 2023-07-21T22:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/3214059183 shutdown_connections 2023-07-21T22:26:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4dea06660 0x55e4dea004a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea05cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/3214059183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4de9fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/3214059183 >> 172.21.15.93:0/3214059183 conn(0x55e4de9f5ae0 msgr2=0x55e4de9fb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/3214059183 shutdown_connections 2023-07-21T22:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/3214059183 wait complete. 2023-07-21T22:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.763+0000 7f4e3a82bb80 1 -- start start 2023-07-21T22:26:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 Processor -- start 2023-07-21T22:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 0x55e4dea09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de8f1a60 con 0x55e4de85c6b0 2023-07-21T22:26:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de8f1760 con 0x55e4dea06660 2023-07-21T22:26:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de7d58c0 con 0x55e4dea06210 2023-07-21T22:26:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e3a82bb80 10 client.0 _ll_register_callbacks cb 0x55e4de8ef700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47588/0 (socket says 172.21.15.93:47588) 2023-07-21T22:26:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e27fff700 1 -- 172.21.15.93:0/4227465539 learned_addr learned my addr 172.21.15.93:0/4227465539 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 0x55e4dea09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.764+0000 7f4e277fe700 1 -- 172.21.15.93:0/4227465539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 msgr2=0x55e4dea09ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e277fe700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 0x55e4dea09ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e277fe700 1 -- 172.21.15.93:0/4227465539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 msgr2=0x55e4dea0c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e277fe700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e277fe700 1 -- 172.21.15.93:0/4227465539 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2810f3f0 con 0x55e4dea06210 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 0x55e4dea09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e27fff700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e277fe700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea098b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0c9f90 tx=0x7f4e1c099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea3bf71c65073e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e077fe700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:26:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c099d70 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e1c09d940 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4de8ab490 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1c0a1d60 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e3a82bb80 10 client.26269 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f4e20003c10 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.765+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c0c5020 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e1c095700 con 0x55e4dea06210 2023-07-21T22:26:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 10 client.26269 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 20 client.26269 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea199a0 0x55e4dea1bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 4 client.26269 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4de94ae50 con 0x55e4dea199a0 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e27fff700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea199a0 0x55e4dea1bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e077fe700 10 client.26269 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.766+0000 7f4e27fff700 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea199a0 0x55e4dea1bdb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce06df2b1258db4 server_cookie=636d5fcdb5a58822 in_seq=0 out_seq=0 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e077fe700 1 -- 172.21.15.93:0/4227465539 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e4de94ae50 con 0x55e4dea199a0 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e077fe700 10 client.26269 handle_command_reply: tid=0 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 client.26269 shutdown 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 msgr2=0x55e4dea098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea098b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0c9f90 tx=0x7f4e1c099910 comp rx=0 tx=0).stop 2023-07-21T22:26:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 shutdown_connections 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea199a0 0x55e4dea1bdb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06660 0x55e4dea09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4dea0c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/4227465539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea098b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 >> 172.21.15.93:0/4227465539 conn(0x55e4de9f5ae0 msgr2=0x55e4de9fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e077fe700 0 client.26269 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 shutdown_connections 2023-07-21T22:26:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/4227465539 wait complete. 2023-07-21T22:26:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.767+0000 7f4e3a82bb80 20 client.26269 trim_cache size 0 max 16384 2023-07-21T22:26:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 Processor -- start 2023-07-21T22:26:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 -- start start 2023-07-21T22:26:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4de9f5ae0 0x55e4dea10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06210 0x55e4dea05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4de9f7c00 con 0x55e4de85c6b0 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4dea0ae20 con 0x55e4dea06210 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4dea07e60 con 0x55e4de9f5ae0 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47598/0 (socket says 172.21.15.93:47598) 2023-07-21T22:26:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e277fe700 1 -- 172.21.15.93:0/709560754 learned_addr learned my addr 172.21.15.93:0/709560754 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.783+0000 7f4e27fff700 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4de9f5ae0 0x55e4dea10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06210 0x55e4dea05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 -- 172.21.15.93:0/709560754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4de9f5ae0 msgr2=0x55e4dea10b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4de9f5ae0 0x55e4dea10b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 -- 172.21.15.93:0/709560754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06210 msgr2=0x55e4dea05050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06210 0x55e4dea05050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 -- 172.21.15.93:0/709560754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4de768c00 con 0x55e4de85c6b0 2023-07-21T22:26:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e277fe700 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c096380 tx=0x7f4e1c0c9fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a406f1c9f7bef7d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e0effd700 1 -- 172.21.15.93:0/709560754 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c014020 con 0x55e4de85c6b0 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e0effd700 1 -- 172.21.15.93:0/709560754 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c0c5070 con 0x55e4de85c6b0 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.784+0000 7f4e0effd700 1 -- 172.21.15.93:0/709560754 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c099060 con 0x55e4de85c6b0 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/709560754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 msgr2=0x55e4de9fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c096380 tx=0x7f4e1c0c9fc0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/709560754 shutdown_connections 2023-07-21T22:26:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4dea06210 0x55e4dea05050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4de9f5ae0 0x55e4dea10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/709560754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de85c6b0 0x55e4de9fff40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/709560754 >> 172.21.15.93:0/709560754 conn(0x55e4dea16140 msgr2=0x55e4dea04940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/709560754 shutdown_connections 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/709560754 wait complete. 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 -- start start 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.785+0000 7f4e3a82bb80 1 Processor -- start 2023-07-21T22:26:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea0d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4dea0df60 con 0x55e4de9f5ae0 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4dea6cd90 con 0x55e4de85c6b0 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4dea6ced0 con 0x55e4dea06210 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e3a82bb80 10 client.0 _ll_register_callbacks cb 0x7f4e1c099270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47604/0 (socket says 172.21.15.93:47604) 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 -- 172.21.15.93:0/2931517269 learned_addr learned my addr 172.21.15.93:0/2931517269 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e2cc43700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e27fff700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea0d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 -- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 msgr2=0x55e4dea6c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea6c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 -- 172.21.15.93:0/2931517269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 msgr2=0x55e4dea0d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea0d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 -- 172.21.15.93:0/2931517269 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e20003c10 con 0x55e4de9f5ae0 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e27fff700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea0d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e277fe700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005950 tx=0x7f4e1c099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a614a27f9120eee7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.786+0000 7f4e077fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c014030 con 0x55e4de9f5ae0 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4e1c0c6080 con 0x55e4de9f5ae0 2023-07-21T22:26:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e2000f040 con 0x55e4de9f5ae0 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1c0a1d60 con 0x55e4de9f5ae0 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 10 client.16929 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55e4de7a61f0 con 0x55e4de9f5ae0 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e2000f040 con 0x55e4de9f5ae0 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4e1c098900 con 0x55e4de9f5ae0 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 10 client.16929 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 20 client.16929 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea06660 0x55e4dea70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 4 client.16929 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e4de90cff0 con 0x55e4dea06660 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.787+0000 7f4e27fff700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea06660 0x55e4dea70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.788+0000 7f4e27fff700 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea06660 0x55e4dea70100 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d795ede20a3b4e4 server_cookie=ce6abcec498a413 in_seq=0 out_seq=0 2023-07-21T22:26:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.788+0000 7f4e077fe700 10 client.16929 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.837+0000 7f4e077fe700 1 -- 172.21.15.93:0/2931517269 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e4de90cff0 con 0x55e4dea06660 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.837+0000 7f4e077fe700 10 client.16929 handle_command_reply: tid=0 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.837+0000 7f4e3a82bb80 1 client.16929 shutdown 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 msgr2=0x55e4dea0d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005950 tx=0x7f4e1c099910 comp rx=0 tx=0).stop 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 shutdown_connections 2023-07-21T22:26:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e4dea06660 0x55e4dea70100 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e4dea06210 0x55e4dea6c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e077fe700 0 client.16929 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4de85c6b0 0x55e4dea0d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 --2- 172.21.15.93:0/2931517269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e4de9f5ae0 0x55e4dea0d6a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 >> 172.21.15.93:0/2931517269 conn(0x55e4dea16140 msgr2=0x55e4dea06aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 shutdown_connections 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 1 -- 172.21.15.93:0/2931517269 wait complete. 2023-07-21T22:26:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e3a82bb80 20 client.16929 trim_cache size 0 max 16384 2023-07-21T22:26:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:26:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:26:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:26:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:26:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 msgr2=0x7f4e100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 0x7f4e100a3870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800b5c0 tx=0x7f4e18009ee0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 msgr2=0x7f4e28198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4e281012e0 tx=0x7f4e1c00dc30 comp rx=0 tx=0).stop 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 shutdown_connections 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e28100e90 0x7f4e2819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e280fec10 0x7f4e28197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4e100a13c0 0x7f4e100a3870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 --2- 172.21.15.93:0/1047546858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e28100a30 0x7f4e28198300 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.838+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 >> 172.21.15.93:0/1047546858 conn(0x7f4e28075f90 msgr2=0x7f4e280fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.839+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 shutdown_connections 2023-07-21T22:26:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:21.839+0000 7f4e2e6a6700 1 -- 172.21.15.93:0/1047546858 wait complete. 2023-07-21T22:26:22.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 546074 6.00 MB/sec execute 419 sec latency 42.327 ms 2023-07-21T22:26:23.339 INFO:tasks.workunit.client.0.smithi093.stdout: 1 547662 6.01 MB/sec execute 420 sec latency 18.320 ms 2023-07-21T22:26:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:23 smithi194 ceph-mon[98288]: pgmap v313: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-07-21T22:26:23.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:23 smithi134 ceph-mon[97873]: pgmap v313: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-07-21T22:26:23.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:23 smithi093 ceph-mon[96579]: pgmap v313: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2023-07-21T22:26:24.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 549526 6.02 MB/sec execute 421 sec latency 17.373 ms 2023-07-21T22:26:25.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 551205 6.03 MB/sec execute 422 sec latency 10.781 ms 2023-07-21T22:26:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:25 smithi194 ceph-mon[98288]: pgmap v314: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:25 smithi134 ceph-mon[97873]: pgmap v314: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:25 smithi093 ceph-mon[96579]: pgmap v314: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:26.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 553105 6.04 MB/sec execute 423 sec latency 16.428 ms 2023-07-21T22:26:27.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 554707 6.05 MB/sec execute 424 sec latency 17.629 ms 2023-07-21T22:26:27.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:27 smithi194 ceph-mon[98288]: pgmap v315: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:27.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:27 smithi134 ceph-mon[97873]: pgmap v315: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:27 smithi093 ceph-mon[96579]: pgmap v315: 97 pgs: 97 active+clean; 243 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2023-07-21T22:26:28.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 556521 6.05 MB/sec execute 425 sec latency 28.351 ms 2023-07-21T22:26:29.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 558155 6.06 MB/sec execute 426 sec latency 15.801 ms 2023-07-21T22:26:29.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:29 smithi194 ceph-mon[98288]: pgmap v316: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-07-21T22:26:29.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:29 smithi134 ceph-mon[97873]: pgmap v316: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-07-21T22:26:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:29 smithi093 ceph-mon[96579]: pgmap v316: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 346 op/s 2023-07-21T22:26:30.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 560073 6.07 MB/sec execute 427 sec latency 18.277 ms 2023-07-21T22:26:31.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 561576 6.08 MB/sec execute 428 sec latency 79.043 ms 2023-07-21T22:26:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:31 smithi194 ceph-mon[98288]: pgmap v317: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 240 op/s 2023-07-21T22:26:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:31 smithi134 ceph-mon[97873]: pgmap v317: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 240 op/s 2023-07-21T22:26:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:31 smithi093 ceph-mon[96579]: pgmap v317: 97 pgs: 97 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 240 op/s 2023-07-21T22:26:32.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 563030 6.08 MB/sec execute 429 sec latency 19.430 ms 2023-07-21T22:26:33.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 564779 6.09 MB/sec execute 430 sec latency 123.606 ms 2023-07-21T22:26:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:33 smithi194 ceph-mon[98288]: pgmap v318: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2023-07-21T22:26:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:33 smithi134 ceph-mon[97873]: pgmap v318: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2023-07-21T22:26:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:33 smithi093 ceph-mon[96579]: pgmap v318: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2023-07-21T22:26:34.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 565977 6.09 MB/sec execute 431 sec latency 17.226 ms 2023-07-21T22:26:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/391750398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc104a40 msgr2=0x7fb0dc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/391750398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc104a40 0x7fb0dc104e10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009990 tx=0x7fb0cc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/391750398 shutdown_connections 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/391750398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc10b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/391750398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc104a40 0x7fb0dc104e10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/391750398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/391750398 >> 172.21.15.93:0/391750398 conn(0x7fb0dc0fa4f0 msgr2=0x7fb0dc0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/391750398 shutdown_connections 2023-07-21T22:26:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.957+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/391750398 wait complete. 2023-07-21T22:26:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.958+0000 7fb0e0fa9700 1 Processor -- start 2023-07-21T22:26:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.958+0000 7fb0e0fa9700 1 -- start start 2023-07-21T22:26:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.958+0000 7fb0e0fa9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.958+0000 7fb0e0fa9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.958+0000 7fb0e0fa9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 0x7fb0dc06a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0e0fa9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc06af00 con 0x7fb0dc0fea30 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0e0fa9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc06b070 con 0x7fb0dc104a40 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0e0fa9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc06b230 con 0x7fb0dc0ff3e0 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47616/0 (socket says 172.21.15.93:47616) 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 -- 172.21.15.93:0/407188362 learned_addr learned my addr 172.21.15.93:0/407188362 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 0x7fb0dc06a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 -- 172.21.15.93:0/407188362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 msgr2=0x7fb0dc071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 -- 172.21.15.93:0/407188362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 msgr2=0x7fb0dc06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 0x7fb0dc06a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 -- 172.21.15.93:0/407188362 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009520 con 0x7fb0dc0fea30 2023-07-21T22:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 0x7fb0dc06a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0da59c700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0cb7fe700 1 -- 172.21.15.93:0/407188362 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc01d070 con 0x7fb0dc0fea30 2023-07-21T22:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0cb7fe700 1 -- 172.21.15.93:0/407188362 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0cc022470 con 0x7fb0dc0fea30 2023-07-21T22:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc0095f0 con 0x7fb0dc0fea30 2023-07-21T22:26:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0cb7fe700 1 -- 172.21.15.93:0/407188362 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc017650 con 0x7fb0dc0fea30 2023-07-21T22:26:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.959+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc06f550 con 0x7fb0dc0fea30 2023-07-21T22:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.962+0000 7fb0cb7fe700 1 -- 172.21.15.93:0/407188362 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb0cc022870 con 0x7fb0dc0fea30 2023-07-21T22:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.962+0000 7fb0cb7fe700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 0x7fb0c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.962+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 0x7fb0c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.962+0000 7fb0cb7fe700 1 -- 172.21.15.93:0/407188362 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0cc0cdf70 con 0x7fb0dc0fea30 2023-07-21T22:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 0x7fb0c40a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009fd0 tx=0x7fb0d0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 Processor -- start 2023-07-21T22:26:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 -- start start 2023-07-21T22:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b448b03660 0x55b448afd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448a02c50 con 0x55b4489596b0 2023-07-21T22:26:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448a559e0 con 0x55b448b03660 2023-07-21T22:26:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.963+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448a4ee70 con 0x55b448b03210 2023-07-21T22:26:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47624/0 (socket says 172.21.15.93:47624) 2023-07-21T22:26:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0dad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b448b03660 0x55b448afd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34706/0 (socket says 172.21.15.93:34706) 2023-07-21T22:26:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0da59c700 1 -- 172.21.15.93:0/964368186 learned_addr learned my addr 172.21.15.93:0/964368186 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/964368186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b448b03660 msgr2=0x55b448afd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b448b03660 0x55b448afd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/964368186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 msgr2=0x55b448b02cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/964368186 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b44887cd80 con 0x55b448b03210 2023-07-21T22:26:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0da59c700 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000e610 tx=0x7fb0d0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f74b5e650d2f0a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0c8ff9700 1 -- 172.21.15.93:0/964368186 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0026630 con 0x55b448b03210 2023-07-21T22:26:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0c8ff9700 1 -- 172.21.15.93:0/964368186 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d0021070 con 0x55b448b03210 2023-07-21T22:26:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.964+0000 7fb0c8ff9700 1 -- 172.21.15.93:0/964368186 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d001d650 con 0x55b448b03210 2023-07-21T22:26:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/964368186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 msgr2=0x55b448afcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000e610 tx=0x7fb0d0014000 comp rx=0 tx=0).stop 2023-07-21T22:26:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/964368186 shutdown_connections 2023-07-21T22:26:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b448b03660 0x55b448afd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b4489596b0 0x55b448b02cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/964368186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448afcf40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/964368186 >> 172.21.15.93:0/964368186 conn(0x55b448af2ae0 msgr2=0x55b448af80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/964368186 shutdown_connections 2023-07-21T22:26:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/964368186 wait complete. 2023-07-21T22:26:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.965+0000 7fb0ed12eb80 1 -- start start 2023-07-21T22:26:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 Processor -- start 2023-07-21T22:26:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4489eea60 con 0x55b448b03660 2023-07-21T22:26:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4489ee760 con 0x55b4489596b0 2023-07-21T22:26:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4488d28c0 con 0x55b448b03210 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0ed12eb80 10 client.0 _ll_register_callbacks cb 0x55b4489ec700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0da59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58792/0 (socket says 172.21.15.93:58792) 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34712/0 (socket says 172.21.15.93:34712) 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0da59c700 1 -- 172.21.15.93:0/780691491 learned_addr learned my addr 172.21.15.93:0/780691491 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/780691491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 msgr2=0x55b448b095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/780691491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 msgr2=0x55b448b06ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b06ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.966+0000 7fb0d9d9b700 1 -- 172.21.15.93:0/780691491 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc06f550 con 0x55b448b03210 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ec00 tx=0x7fb0d001cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6fbf07a7122563 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0005040 con 0x55b448b03210 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0d00051a0 con 0x55b448b03210 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4489a8490 con 0x55b448b03210 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d0005360 con 0x55b448b03210 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 10 client.26287 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fb0d0017050 con 0x55b448b03210 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4489a8490 con 0x55b448b03210 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0d0004760 con 0x55b448b03210 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 10 client.26287 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 10 client.26287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 20 client.26287 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b1af30 0x55b448b1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 4 client.26287 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b448a47e50 con 0x55b448b1af30 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.967+0000 7fb0da59c700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b1af30 0x55b448b1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.968+0000 7fb0c1ffb700 10 client.26287 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.968+0000 7fb0da59c700 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b1af30 0x55b448b1d340 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf52027fa53971e server_cookie=5772c62e02983398 in_seq=0 out_seq=0 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.968+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/780691491 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b448a47e50 con 0x55b448b1af30 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.968+0000 7fb0c1ffb700 10 client.26287 handle_command_reply: tid=0 2023-07-21T22:26:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.968+0000 7fb0ed12eb80 1 client.26287 shutdown 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 msgr2=0x55b448b068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000ec00 tx=0x7fb0d001cd90 comp rx=0 tx=0).stop 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 shutdown_connections 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b1af30 0x55b448b1d340 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/780691491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b068b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0c1ffb700 0 client.26287 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 >> 172.21.15.93:0/780691491 conn(0x55b448af2ae0 msgr2=0x55b448af73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 shutdown_connections 2023-07-21T22:26:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/780691491 wait complete. 2023-07-21T22:26:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.969+0000 7fb0ed12eb80 20 client.26287 trim_cache size 0 max 16384 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 Processor -- start 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 -- start start 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b02010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448afdcd0 con 0x55b448b03660 2023-07-21T22:26:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448b028d0 con 0x55b4489596b0 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448b014f0 con 0x55b448b03210 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.985+0000 7fb0dad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0d9d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b02010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47642/0 (socket says 172.21.15.93:47642) 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1014720494 learned_addr learned my addr 172.21.15.93:0/1014720494 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0da59c700 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1014720494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 msgr2=0x55b448b02010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b02010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1014720494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 msgr2=0x55b448b11fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b11fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1014720494 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b448b21f30 con 0x55b448b03660 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0da59c700 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007580 tx=0x7fb0d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee56832a2a242ec0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0c97fa700 1 -- 172.21.15.93:0/1014720494 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e040 con 0x55b448b03660 2023-07-21T22:26:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0c97fa700 1 -- 172.21.15.93:0/1014720494 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d400f040 con 0x55b448b03660 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.986+0000 7fb0c97fa700 1 -- 172.21.15.93:0/1014720494 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4017410 con 0x55b448b03660 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1014720494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 msgr2=0x55b448af7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007580 tx=0x7fb0d400b040 comp rx=0 tx=0).stop 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1014720494 shutdown_connections 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448af7b60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1014720494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b02010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1014720494 >> 172.21.15.93:0/1014720494 conn(0x55b448af2ae0 msgr2=0x55b448b01900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1014720494 shutdown_connections 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1014720494 wait complete. 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 -- start start 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.987+0000 7fb0ed12eb80 1 Processor -- start 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b06720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448b0dbb0 con 0x55b448b03660 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448b0dd20 con 0x55b4489596b0 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b448b053a0 con 0x55b448b03210 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 10 client.0 _ll_register_callbacks cb 0x7fb0cc005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47648/0 (socket says 172.21.15.93:47648) 2023-07-21T22:26:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1063250814 learned_addr learned my addr 172.21.15.93:0/1063250814 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0d9d9b700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b06720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0da59c700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 msgr2=0x55b448b06720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b06720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1063250814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 msgr2=0x55b448b061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 -- 172.21.15.93:0/1063250814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d4009250 con 0x55b448b03660 2023-07-21T22:26:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0dad9d700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007550 tx=0x7fb0d4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bcee246c7b440a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:26:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e030 con 0x55b448b03660 2023-07-21T22:26:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb0d4007e10 con 0x55b448b03660 2023-07-21T22:26:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0d0021070 con 0x55b448b03660 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d4024070 con 0x55b448b03660 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 10 client.16947 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.988+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55b448865c00 con 0x55b448b03660 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d4035080 con 0x55b448b03660 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb0d402f030 con 0x55b448b03660 2023-07-21T22:26:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 10 client.16947 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 20 client.16947 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b74020 0x55b448b76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 4 client.16947 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b448a09ff0 con 0x55b448b74020 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0da59c700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b74020 0x55b448b76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0da59c700 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b74020 0x55b448b76430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feebf1fded0cc50b server_cookie=2bd5cf8114fdae8d in_seq=0 out_seq=0 2023-07-21T22:26:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:34.989+0000 7fb0c1ffb700 10 client.16947 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0c1ffb700 1 -- 172.21.15.93:0/1063250814 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b448a09ff0 con 0x55b448b74020 2023-07-21T22:26:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0c1ffb700 10 client.16947 handle_command_reply: tid=0 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 client.16947 shutdown 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 msgr2=0x55b448b04e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4007550 tx=0x7fb0d4004d00 comp rx=0 tx=0).stop 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 shutdown_connections 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b448b74020 0x55b448b76430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b448b03660 0x55b448b04e60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4489596b0 0x55b448b061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 --2- 172.21.15.93:0/1063250814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b448b03210 0x55b448b06720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 >> 172.21.15.93:0/1063250814 conn(0x55b448af2ae0 msgr2=0x55b448b02190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0c1ffb700 0 client.16947 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 shutdown_connections 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 1 -- 172.21.15.93:0/1063250814 wait complete. 2023-07-21T22:26:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.032+0000 7fb0ed12eb80 20 client.16947 trim_cache size 0 max 16384 2023-07-21T22:26:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:26:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:26:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:26:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:26:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 msgr2=0x7fb0c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 0x7fb0c40a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009fd0 tx=0x7fb0d0009260 comp rx=0 tx=0).stop 2023-07-21T22:26:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 msgr2=0x7fb0dc0719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc009580 comp rx=0 tx=0).stop 2023-07-21T22:26:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 shutdown_connections 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0dc104a40 0x7fb0dc06a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb0c40a12c0 0x7fb0c40a3770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0dc0fea30 0x7fb0dc0719c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 --2- 172.21.15.93:0/407188362 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0dc0ff3e0 0x7fb0dc071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.033+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 >> 172.21.15.93:0/407188362 conn(0x7fb0dc0fa4f0 msgr2=0x7fb0dc105890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.036+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 shutdown_connections 2023-07-21T22:26:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:35.036+0000 7fb0e0fa9700 1 -- 172.21.15.93:0/407188362 wait complete. 2023-07-21T22:26:35.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 567983 6.10 MB/sec execute 432 sec latency 37.385 ms 2023-07-21T22:26:35.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:35 smithi194 ceph-mon[98288]: pgmap v319: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:35 smithi134 ceph-mon[97873]: pgmap v319: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:35 smithi093 ceph-mon[96579]: pgmap v319: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:36.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 569413 6.10 MB/sec execute 433 sec latency 12.630 ms 2023-07-21T22:26:37.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 571549 6.12 MB/sec execute 434 sec latency 16.430 ms 2023-07-21T22:26:37.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:37 smithi194 ceph-mon[98288]: pgmap v320: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:37 smithi134 ceph-mon[97873]: pgmap v320: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:37 smithi093 ceph-mon[96579]: pgmap v320: 97 pgs: 97 active+clean; 258 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2023-07-21T22:26:38.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 572795 6.12 MB/sec execute 435 sec latency 31.311 ms 2023-07-21T22:26:38.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:38 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:38.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:38 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:38.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:38 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:39.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 574446 6.12 MB/sec execute 436 sec latency 31.314 ms 2023-07-21T22:26:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:39 smithi194 ceph-mon[98288]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 347 op/s 2023-07-21T22:26:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:39 smithi134 ceph-mon[97873]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 347 op/s 2023-07-21T22:26:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:39 smithi093 ceph-mon[96579]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 347 op/s 2023-07-21T22:26:40.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 575885 6.13 MB/sec execute 437 sec latency 14.184 ms 2023-07-21T22:26:41.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 577326 6.13 MB/sec execute 438 sec latency 25.598 ms 2023-07-21T22:26:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:41 smithi194 ceph-mon[98288]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2023-07-21T22:26:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:41 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:41 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:41 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:41 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:41 smithi134 ceph-mon[97873]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2023-07-21T22:26:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:41 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:41 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:41 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:41 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:41 smithi093 ceph-mon[96579]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2023-07-21T22:26:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:41 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:41 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:41 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:41 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 579336 6.15 MB/sec execute 439 sec latency 16.215 ms 2023-07-21T22:26:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:26:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:42.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:26:42.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:42.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:42.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:43.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 580672 6.14 MB/sec execute 440 sec latency 58.691 ms 2023-07-21T22:26:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: pgmap v323: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2023-07-21T22:26:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:26:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:43.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:43.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:43.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: pgmap v323: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2023-07-21T22:26:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:26:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:43.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:43.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: pgmap v323: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 344 op/s 2023-07-21T22:26:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:26:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:26:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:26:44.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 582766 6.16 MB/sec execute 441 sec latency 21.051 ms 2023-07-21T22:26:45.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 584189 6.16 MB/sec execute 442 sec latency 17.601 ms 2023-07-21T22:26:45.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:45 smithi194 ceph-mon[98288]: pgmap v324: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:45.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:45 smithi134 ceph-mon[97873]: pgmap v324: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:45 smithi093 ceph-mon[96579]: pgmap v324: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:46.340 INFO:tasks.workunit.client.0.smithi093.stdout: 1 586343 6.18 MB/sec execute 443 sec latency 15.525 ms 2023-07-21T22:26:47.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 587775 6.18 MB/sec execute 444 sec latency 16.238 ms 2023-07-21T22:26:47.588 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:47 smithi194 ceph-mon[98288]: pgmap v325: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:47.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:47 smithi134 ceph-mon[97873]: pgmap v325: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:47 smithi093 ceph-mon[96579]: pgmap v325: 97 pgs: 97 active+clean; 264 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2023-07-21T22:26:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 -- 172.21.15.93:0/390239553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 msgr2=0x7fedf010b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 --2- 172.21.15.93:0/390239553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf010b840 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fede800b0d0 tx=0x7fede800b3e0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 -- 172.21.15.93:0/390239553 shutdown_connections 2023-07-21T22:26:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 --2- 172.21.15.93:0/390239553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf010b840 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 --2- 172.21.15.93:0/390239553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 0x7fedf0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 --2- 172.21.15.93:0/390239553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 0x7fedf00ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 -- 172.21.15.93:0/390239553 >> 172.21.15.93:0/390239553 conn(0x7fedf0075f90 msgr2=0x7fedf0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 -- 172.21.15.93:0/390239553 shutdown_connections 2023-07-21T22:26:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.156+0000 7fedf4859700 1 -- 172.21.15.93:0/390239553 wait complete. 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 Processor -- start 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 -- start start 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 0x7fedf019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 0x7fedf0195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0196120 con 0x7fedf00ffab0 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0196290 con 0x7fedf00ff100 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedf4859700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0196450 con 0x7fedf01039b0 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47658/0 (socket says 172.21.15.93:47658) 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 -- 172.21.15.93:0/1335214128 learned_addr learned my addr 172.21.15.93:0/1335214128 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedeed9d700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 0x7fedf0195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fedee59c700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 0x7fedf019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 -- 172.21.15.93:0/1335214128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 msgr2=0x7fedf0195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 0x7fedf0195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 -- 172.21.15.93:0/1335214128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 msgr2=0x7fedf019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 0x7fedf019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.157+0000 7fededd9b700 1 -- 172.21.15.93:0/1335214128 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede8009d00 con 0x7fedf00ffab0 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7fededd9b700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fede400d950 tx=0x7fede400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7feddf7fe700 1 -- 172.21.15.93:0/1335214128 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4009940 con 0x7fedf00ffab0 2023-07-21T22:26:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf0196730 con 0x7fedf00ffab0 2023-07-21T22:26:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7feddf7fe700 1 -- 172.21.15.93:0/1335214128 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede4010460 con 0x7fedf00ffab0 2023-07-21T22:26:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7feddf7fe700 1 -- 172.21.15.93:0/1335214128 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede400f620 con 0x7fedf00ffab0 2023-07-21T22:26:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.158+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf01a2c40 con 0x7fedf00ffab0 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.160+0000 7feddf7fe700 1 -- 172.21.15.93:0/1335214128 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fede40108d0 con 0x7fedf00ffab0 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.161+0000 7feddf7fe700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 0x7fedd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.161+0000 7fedee59c700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 0x7fedd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.161+0000 7feddf7fe700 1 -- 172.21.15.93:0/1335214128 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fede40ccb90 con 0x7fedf00ffab0 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.161+0000 7fedee59c700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 0x7fedd80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fede0009bf0 tx=0x7fede0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.161+0000 7fee009deb80 1 Processor -- start 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 -- start start 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615bc0c50 con 0x55d615cc1210 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615c139e0 con 0x55d615cc1660 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fee009deb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615c0ce70 con 0x55d615b176b0 2023-07-21T22:26:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedee59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedee59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34744/0 (socket says 172.21.15.93:34744) 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47666/0 (socket says 172.21.15.93:47666) 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedee59c700 1 -- 172.21.15.93:0/2712542236 learned_addr learned my addr 172.21.15.93:0/2712542236 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedeed9d700 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedee59c700 1 -- 172.21.15.93:0/2712542236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 msgr2=0x55d615cbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.162+0000 7fedee59c700 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7fedee59c700 1 -- 172.21.15.93:0/2712542236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 msgr2=0x55d615cbaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7fedee59c700 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7fedee59c700 1 -- 172.21.15.93:0/2712542236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d615a3ad80 con 0x55d615b176b0 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7fededd9b700 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7fedee59c700 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fede0011e00 tx=0x7fede0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3c3c28463590a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7feddcff9700 1 -- 172.21.15.93:0/2712542236 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0012bc0 con 0x55d615b176b0 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7feddcff9700 1 -- 172.21.15.93:0/2712542236 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede0016070 con 0x55d615b176b0 2023-07-21T22:26:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.163+0000 7feddcff9700 1 -- 172.21.15.93:0/2712542236 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0003cd0 con 0x55d615b176b0 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- 172.21.15.93:0/2712542236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 msgr2=0x55d615cc0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fede0011e00 tx=0x7fede0014000 comp rx=0 tx=0).stop 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- 172.21.15.93:0/2712542236 shutdown_connections 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc0cd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 --2- 172.21.15.93:0/2712542236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cbaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- 172.21.15.93:0/2712542236 >> 172.21.15.93:0/2712542236 conn(0x55d615cb0ae0 msgr2=0x55d615cb60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- 172.21.15.93:0/2712542236 shutdown_connections 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- 172.21.15.93:0/2712542236 wait complete. 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 -- start start 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.164+0000 7fee009deb80 1 Processor -- start 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615baca60 con 0x55d615cc1210 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615bac760 con 0x55d615cc1660 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615a908c0 con 0x55d615b176b0 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 10 client.0 _ll_register_callbacks cb 0x55d615baa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47672/0 (socket says 172.21.15.93:47672) 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 -- 172.21.15.93:0/3531241875 learned_addr learned my addr 172.21.15.93:0/3531241875 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fedee59c700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fedeed9d700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 -- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 msgr2=0x55d615cc75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 -- 172.21.15.93:0/3531241875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 msgr2=0x55d615cc4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 -- 172.21.15.93:0/3531241875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf01a2c40 con 0x55d615cc1210 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fededd9b700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fede4096110 tx=0x7fede409dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec66fcd7092db1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fedd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:26:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40964f0 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d615b66490 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.165+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fede40b2200 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede40b23c0 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 10 client.16959 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fede0011ae0 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede40c8080 con 0x55d615cc1210 2023-07-21T22:26:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fede4098bc0 con 0x55d615cc1210 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 10 client.16959 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 20 client.16959 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615cd4ac0 0x55d615cd6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 4 client.16959 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d615c05e50 con 0x55d615cd4ac0 2023-07-21T22:26:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedee59c700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615cd4ac0 0x55d615cd6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedd5ffb700 10 client.16959 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.166+0000 7fedee59c700 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615cd4ac0 0x55d615cd6ed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c52619197e3158 server_cookie=46b57fb1d0d00cef in_seq=0 out_seq=0 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3531241875 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d615c05e50 con 0x55d615cd4ac0 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fedd5ffb700 10 client.16959 handle_command_reply: tid=0 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 client.16959 shutdown 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 msgr2=0x55d615cc47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fede4096110 tx=0x7fede409dd10 comp rx=0 tx=0).stop 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 shutdown_connections 2023-07-21T22:26:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615cd4ac0 0x55d615cd6ed0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615cc75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 --2- 172.21.15.93:0/3531241875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc47d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fedd5ffb700 0 client.16959 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 >> 172.21.15.93:0/3531241875 conn(0x55d615cb0ae0 msgr2=0x55d615cb53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 shutdown_connections 2023-07-21T22:26:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 1 -- 172.21.15.93:0/3531241875 wait complete. 2023-07-21T22:26:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.167+0000 7fee009deb80 20 client.16959 trim_cache size 0 max 16384 2023-07-21T22:26:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 Processor -- start 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 -- start start 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615ccbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615cbbf20 con 0x55d615cc1210 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615cd0a50 con 0x55d615cc1660 2023-07-21T22:26:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fee009deb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615cbf4a0 con 0x55d615b176b0 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47680/0 (socket says 172.21.15.93:47680) 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fededd9b700 1 -- 172.21.15.93:0/1249231889 learned_addr learned my addr 172.21.15.93:0/1249231889 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fedee59c700 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615ccbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.183+0000 7fedeed9d700 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 -- 172.21.15.93:0/1249231889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 msgr2=0x55d615ccbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615ccbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 -- 172.21.15.93:0/1249231889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 msgr2=0x55d615cc70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 -- 172.21.15.93:0/1249231889 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d615cdbb00 con 0x55d615cc1210 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fededd9b700 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fede40ca8e0 tx=0x7fede4099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cdd344cc9f2d2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7feddd7fa700 1 -- 172.21.15.93:0/1249231889 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4014040 con 0x55d615cc1210 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7feddd7fa700 1 -- 172.21.15.93:0/1249231889 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d615b66490 con 0x55d615cc1210 2023-07-21T22:26:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7feddd7fa700 1 -- 172.21.15.93:0/1249231889 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede40a1c20 con 0x55d615cc1210 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fee009deb80 1 -- 172.21.15.93:0/1249231889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 msgr2=0x55d615cc21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.184+0000 7fee009deb80 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fede40ca8e0 tx=0x7fede4099b60 comp rx=0 tx=0).stop 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- 172.21.15.93:0/1249231889 shutdown_connections 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615cc1660 0x55d615cc70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615b176b0 0x55d615ccbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- 172.21.15.93:0/1249231889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc21b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- 172.21.15.93:0/1249231889 >> 172.21.15.93:0/1249231889 conn(0x55d615cb0ae0 msgr2=0x55d615cc1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- 172.21.15.93:0/1249231889 shutdown_connections 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- 172.21.15.93:0/1249231889 wait complete. 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- start start 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 Processor -- start 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615cc1660 0x55d615cc2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615ccc070 con 0x55d615cc1210 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615cc3270 con 0x55d615b176b0 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d615cc3430 con 0x55d615cc1660 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fee009deb80 10 client.0 _ll_register_callbacks cb 0x7fede4099600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fedee59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fededd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47686/0 (socket says 172.21.15.93:47686) 2023-07-21T22:26:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fedeed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615cc1660 0x55d615cc2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fededd9b700 1 -- 172.21.15.93:0/3591658758 learned_addr learned my addr 172.21.15.93:0/3591658758 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.185+0000 7fedee59c700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58846/0 (socket says 172.21.15.93:58846) 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 -- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615cc1660 msgr2=0x55d615cc2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615cc1660 0x55d615cc2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 -- 172.21.15.93:0/3591658758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 msgr2=0x55d615cc3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 -- 172.21.15.93:0/3591658758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0011ae0 con 0x55d615cc1210 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedee59c700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fededd9b700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fede40a1b80 tx=0x7fede4010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddb97c360264ca9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4014070 con 0x55d615cc1210 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fede40b2410 con 0x55d615cc1210 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fede0016070 con 0x55d615cc1210 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede4099350 con 0x55d615cc1210 2023-07-21T22:26:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 10 client.16965 fetch_fsmap learned FSMap version 55 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55d615a23c00 con 0x55d615cc1210 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede0016070 con 0x55d615cc1210 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fede4098bc0 con 0x55d615cc1210 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 10 client.16965 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 20 client.16965 populate_metadata read hostname 'smithi093' 2023-07-21T22:26:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615d29180 0x55d615d2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 4 client.16965 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:26:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.186+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d615bc7ff0 con 0x55d615d29180 2023-07-21T22:26:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.187+0000 7fedee59c700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615d29180 0x55d615d2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.187+0000 7fedd5ffb700 10 client.16965 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.187+0000 7fedee59c700 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615d29180 0x55d615d2b560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca40ec91aff7ce8 server_cookie=f063efdfa0a67dd2 in_seq=0 out_seq=0 2023-07-21T22:26:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fedd5ffb700 1 -- 172.21.15.93:0/3591658758 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d615bc7ff0 con 0x55d615d29180 2023-07-21T22:26:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fedd5ffb700 10 client.16965 handle_command_reply: tid=0 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 client.16965 shutdown 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 msgr2=0x55d615cc44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fede40a1b80 tx=0x7fede4010cd0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 shutdown_connections 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55d615d29180 0x55d615d2b560 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d615cc1660 0x55d615cc2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d615b176b0 0x55d615cc3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 --2- 172.21.15.93:0/3591658758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d615cc1210 0x55d615cc44b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 >> 172.21.15.93:0/3591658758 conn(0x55d615cb0ae0 msgr2=0x55d615cbf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fedd5ffb700 0 client.16965 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 shutdown_connections 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.232+0000 7fee009deb80 1 -- 172.21.15.93:0/3591658758 wait complete. 2023-07-21T22:26:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fee009deb80 20 client.16965 trim_cache size 0 max 16384 2023-07-21T22:26:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:26:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:26:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:26:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 msgr2=0x7fedd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 0x7fedd80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fede0009bf0 tx=0x7fede0009260 comp rx=0 tx=0).stop 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 msgr2=0x7fedf019c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fede400d950 tx=0x7fede400dc60 comp rx=0 tx=0).stop 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 shutdown_connections 2023-07-21T22:26:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedf01039b0 0x7fedf0195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf00ff100 0x7fedf019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fedd80a12c0 0x7fedd80a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 --2- 172.21.15.93:0/1335214128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedf00ffab0 0x7fedf019c190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.233+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 >> 172.21.15.93:0/1335214128 conn(0x7fedf0075f90 msgr2=0x7fedf00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.234+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 shutdown_connections 2023-07-21T22:26:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:26:48.234+0000 7fedf4859700 1 -- 172.21.15.93:0/1335214128 wait complete. 2023-07-21T22:26:48.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 589829 6.19 MB/sec execute 445 sec latency 42.737 ms 2023-07-21T22:26:49.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 591045 6.19 MB/sec execute 446 sec latency 26.710 ms 2023-07-21T22:26:49.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:49 smithi194 ceph-mon[98288]: pgmap v326: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 348 op/s 2023-07-21T22:26:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:49 smithi134 ceph-mon[97873]: pgmap v326: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 348 op/s 2023-07-21T22:26:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:49 smithi093 ceph-mon[96579]: pgmap v326: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 348 op/s 2023-07-21T22:26:50.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 593226 6.20 MB/sec execute 447 sec latency 23.226 ms 2023-07-21T22:26:51.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 594574 6.20 MB/sec execute 448 sec latency 15.530 ms 2023-07-21T22:26:51.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:51 smithi194 ceph-mon[98288]: pgmap v327: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2023-07-21T22:26:51.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:51 smithi134 ceph-mon[97873]: pgmap v327: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2023-07-21T22:26:51.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:51 smithi093 ceph-mon[96579]: pgmap v327: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 228 op/s 2023-07-21T22:26:52.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 596772 6.22 MB/sec execute 449 sec latency 17.869 ms 2023-07-21T22:26:53.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 597932 6.21 MB/sec execute 450 sec latency 29.780 ms 2023-07-21T22:26:53.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:53 smithi194 ceph-mon[98288]: pgmap v328: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2023-07-21T22:26:53.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:53 smithi134 ceph-mon[97873]: pgmap v328: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2023-07-21T22:26:53.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:53 smithi093 ceph-mon[96579]: pgmap v328: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2023-07-21T22:26:54.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 599907 6.22 MB/sec execute 451 sec latency 22.308 ms 2023-07-21T22:26:55.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 601293 6.23 MB/sec execute 452 sec latency 18.915 ms 2023-07-21T22:26:55.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:55 smithi194 ceph-mon[98288]: pgmap v329: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:55 smithi134 ceph-mon[97873]: pgmap v329: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:55 smithi093 ceph-mon[96579]: pgmap v329: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:56.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 603235 6.24 MB/sec execute 453 sec latency 22.873 ms 2023-07-21T22:26:57.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 604716 6.24 MB/sec execute 454 sec latency 11.626 ms 2023-07-21T22:26:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:57 smithi194 ceph-mon[98288]: pgmap v330: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:57 smithi134 ceph-mon[97873]: pgmap v330: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:57 smithi093 ceph-mon[96579]: pgmap v330: 97 pgs: 97 active+clean; 267 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2023-07-21T22:26:58.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 606412 6.24 MB/sec execute 455 sec latency 20.331 ms 2023-07-21T22:26:59.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 607898 6.25 MB/sec execute 456 sec latency 27.188 ms 2023-07-21T22:26:59.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:26:59 smithi194 ceph-mon[98288]: pgmap v331: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2023-07-21T22:26:59.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:26:59 smithi134 ceph-mon[97873]: pgmap v331: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2023-07-21T22:26:59.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:26:59 smithi093 ceph-mon[96579]: pgmap v331: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 359 op/s 2023-07-21T22:27:00.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 609609 6.25 MB/sec execute 457 sec latency 43.961 ms 2023-07-21T22:27:01.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 610954 6.26 MB/sec execute 458 sec latency 46.821 ms 2023-07-21T22:27:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 -- 172.21.15.93:0/67598602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8100780 msgr2=0x7f7bb8104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 --2- 172.21.15.93:0/67598602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8100780 0x7f7bb8104740 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40099c0 tx=0x7f7bb4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 -- 172.21.15.93:0/67598602 shutdown_connections 2023-07-21T22:27:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 --2- 172.21.15.93:0/67598602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8104e20 0x7f7bb810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 --2- 172.21.15.93:0/67598602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 0x7f7bb81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 --2- 172.21.15.93:0/67598602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8100780 0x7f7bb8104740 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 -- 172.21.15.93:0/67598602 >> 172.21.15.93:0/67598602 conn(0x7f7bb80fb6f0 msgr2=0x7f7bb80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 -- 172.21.15.93:0/67598602 shutdown_connections 2023-07-21T22:27:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.346+0000 7f7bc0079700 1 -- 172.21.15.93:0/67598602 wait complete. 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 Processor -- start 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 -- start start 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 0x7f7bb819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8100780 0x7f7bb819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8196120 con 0x7f7bb8104e20 2023-07-21T22:27:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8196290 con 0x7f7bb8100780 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bc0079700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8196450 con 0x7f7bb80ffde0 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 0x7f7bb819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47702/0 (socket says 172.21.15.93:47702) 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 -- 172.21.15.93:0/3577959462 learned_addr learned my addr 172.21.15.93:0/3577959462 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbd614700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8100780 0x7f7bb819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 -- 172.21.15.93:0/3577959462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 msgr2=0x7f7bb819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 0x7f7bb819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 -- 172.21.15.93:0/3577959462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8100780 msgr2=0x7f7bb819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8100780 0x7f7bb819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 -- 172.21.15.93:0/3577959462 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40095f0 con 0x7f7bb8104e20 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.347+0000 7f7bbe616700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000ef30 tx=0x7f7bb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.348+0000 7f7baeffd700 1 -- 172.21.15.93:0/3577959462 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0018070 con 0x7f7bb8104e20 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.348+0000 7f7baeffd700 1 -- 172.21.15.93:0/3577959462 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb000f040 con 0x7f7bb8104e20 2023-07-21T22:27:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.348+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb81966d0 con 0x7f7bb8104e20 2023-07-21T22:27:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.348+0000 7f7baeffd700 1 -- 172.21.15.93:0/3577959462 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000cd00 con 0x7f7bb8104e20 2023-07-21T22:27:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.348+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb81a2c40 con 0x7f7bb8104e20 2023-07-21T22:27:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.351+0000 7f7bcc1feb80 1 Processor -- start 2023-07-21T22:27:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.351+0000 7f7baeffd700 1 -- 172.21.15.93:0/3577959462 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7bb0003a00 con 0x7f7bb8104e20 2023-07-21T22:27:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.351+0000 7f7bcc1feb80 1 -- start start 2023-07-21T22:27:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bd8fc50 con 0x55ad7be90210 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bde29e0 con 0x55ad7bce66b0 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bddbe70 con 0x55ad7be90660 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47706/0 (socket says 172.21.15.93:47706) 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 -- 172.21.15.93:0/2846934382 learned_addr learned my addr 172.21.15.93:0/2846934382 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7baeffd700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 0x7f7ba40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbe616700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 0x7f7ba40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 -- 172.21.15.93:0/2846934382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 msgr2=0x55ad7be8a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be8a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 -- 172.21.15.93:0/2846934382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 msgr2=0x55ad7be8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 -- 172.21.15.93:0/2846934382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad7bc09d80 con 0x55ad7be90210 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7baeffd700 1 -- 172.21.15.93:0/3577959462 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7bb00cc790 con 0x7f7bb8104e20 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009db0 tx=0x7f7ba8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181f381f72beb200 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.352+0000 7f7ba3fff700 1 -- 172.21.15.93:0/2846934382 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8019630 con 0x55ad7be90210 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bbe616700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 0x7f7ba40a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0099880 tx=0x7f7bb006b960 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7ba3fff700 1 -- 172.21.15.93:0/2846934382 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba8019c50 con 0x55ad7be90210 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7ba3fff700 1 -- 172.21.15.93:0/2846934382 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800eb70 con 0x55ad7be90210 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2846934382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 msgr2=0x55ad7be89f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009db0 tx=0x7f7ba8011720 comp rx=0 tx=0).stop 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2846934382 shutdown_connections 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be8a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2846934382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be89f40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2846934382 >> 172.21.15.93:0/2846934382 conn(0x55ad7be7fae0 msgr2=0x55ad7be850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2846934382 shutdown_connections 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.353+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2846934382 wait complete. 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 -- start start 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 Processor -- start 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bd7ba60 con 0x55ad7be90210 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bd7b760 con 0x55ad7bce66b0 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bc5f8c0 con 0x55ad7be90660 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.354+0000 7f7bcc1feb80 10 client.0 _ll_register_callbacks cb 0x55ad7bd79700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbd614700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47714/0 (socket says 172.21.15.93:47714) 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 -- 172.21.15.93:0/3152542854 learned_addr learned my addr 172.21.15.93:0/3152542854 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbd614700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be96580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58874/0 (socket says 172.21.15.93:58874) 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbd614700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 -- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 msgr2=0x55ad7be93e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be93e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 -- 172.21.15.93:0/3152542854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 msgr2=0x55ad7be96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 -- 172.21.15.93:0/3152542854 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb810d230 con 0x55ad7be90210 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bbde15700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800e6f0 tx=0x7f7ba800e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2f33190acefbe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7ba17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800ec00 con 0x55ad7be90210 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ba8019dc0 con 0x55ad7be90210 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad7bd35490 con 0x55ad7be90210 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba8005ce0 con 0x55ad7be90210 2023-07-21T22:27:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bcc1feb80 10 client.16977 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f7ba80099e0 con 0x55ad7be90210 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.355+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad7bd35490 con 0x55ad7be90210 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ba8028020 con 0x55ad7be90210 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 10 client.16977 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 20 client.16977 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7bea38f0 0x55ad7bea5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 4 client.16977 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad7bdd4e50 con 0x55ad7bea38f0 2023-07-21T22:27:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bbd614700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7bea38f0 0x55ad7bea5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7ba17fa700 10 client.16977 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.356+0000 7f7bbd614700 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7bea38f0 0x55ad7bea5d00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6523fdaea76e239 server_cookie=fc3b1c8128b056cc in_seq=0 out_seq=0 2023-07-21T22:27:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7ba17fa700 1 -- 172.21.15.93:0/3152542854 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ad7bdd4e50 con 0x55ad7bea38f0 2023-07-21T22:27:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7ba17fa700 10 client.16977 handle_command_reply: tid=0 2023-07-21T22:27:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 client.16977 shutdown 2023-07-21T22:27:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 msgr2=0x55ad7be93800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800e6f0 tx=0x7f7ba800e3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 shutdown_connections 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7bea38f0 0x55ad7bea5d00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90660 0x55ad7be93e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/3152542854 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be90210 0x55ad7be93800 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7ba17fa700 0 client.16977 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 >> 172.21.15.93:0/3152542854 conn(0x55ad7be7fae0 msgr2=0x55ad7be843e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 shutdown_connections 2023-07-21T22:27:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/3152542854 wait complete. 2023-07-21T22:27:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.357+0000 7f7bcc1feb80 20 client.16977 trim_cache size 0 max 16384 2023-07-21T22:27:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 Processor -- start 2023-07-21T22:27:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 -- start start 2023-07-21T22:27:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 0x55ad7be93780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7be90210 0x55ad7be9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7be94430 con 0x55ad7be7fae0 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7be8e220 con 0x55ad7be90210 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7be91d60 con 0x55ad7bce66b0 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47720/0 (socket says 172.21.15.93:47720) 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 -- 172.21.15.93:0/2951214576 learned_addr learned my addr 172.21.15.93:0/2951214576 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 0x55ad7be93780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 -- 172.21.15.93:0/2951214576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 msgr2=0x55ad7be93780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 0x55ad7be93780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 -- 172.21.15.93:0/2951214576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7be90210 msgr2=0x55ad7be9ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7be90210 0x55ad7be9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 -- 172.21.15.93:0/2951214576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad7bbf2c00 con 0x55ad7be7fae0 2023-07-21T22:27:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 0x55ad7be93780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.372+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4005850 tx=0x7f7bb4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55343c965c429df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bacff9700 1 -- 172.21.15.93:0/2951214576 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4003e30 con 0x55ad7be7fae0 2023-07-21T22:27:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bacff9700 1 -- 172.21.15.93:0/2951214576 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb401e040 con 0x55ad7be7fae0 2023-07-21T22:27:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bacff9700 1 -- 172.21.15.93:0/2951214576 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400fad0 con 0x55ad7be7fae0 2023-07-21T22:27:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2951214576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 msgr2=0x55ad7be89f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4005850 tx=0x7f7bb4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2951214576 shutdown_connections 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7be90210 0x55ad7be9ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be89f40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2951214576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7bce66b0 0x55ad7be93780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2951214576 >> 172.21.15.93:0/2951214576 conn(0x55ad7be8a480 msgr2=0x55ad7be93070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2951214576 shutdown_connections 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.373+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2951214576 wait complete. 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 -- start start 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 Processor -- start 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90210 0x55ad7be8c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bef77b0 con 0x55ad7be7fae0 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bef7920 con 0x55ad7bce66b0 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7bef7ae0 con 0x55ad7be90210 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bcc1feb80 10 client.0 _ll_register_callbacks cb 0x7f7bb00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47728/0 (socket says 172.21.15.93:47728) 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 -- 172.21.15.93:0/2118887469 learned_addr learned my addr 172.21.15.93:0/2118887469 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90210 0x55ad7be8c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 -- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90210 msgr2=0x55ad7be8c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90210 0x55ad7be8c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 -- 172.21.15.93:0/2118887469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 msgr2=0x55ad7be8bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.374+0000 7f7bbde15700 1 -- 172.21.15.93:0/2118887469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4003930 con 0x55ad7be7fae0 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bbde15700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8011760 tx=0x7f7ba800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee869c2fd501bf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800ee10 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ba8019dc0 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ba8019c50 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba8011aa0 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 10 client.16983 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55ad7bc301f0 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba8019c50 con 0x55ad7be7fae0 2023-07-21T22:27:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ba8011790 con 0x55ad7be7fae0 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 10 client.16983 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 20 client.16983 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7be90660 0x55ad7befad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 4 client.16983 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ad7bd96ff0 con 0x55ad7be90660 2023-07-21T22:27:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.375+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7be90660 0x55ad7befad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.376+0000 7f7ba17fa700 10 client.16983 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.376+0000 7f7bbd614700 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7be90660 0x55ad7befad50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8226c4bd76202a1 server_cookie=85d28f316e27979 in_seq=0 out_seq=0 2023-07-21T22:27:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7ba17fa700 1 -- 172.21.15.93:0/2118887469 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ad7bd96ff0 con 0x55ad7be90660 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7ba17fa700 10 client.16983 handle_command_reply: tid=0 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 client.16983 shutdown 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 msgr2=0x55ad7be8c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8011760 tx=0x7f7ba800e720 comp rx=0 tx=0).stop 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 shutdown_connections 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ad7be90660 0x55ad7befad50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ad7be90210 0x55ad7be8c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ad7bce66b0 0x55ad7be8bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 --2- 172.21.15.93:0/2118887469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ad7be7fae0 0x55ad7be8c310 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.426+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 >> 172.21.15.93:0/2118887469 conn(0x55ad7be8a480 msgr2=0x55ad7be90a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7ba17fa700 0 client.16983 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 shutdown_connections 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bcc1feb80 1 -- 172.21.15.93:0/2118887469 wait complete. 2023-07-21T22:27:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bcc1feb80 20 client.16983 trim_cache size 0 max 16384 2023-07-21T22:27:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:27:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:27:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:27:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 msgr2=0x7f7ba40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 0x7f7ba40a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0099880 tx=0x7f7bb006b960 comp rx=0 tx=0).stop 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 msgr2=0x7f7bb8195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000ef30 tx=0x7f7bb000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 shutdown_connections 2023-07-21T22:27:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7ba40a12c0 0x7f7ba40a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb8104e20 0x7f7bb8195ab0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80ffde0 0x7f7bb819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 --2- 172.21.15.93:0/3577959462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bb8100780 0x7f7bb819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 >> 172.21.15.93:0/3577959462 conn(0x7f7bb80fb6f0 msgr2=0x7f7bb8103760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 shutdown_connections 2023-07-21T22:27:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:01.427+0000 7f7bc0079700 1 -- 172.21.15.93:0/3577959462 wait complete. 2023-07-21T22:27:01.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:01 smithi194 ceph-mon[98288]: pgmap v332: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2023-07-21T22:27:01.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:01 smithi134 ceph-mon[97873]: pgmap v332: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2023-07-21T22:27:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:01 smithi093 ceph-mon[96579]: pgmap v332: 97 pgs: 97 active+clean; 274 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 241 op/s 2023-07-21T22:27:02.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 612058 6.26 MB/sec execute 459 sec latency 24.687 ms 2023-07-21T22:27:03.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 613865 6.27 MB/sec execute 460 sec latency 58.669 ms 2023-07-21T22:27:03.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:03 smithi194 ceph-mon[98288]: pgmap v333: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 393 op/s 2023-07-21T22:27:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:03 smithi134 ceph-mon[97873]: pgmap v333: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 393 op/s 2023-07-21T22:27:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:03 smithi093 ceph-mon[96579]: pgmap v333: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 393 op/s 2023-07-21T22:27:04.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 615125 6.27 MB/sec execute 461 sec latency 19.232 ms 2023-07-21T22:27:05.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 616928 6.27 MB/sec execute 462 sec latency 19.621 ms 2023-07-21T22:27:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:05 smithi194 ceph-mon[98288]: pgmap v334: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-21T22:27:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:05 smithi194 ceph-mon[98288]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:27:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:05 smithi134 ceph-mon[97873]: pgmap v334: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-21T22:27:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:05 smithi134 ceph-mon[97873]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:27:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:05 smithi093 ceph-mon[96579]: pgmap v334: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2023-07-21T22:27:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:05 smithi093 ceph-mon[96579]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:27:06.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 618542 6.28 MB/sec execute 463 sec latency 13.354 ms 2023-07-21T22:27:07.341 INFO:tasks.workunit.client.0.smithi093.stdout: 1 620421 6.28 MB/sec execute 464 sec latency 23.577 ms 2023-07-21T22:27:07.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:07 smithi194 ceph-mon[98288]: pgmap v336: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T22:27:07.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:07 smithi134 ceph-mon[97873]: pgmap v336: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T22:27:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:07 smithi093 ceph-mon[96579]: pgmap v336: 97 pgs: 97 active+clean; 272 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 319 op/s 2023-07-21T22:27:08.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 621954 6.30 MB/sec execute 465 sec latency 34.890 ms 2023-07-21T22:27:09.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 623388 6.29 MB/sec execute 466 sec latency 28.348 ms 2023-07-21T22:27:09.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:09 smithi194 ceph-mon[98288]: pgmap v337: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:09 smithi134 ceph-mon[97873]: pgmap v337: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:09 smithi093 ceph-mon[96579]: pgmap v337: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:10.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 625399 6.31 MB/sec execute 467 sec latency 20.302 ms 2023-07-21T22:27:11.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 626903 6.31 MB/sec execute 468 sec latency 16.677 ms 2023-07-21T22:27:11.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:11 smithi194 ceph-mon[98288]: pgmap v338: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:11.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:11 smithi134 ceph-mon[97873]: pgmap v338: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:11 smithi093 ceph-mon[96579]: pgmap v338: 97 pgs: 97 active+clean; 280 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2023-07-21T22:27:12.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 628897 6.32 MB/sec execute 469 sec latency 17.329 ms 2023-07-21T22:27:13.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 630310 6.32 MB/sec execute 470 sec latency 16.956 ms 2023-07-21T22:27:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:13 smithi194 ceph-mon[98288]: pgmap v339: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:13 smithi194 ceph-mon[98288]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:27:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:13 smithi134 ceph-mon[97873]: pgmap v339: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:13 smithi134 ceph-mon[97873]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:27:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:13 smithi093 ceph-mon[96579]: pgmap v339: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:13 smithi093 ceph-mon[96579]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:27:14.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 632322 6.33 MB/sec execute 471 sec latency 59.659 ms 2023-07-21T22:27:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 -- 172.21.15.93:0/3110219286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b4104990 msgr2=0x7f74b4104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 --2- 172.21.15.93:0/3110219286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b4104990 0x7f74b4104d60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009990 tx=0x7f74a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 -- 172.21.15.93:0/3110219286 shutdown_connections 2023-07-21T22:27:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 --2- 172.21.15.93:0/3110219286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b40ff330 0x7f74b410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 --2- 172.21.15.93:0/3110219286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b4104990 0x7f74b4104d60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 --2- 172.21.15.93:0/3110219286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40fe980 0x7f74b40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 -- 172.21.15.93:0/3110219286 >> 172.21.15.93:0/3110219286 conn(0x7f74b4074680 msgr2=0x7f74b4074a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 -- 172.21.15.93:0/3110219286 shutdown_connections 2023-07-21T22:27:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.542+0000 7f74b898f700 1 -- 172.21.15.93:0/3110219286 wait complete. 2023-07-21T22:27:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 Processor -- start 2023-07-21T22:27:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 -- start start 2023-07-21T22:27:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40ff330 0x7f74b419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b4104990 0x7f74b41948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4194f60 con 0x7f74b40fe980 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b41950d0 con 0x7f74b40ff330 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b898f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b4195290 con 0x7f74b4104990 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47740/0 (socket says 172.21.15.93:47740) 2023-07-21T22:27:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 -- 172.21.15.93:0/3447894609 learned_addr learned my addr 172.21.15.93:0/3447894609 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b2d9d700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b4104990 0x7f74b41948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40ff330 0x7f74b419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 -- 172.21.15.93:0/3447894609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b4104990 msgr2=0x7f74b41948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b4104990 0x7f74b41948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 -- 172.21.15.93:0/3447894609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40ff330 msgr2=0x7f74b419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40ff330 0x7f74b419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.543+0000 7f74b259c700 1 -- 172.21.15.93:0/3447894609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40095f0 con 0x7f74b40fe980 2023-07-21T22:27:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74b259c700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f74a400b2e0 tx=0x7f74a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74a37fe700 1 -- 172.21.15.93:0/3447894609 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a401d070 con 0x7f74b40fe980 2023-07-21T22:27:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4195510 con 0x7f74b40fe980 2023-07-21T22:27:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74a37fe700 1 -- 172.21.15.93:0/3447894609 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a4005030 con 0x7f74b40fe980 2023-07-21T22:27:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74a37fe700 1 -- 172.21.15.93:0/3447894609 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4004a40 con 0x7f74b40fe980 2023-07-21T22:27:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.544+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b41a1870 con 0x7f74b40fe980 2023-07-21T22:27:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.546+0000 7f74a37fe700 1 -- 172.21.15.93:0/3447894609 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f74a4005430 con 0x7f74b40fe980 2023-07-21T22:27:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.546+0000 7f74a37fe700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 0x7f749c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.547+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 0x7f749c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.547+0000 7f74a37fe700 1 -- 172.21.15.93:0/3447894609 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74a40cdd50 con 0x7f74b40fe980 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.547+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 0x7f749c0a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74b40fc900 tx=0x7f74a80073d0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.547+0000 7f74c4b14b80 1 Processor -- start 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 -- start start 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682add40 0x55ae682ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682aca10 0x55ae682bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae681b1c50 con 0x55ae681086b0 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae682049e0 con 0x55ae682add40 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae681fde70 con 0x55ae682aca10 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47748/0 (socket says 172.21.15.93:47748) 2023-07-21T22:27:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 -- 172.21.15.93:0/3375442734 learned_addr learned my addr 172.21.15.93:0/3375442734 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b2d9d700 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682aca10 0x55ae682bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682add40 0x55ae682ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 -- 172.21.15.93:0/3375442734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682aca10 msgr2=0x55ae682bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682aca10 0x55ae682bc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 -- 172.21.15.93:0/3375442734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682add40 msgr2=0x55ae682ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682add40 0x55ae682ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.548+0000 7f74b259c700 1 -- 172.21.15.93:0/3375442734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6802bd80 con 0x55ae681086b0 2023-07-21T22:27:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74b259c700 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f74a4097350 tx=0x7f74a409ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52f8e54f5ab8d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74a0ff9700 1 -- 172.21.15.93:0/3375442734 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a3210 con 0x55ae681086b0 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74a0ff9700 1 -- 172.21.15.93:0/3375442734 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a40991f0 con 0x55ae681086b0 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74a0ff9700 1 -- 172.21.15.93:0/3375442734 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4004e00 con 0x55ae681086b0 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 -- 172.21.15.93:0/3375442734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 msgr2=0x55ae682abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f74a4097350 tx=0x7f74a409ae50 comp rx=0 tx=0).stop 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 -- 172.21.15.93:0/3375442734 shutdown_connections 2023-07-21T22:27:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682aca10 0x55ae682bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682abf40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/3375442734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682add40 0x55ae682ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.549+0000 7f74c4b14b80 1 -- 172.21.15.93:0/3375442734 >> 172.21.15.93:0/3375442734 conn(0x55ae682a1ae0 msgr2=0x55ae682a70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- 172.21.15.93:0/3375442734 shutdown_connections 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- 172.21.15.93:0/3375442734 wait complete. 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- start start 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 Processor -- start 2023-07-21T22:27:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6819da60 con 0x55ae682add40 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6819d760 con 0x55ae682aca10 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.550+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae680818c0 con 0x55ae681086b0 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74c4b14b80 10 client.0 _ll_register_callbacks cb 0x55ae6819b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:34834/0 (socket says 172.21.15.93:34834) 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47758/0 (socket says 172.21.15.93:47758) 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 -- 172.21.15.93:0/565422338 learned_addr learned my addr 172.21.15.93:0/565422338 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 -- 172.21.15.93:0/565422338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 msgr2=0x55ae682b97a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:27:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b97a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 -- 172.21.15.93:0/565422338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 msgr2=0x55ae682b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 -- 172.21.15.93:0/565422338 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b41a1870 con 0x55ae681086b0 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b2d9d700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74b259c700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a2ce0 tx=0x7f74a4004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6ecb3ab60807f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f7499ffb700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:27:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a3290 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae68157490 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74a409b2c0 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a409a340 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74c4b14b80 10 client.26341 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.551+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55ae68014c00 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a4064070 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f74a409d070 con 0x55ae681086b0 2023-07-21T22:27:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 20 client.26341 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682c9700 0x55ae682cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 4 client.26341 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae681f6e50 con 0x55ae682c9700 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682c9700 0x55ae682cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f7499ffb700 10 client.26341 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.552+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682c9700 0x55ae682cbb10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55313dfcd31d3e0 server_cookie=caddaf9394f09bb7 in_seq=0 out_seq=0 2023-07-21T22:27:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.553+0000 7f7499ffb700 1 -- 172.21.15.93:0/565422338 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae681f6e50 con 0x55ae682c9700 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.553+0000 7f7499ffb700 10 client.26341 handle_command_reply: tid=0 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.553+0000 7f74c4b14b80 1 client.26341 shutdown 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 msgr2=0x55ae682bc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a2ce0 tx=0x7f74a4004f70 comp rx=0 tx=0).stop 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 shutdown_connections 2023-07-21T22:27:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682c9700 0x55ae682cbb10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682add40 0x55ae682b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682bc590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/565422338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f7499ffb700 0 client.26341 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 >> 172.21.15.93:0/565422338 conn(0x55ae682a1ae0 msgr2=0x55ae682a63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 shutdown_connections 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 1 -- 172.21.15.93:0/565422338 wait complete. 2023-07-21T22:27:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.554+0000 7f74c4b14b80 20 client.26341 trim_cache size 0 max 16384 2023-07-21T22:27:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 Processor -- start 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 -- start start 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6829eb20 con 0x55ae682a1ae0 2023-07-21T22:27:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae682a3c00 con 0x55ae682aca10 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae682acf20 con 0x55ae681086b0 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47762/0 (socket says 172.21.15.93:47762) 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74b259c700 1 -- 172.21.15.93:0/2416249090 learned_addr learned my addr 172.21.15.93:0/2416249090 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74b2d9d700 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.569+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 -- 172.21.15.93:0/2416249090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 msgr2=0x55ae682c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 -- 172.21.15.93:0/2416249090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 msgr2=0x55ae682b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 -- 172.21.15.93:0/2416249090 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae680521f0 con 0x55ae682a1ae0 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74b259c700 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f74a4097350 tx=0x7f74a40cba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9653aad444d3e8c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74a17fa700 1 -- 172.21.15.93:0/2416249090 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a3210 con 0x55ae682a1ae0 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74a17fa700 1 -- 172.21.15.93:0/2416249090 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a4064030 con 0x55ae682a1ae0 2023-07-21T22:27:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.570+0000 7f74a17fa700 1 -- 172.21.15.93:0/2416249090 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a2d80 con 0x55ae682a1ae0 2023-07-21T22:27:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- 172.21.15.93:0/2416249090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 msgr2=0x55ae682abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f74a4097350 tx=0x7f74a40cba00 comp rx=0 tx=0).stop 2023-07-21T22:27:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- 172.21.15.93:0/2416249090 shutdown_connections 2023-07-21T22:27:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae682b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae682a1ae0 0x55ae682abf40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/2416249090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae681086b0 0x55ae682c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- 172.21.15.93:0/2416249090 >> 172.21.15.93:0/2416249090 conn(0x55ae682c58a0 msgr2=0x55ae682b7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- 172.21.15.93:0/2416249090 shutdown_connections 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- 172.21.15.93:0/2416249090 wait complete. 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 -- start start 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.571+0000 7f74c4b14b80 1 Processor -- start 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682a1ae0 0x55ae682bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae6831c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6831cb00 con 0x55ae681086b0 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6831cc40 con 0x55ae682aca10 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6831cd80 con 0x55ae682a1ae0 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 10 client.0 _ll_register_callbacks cb 0x7f74a8007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47768/0 (socket says 172.21.15.93:47768) 2023-07-21T22:27:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 -- 172.21.15.93:0/1806564821 learned_addr learned my addr 172.21.15.93:0/1806564821 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b2d9d700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae6831c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682a1ae0 0x55ae682bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 -- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682a1ae0 msgr2=0x55ae682bf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682a1ae0 0x55ae682bf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 -- 172.21.15.93:0/1806564821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 msgr2=0x55ae6831c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae6831c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 -- 172.21.15.93:0/1806564821 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae68014c00 con 0x55ae681086b0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74b259c700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f74a40095c0 tx=0x7f74a409f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec14ea36f201f9d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f7499ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a3210 con 0x55ae681086b0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74a409a340 con 0x55ae681086b0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.572+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74a40991f0 con 0x55ae681086b0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a409a500 con 0x55ae681086b0 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 10 client.17001 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55ae6802a250 con 0x55ae681086b0 2023-07-21T22:27:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74a40991f0 con 0x55ae681086b0 2023-07-21T22:27:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f74a4079d20 con 0x55ae681086b0 2023-07-21T22:27:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 20 client.17001 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682add40 0x55ae6831ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 4 client.17001 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae681b8ff0 con 0x55ae682add40 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682add40 0x55ae6831ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f74b1d9b700 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682add40 0x55ae6831ffa0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55f4b879135e915 server_cookie=b7312cd59e2aff50 in_seq=0 out_seq=0 2023-07-21T22:27:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.573+0000 7f7499ffb700 10 client.17001 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.618+0000 7f7499ffb700 1 -- 172.21.15.93:0/1806564821 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae681b8ff0 con 0x55ae682add40 2023-07-21T22:27:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.618+0000 7f7499ffb700 10 client.17001 handle_command_reply: tid=0 2023-07-21T22:27:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.618+0000 7f74c4b14b80 1 client.17001 shutdown 2023-07-21T22:27:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 msgr2=0x55ae682bef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f74a40095c0 tx=0x7f74a409f000 comp rx=0 tx=0).stop 2023-07-21T22:27:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 shutdown_connections 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ae682add40 0x55ae6831ffa0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae682aca10 0x55ae6831c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f7499ffb700 0 client.17001 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ae681086b0 0x55ae682bef70 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 --2- 172.21.15.93:0/1806564821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae682a1ae0 0x55ae682bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 >> 172.21.15.93:0/1806564821 conn(0x55ae682c58a0 msgr2=0x55ae682bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 shutdown_connections 2023-07-21T22:27:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.619+0000 7f74c4b14b80 1 -- 172.21.15.93:0/1806564821 wait complete. 2023-07-21T22:27:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74c4b14b80 20 client.17001 trim_cache size 0 max 16384 2023-07-21T22:27:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:27:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:27:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:27:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:27:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 msgr2=0x7f749c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 0x7f749c0a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74b40fc900 tx=0x7f74a80073d0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 msgr2=0x7f74b419a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f74a400b2e0 tx=0x7f74a4004730 comp rx=0 tx=0).stop 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 shutdown_connections 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74b4104990 0x7f74b41948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f749c0a12c0 0x7f749c0a3770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74b40fe980 0x7f74b419a880 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 --2- 172.21.15.93:0/3447894609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74b40ff330 0x7f74b419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 >> 172.21.15.93:0/3447894609 conn(0x7f74b4074680 msgr2=0x7f74b40fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 shutdown_connections 2023-07-21T22:27:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:14.620+0000 7f74b898f700 1 -- 172.21.15.93:0/3447894609 wait complete. 2023-07-21T22:27:15.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 633576 6.34 MB/sec execute 472 sec latency 41.186 ms 2023-07-21T22:27:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:15 smithi194 ceph-mon[98288]: pgmap v341: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-21T22:27:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:15 smithi134 ceph-mon[97873]: pgmap v341: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-21T22:27:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:15 smithi093 ceph-mon[96579]: pgmap v341: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2023-07-21T22:27:16.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 635781 6.35 MB/sec execute 473 sec latency 17.112 ms 2023-07-21T22:27:17.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 636803 6.34 MB/sec execute 474 sec latency 46.189 ms 2023-07-21T22:27:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:17 smithi194 ceph-mon[98288]: pgmap v342: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:17 smithi134 ceph-mon[97873]: pgmap v342: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:17 smithi093 ceph-mon[96579]: pgmap v342: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 270 op/s 2023-07-21T22:27:18.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 638882 6.35 MB/sec execute 475 sec latency 16.946 ms 2023-07-21T22:27:19.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 640171 6.35 MB/sec execute 476 sec latency 28.904 ms 2023-07-21T22:27:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:19 smithi194 ceph-mon[98288]: pgmap v343: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:19 smithi134 ceph-mon[97873]: pgmap v343: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:19 smithi093 ceph-mon[96579]: pgmap v343: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:20.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 642255 6.36 MB/sec execute 477 sec latency 20.570 ms 2023-07-21T22:27:21.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 643789 6.37 MB/sec execute 478 sec latency 10.895 ms 2023-07-21T22:27:21.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:21 smithi194 ceph-mon[98288]: pgmap v344: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:21.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:21 smithi134 ceph-mon[97873]: pgmap v344: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:21 smithi093 ceph-mon[96579]: pgmap v344: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 289 op/s 2023-07-21T22:27:22.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 645731 6.38 MB/sec execute 479 sec latency 26.276 ms 2023-07-21T22:27:23.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 647262 6.38 MB/sec execute 480 sec latency 16.740 ms 2023-07-21T22:27:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:23 smithi194 ceph-mon[98288]: pgmap v345: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2023-07-21T22:27:23.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:23 smithi134 ceph-mon[97873]: pgmap v345: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2023-07-21T22:27:23.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:23 smithi093 ceph-mon[96579]: pgmap v345: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2023-07-21T22:27:24.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 648952 6.38 MB/sec execute 481 sec latency 57.142 ms 2023-07-21T22:27:25.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 650484 6.39 MB/sec execute 482 sec latency 19.617 ms 2023-07-21T22:27:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:25 smithi194 ceph-mon[98288]: pgmap v346: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:25 smithi134 ceph-mon[97873]: pgmap v346: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:25 smithi093 ceph-mon[96579]: pgmap v346: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:26.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 652327 6.39 MB/sec execute 483 sec latency 32.986 ms 2023-07-21T22:27:27.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 653887 6.40 MB/sec execute 484 sec latency 20.379 ms 2023-07-21T22:27:27.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:27 smithi194 ceph-mon[98288]: pgmap v347: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:27.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:27 smithi134 ceph-mon[97873]: pgmap v347: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:27 smithi093 ceph-mon[96579]: pgmap v347: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 247 op/s 2023-07-21T22:27:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 -- 172.21.15.93:0/763886419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920100d70 msgr2=0x7ff920101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 --2- 172.21.15.93:0/763886419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920100d70 0x7ff920101140 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff910009990 tx=0x7ff910009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 -- 172.21.15.93:0/763886419 shutdown_connections 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 --2- 172.21.15.93:0/763886419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff92010a0b0 0x7ff92010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 --2- 172.21.15.93:0/763886419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920100d70 0x7ff920101140 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 --2- 172.21.15.93:0/763886419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff920101680 0x7ff920109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 -- 172.21.15.93:0/763886419 >> 172.21.15.93:0/763886419 conn(0x7ff9200fa500 msgr2=0x7ff9200fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 -- 172.21.15.93:0/763886419 shutdown_connections 2023-07-21T22:27:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.737+0000 7ff92516f700 1 -- 172.21.15.93:0/763886419 wait complete. 2023-07-21T22:27:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 Processor -- start 2023-07-21T22:27:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 -- start start 2023-07-21T22:27:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920100d70 0x7ff9200feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff92010a0b0 0x7ff9201a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4350 con 0x7ff920101680 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4490 con 0x7ff920100d70 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff92516f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4600 con 0x7ff92010a0b0 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47782/0 (socket says 172.21.15.93:47782) 2023-07-21T22:27:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 -- 172.21.15.93:0/3038314247 learned_addr learned my addr 172.21.15.93:0/3038314247 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91f59e700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff92010a0b0 0x7ff9201a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920100d70 0x7ff9200feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 -- 172.21.15.93:0/3038314247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff92010a0b0 msgr2=0x7ff9201a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff92010a0b0 0x7ff9201a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 -- 172.21.15.93:0/3038314247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920100d70 msgr2=0x7ff9200feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920100d70 0x7ff9200feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.738+0000 7ff91e59c700 1 -- 172.21.15.93:0/3038314247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100095f0 con 0x7ff920101680 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff91e59c700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff91400e9a0 tx=0x7ff91400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff907fff700 1 -- 172.21.15.93:0/3038314247 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914009790 con 0x7ff920101680 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff907fff700 1 -- 172.21.15.93:0/3038314247 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff914010430 con 0x7ff920101680 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff907fff700 1 -- 172.21.15.93:0/3038314247 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914018680 con 0x7ff920101680 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9201a48e0 con 0x7ff920101680 2023-07-21T22:27:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.739+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201a4da0 con 0x7ff920101680 2023-07-21T22:27:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.741+0000 7ff907fff700 1 -- 172.21.15.93:0/3038314247 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff9140105a0 con 0x7ff920101680 2023-07-21T22:27:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.741+0000 7ff907fff700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 0x7ff9080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.742+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 0x7ff9080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.742+0000 7ff907fff700 1 -- 172.21.15.93:0/3038314247 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff914014070 con 0x7ff920101680 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.742+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 0x7ff9080a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b2e0 tx=0x7ff910005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 Processor -- start 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 -- start start 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d040bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 0x5563d04064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d030bc50 con 0x5563d040c210 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d035e9e0 con 0x5563d040c660 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0357e70 con 0x5563d02626b0 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff91e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d040bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff91e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47790/0 (socket says 172.21.15.93:47790) 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.743+0000 7ff91e59c700 1 -- 172.21.15.93:0/741429167 learned_addr learned my addr 172.21.15.93:0/741429167 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91f59e700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 0x5563d04064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 -- 172.21.15.93:0/741429167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 msgr2=0x5563d040bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d040bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 -- 172.21.15.93:0/741429167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 msgr2=0x5563d04064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 0x5563d04064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 -- 172.21.15.93:0/741429167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563d0185d80 con 0x5563d040c210 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d040bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91f59e700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 0x5563d04064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff91e59c700 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9140a1ed0 tx=0x7ff914099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb60a907209c0b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff9057fa700 1 -- 172.21.15.93:0/741429167 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914010880 con 0x5563d040c210 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff9057fa700 1 -- 172.21.15.93:0/741429167 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9140109e0 con 0x5563d040c210 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.744+0000 7ff9057fa700 1 -- 172.21.15.93:0/741429167 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9140a1f70 con 0x5563d040c210 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- 172.21.15.93:0/741429167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 msgr2=0x5563d0405f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff9140a1ed0 tx=0x7ff914099fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- 172.21.15.93:0/741429167 shutdown_connections 2023-07-21T22:27:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c660 0x5563d04064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d040bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/741429167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c210 0x5563d0405f40 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- 172.21.15.93:0/741429167 >> 172.21.15.93:0/741429167 conn(0x5563d03fbae0 msgr2=0x5563d04010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- 172.21.15.93:0/741429167 shutdown_connections 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- 172.21.15.93:0/741429167 wait complete. 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 -- start start 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.745+0000 7ff9312f4b80 1 Processor -- start 2023-07-21T22:27:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c210 0x5563d040f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d040ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d02f7a60 con 0x5563d040c660 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d02f7760 con 0x5563d02626b0 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d01db8c0 con 0x5563d040c210 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff9312f4b80 10 client.0 _ll_register_callbacks cb 0x5563d02f5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d040ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:58956/0 (socket says 172.21.15.93:58956) 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 -- 172.21.15.93:0/128369867 learned_addr learned my addr 172.21.15.93:0/128369867 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91e59c700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c210 0x5563d040f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 -- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c210 msgr2=0x5563d040f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c210 0x5563d040f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 -- 172.21.15.93:0/128369867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 msgr2=0x5563d040ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d040ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 -- 172.21.15.93:0/128369867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92010d1e0 con 0x5563d02626b0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91f59e700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d040ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff910015000 tx=0x7ff910020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7682f51fc54bfd0f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.746+0000 7ff902ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91002d040 con 0x5563d02626b0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563d02b1490 con 0x5563d02626b0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff910004df0 con 0x5563d02626b0 2023-07-21T22:27:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff91000f460 con 0x5563d02626b0 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 10 client.26408 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7ff918011230 con 0x5563d02626b0 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563d02b1490 con 0x5563d02626b0 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff91003b820 con 0x5563d02626b0 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 10 client.26408 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 20 client.26408 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d041fb20 0x5563d0421f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 4 client.26408 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563d0350e50 con 0x5563d041fb20 2023-07-21T22:27:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.747+0000 7ff91e59c700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d041fb20 0x5563d0421f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.748+0000 7ff902ff5700 10 client.26408 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.748+0000 7ff91e59c700 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d041fb20 0x5563d0421f30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2ef42aaa881b43 server_cookie=3cb767aee713472a in_seq=0 out_seq=0 2023-07-21T22:27:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.748+0000 7ff902ff5700 1 -- 172.21.15.93:0/128369867 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5563d0350e50 con 0x5563d041fb20 2023-07-21T22:27:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.748+0000 7ff902ff5700 10 client.26408 handle_command_reply: tid=0 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.748+0000 7ff9312f4b80 1 client.26408 shutdown 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 msgr2=0x5563d0412600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff910015000 tx=0x7ff910020000 comp rx=0 tx=0).stop 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 shutdown_connections 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d041fb20 0x5563d0421f30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d040ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d02626b0 0x5563d0412600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff902ff5700 0 client.26408 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/128369867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c210 0x5563d040f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 >> 172.21.15.93:0/128369867 conn(0x5563d03fbae0 msgr2=0x5563d04003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 shutdown_connections 2023-07-21T22:27:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 1 -- 172.21.15.93:0/128369867 wait complete. 2023-07-21T22:27:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.749+0000 7ff9312f4b80 20 client.26408 trim_cache size 0 max 16384 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 Processor -- start 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 -- start start 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d040d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c660 0x5563d04121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0402a60 con 0x5563d02626b0 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d03fdc00 con 0x5563d040c210 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d041c560 con 0x5563d040c660 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47802/0 (socket says 172.21.15.93:47802) 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff91ed9d700 1 -- 172.21.15.93:0/2011423282 learned_addr learned my addr 172.21.15.93:0/2011423282 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff91e59c700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d040d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.765+0000 7ff91f59e700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c660 0x5563d04121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 -- 172.21.15.93:0/2011423282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c660 msgr2=0x5563d04121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c660 0x5563d04121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 -- 172.21.15.93:0/2011423282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 msgr2=0x5563d040d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d040d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 -- 172.21.15.93:0/2011423282 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9201a4da0 con 0x5563d02626b0 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91e59c700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d040d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098e0 tx=0x7ff91003bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d0e2419a2bf8df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff905ffb700 1 -- 172.21.15.93:0/2011423282 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91002d070 con 0x5563d02626b0 2023-07-21T22:27:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff905ffb700 1 -- 172.21.15.93:0/2011423282 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91001e040 con 0x5563d02626b0 2023-07-21T22:27:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.766+0000 7ff905ffb700 1 -- 172.21.15.93:0/2011423282 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91000f790 con 0x5563d02626b0 2023-07-21T22:27:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2011423282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 msgr2=0x5563d0416ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098e0 tx=0x7ff91003bef0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2011423282 shutdown_connections 2023-07-21T22:27:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d040c660 0x5563d04121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d02626b0 0x5563d0416ba0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2011423282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d040d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2011423282 >> 172.21.15.93:0/2011423282 conn(0x5563d03fbae0 msgr2=0x5563d040caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2011423282 shutdown_connections 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2011423282 wait complete. 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- start start 2023-07-21T22:27:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 Processor -- start 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d04131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d04136e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0413e70 con 0x5563d040c660 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0413fe0 con 0x5563d040c210 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d0472ed0 con 0x5563d02626b0 2023-07-21T22:27:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff9312f4b80 10 client.0 _ll_register_callbacks cb 0x7ff910005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.767+0000 7ff91f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91ed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d04131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47812/0 (socket says 172.21.15.93:47812) 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 -- 172.21.15.93:0/2192699300 learned_addr learned my addr 172.21.15.93:0/2192699300 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91e59c700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d04136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 -- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 msgr2=0x5563d04131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d04131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 -- 172.21.15.93:0/2192699300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 msgr2=0x5563d04136e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d04136e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 -- 172.21.15.93:0/2192699300 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563d0185d80 con 0x5563d040c660 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff91f59e700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9180040f0 tx=0x7ff918004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf7ef2ff51218cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff902ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91801c070 con 0x5563d040c660 2023-07-21T22:27:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff918004bb0 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9140109e0 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff91800e040 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff9312f4b80 10 client.17019 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7ff918011230 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.768+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91802d020 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff918029460 con 0x5563d040c660 2023-07-21T22:27:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 20 client.17019 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d04745c0 0x5563d04769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 4 client.17019 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5563d0312ff0 con 0x5563d04745c0 2023-07-21T22:27:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d04745c0 0x5563d04769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff902ff5700 10 client.17019 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.769+0000 7ff91ed9d700 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d04745c0 0x5563d04769a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e3734c23bf8e05 server_cookie=fdd1d2203d313563 in_seq=0 out_seq=0 2023-07-21T22:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.819+0000 7ff902ff5700 1 -- 172.21.15.93:0/2192699300 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5563d0312ff0 con 0x5563d04745c0 2023-07-21T22:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.819+0000 7ff902ff5700 10 client.17019 handle_command_reply: tid=0 2023-07-21T22:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.819+0000 7ff9312f4b80 1 client.17019 shutdown 2023-07-21T22:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.819+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 msgr2=0x5563d0472990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.819+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff9180040f0 tx=0x7ff918004400 comp rx=0 tx=0).stop 2023-07-21T22:27:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 shutdown_connections 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563d04745c0 0x5563d04769a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563d040c660 0x5563d0472990 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563d02626b0 0x5563d04131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 --2- 172.21.15.93:0/2192699300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563d040c210 0x5563d04136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 >> 172.21.15.93:0/2192699300 conn(0x5563d03fbae0 msgr2=0x5563d040d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff902ff5700 0 client.17019 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 shutdown_connections 2023-07-21T22:27:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 1 -- 172.21.15.93:0/2192699300 wait complete. 2023-07-21T22:27:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.820+0000 7ff9312f4b80 20 client.17019 trim_cache size 0 max 16384 2023-07-21T22:27:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:27:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:27:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:27:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:27:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 msgr2=0x7ff9080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 0x7ff9080a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff91000b2e0 tx=0x7ff910005fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 msgr2=0x7ff9200ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff91400e9a0 tx=0x7ff91400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 shutdown_connections 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff92010a0b0 0x7ff9201a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920100d70 0x7ff9200feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff9080a12c0 0x7ff9080a3770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 --2- 172.21.15.93:0/3038314247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff920101680 0x7ff9200ff0e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 >> 172.21.15.93:0/3038314247 conn(0x7ff9200fa500 msgr2=0x7ff92010c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 shutdown_connections 2023-07-21T22:27:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:27.823+0000 7ff92516f700 1 -- 172.21.15.93:0/3038314247 wait complete. 2023-07-21T22:27:28.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 655531 6.40 MB/sec execute 485 sec latency 34.334 ms 2023-07-21T22:27:29.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 657293 6.42 MB/sec execute 486 sec latency 32.059 ms 2023-07-21T22:27:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:29 smithi194 ceph-mon[98288]: pgmap v348: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2023-07-21T22:27:29.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:29 smithi134 ceph-mon[97873]: pgmap v348: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2023-07-21T22:27:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:29 smithi093 ceph-mon[96579]: pgmap v348: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 361 op/s 2023-07-21T22:27:30.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 658632 6.42 MB/sec execute 487 sec latency 18.990 ms 2023-07-21T22:27:31.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 660739 6.43 MB/sec execute 488 sec latency 15.018 ms 2023-07-21T22:27:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:31 smithi194 ceph-mon[98288]: pgmap v349: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2023-07-21T22:27:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:31 smithi134 ceph-mon[97873]: pgmap v349: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2023-07-21T22:27:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:31 smithi093 ceph-mon[96579]: pgmap v349: 97 pgs: 97 active+clean; 299 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2023-07-21T22:27:32.342 INFO:tasks.workunit.client.0.smithi093.stdout: 1 661955 6.43 MB/sec execute 489 sec latency 20.195 ms 2023-07-21T22:27:33.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 663970 6.44 MB/sec execute 490 sec latency 17.220 ms 2023-07-21T22:27:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:33 smithi194 ceph-mon[98288]: pgmap v350: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 360 op/s 2023-07-21T22:27:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:33 smithi134 ceph-mon[97873]: pgmap v350: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 360 op/s 2023-07-21T22:27:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:33 smithi093 ceph-mon[96579]: pgmap v350: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 360 op/s 2023-07-21T22:27:34.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 665053 6.43 MB/sec execute 491 sec latency 24.184 ms 2023-07-21T22:27:35.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 666986 6.44 MB/sec execute 492 sec latency 19.484 ms 2023-07-21T22:27:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:35 smithi194 ceph-mon[98288]: pgmap v351: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:35 smithi134 ceph-mon[97873]: pgmap v351: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:35 smithi093 ceph-mon[96579]: pgmap v351: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:36.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 668403 6.44 MB/sec execute 493 sec latency 11.753 ms 2023-07-21T22:27:37.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 670230 6.45 MB/sec execute 494 sec latency 40.904 ms 2023-07-21T22:27:37.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:37 smithi194 ceph-mon[98288]: pgmap v352: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:37 smithi134 ceph-mon[97873]: pgmap v352: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:37 smithi093 ceph-mon[96579]: pgmap v352: 97 pgs: 97 active+clean; 298 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 240 op/s 2023-07-21T22:27:38.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 671704 6.46 MB/sec execute 495 sec latency 35.892 ms 2023-07-21T22:27:39.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 673578 6.46 MB/sec execute 496 sec latency 51.129 ms 2023-07-21T22:27:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:39 smithi194 ceph-mon[98288]: pgmap v353: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2023-07-21T22:27:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:39 smithi134 ceph-mon[97873]: pgmap v353: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2023-07-21T22:27:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:39 smithi093 ceph-mon[96579]: pgmap v353: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2023-07-21T22:27:40.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 675158 6.47 MB/sec execute 497 sec latency 11.107 ms 2023-07-21T22:27:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 -- 172.21.15.93:0/303880741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 msgr2=0x7f375c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 --2- 172.21.15.93:0/303880741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c104e00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f374c009990 tx=0x7f374c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 -- 172.21.15.93:0/303880741 shutdown_connections 2023-07-21T22:27:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 --2- 172.21.15.93:0/303880741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 0x7f375c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 --2- 172.21.15.93:0/303880741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c104e00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 --2- 172.21.15.93:0/303880741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 0x7f375c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 -- 172.21.15.93:0/303880741 >> 172.21.15.93:0/303880741 conn(0x7f375c074760 msgr2=0x7f375c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 -- 172.21.15.93:0/303880741 shutdown_connections 2023-07-21T22:27:40.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 -- 172.21.15.93:0/303880741 wait complete. 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.940+0000 7f3760d94700 1 Processor -- start 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 -- start start 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 0x7f375c196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 0x7f375c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a2050 con 0x7f375c104a30 2023-07-21T22:27:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a21c0 con 0x7f375c0ff3e0 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3760d94700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a2380 con 0x7f375c0fea30 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 0x7f375c196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47826/0 (socket says 172.21.15.93:47826) 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 -- 172.21.15.93:0/4177616935 learned_addr learned my addr 172.21.15.93:0/4177616935 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f3759d9b700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 0x7f375c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 -- 172.21.15.93:0/4177616935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 msgr2=0x7f375c196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 0x7f375c196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 -- 172.21.15.93:0/4177616935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 msgr2=0x7f375c197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 0x7f375c197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 -- 172.21.15.93:0/4177616935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0095f0 con 0x7f375c104a30 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f375ad9d700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3754007ee0 tx=0x7f375400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.941+0000 7f374b7fe700 1 -- 172.21.15.93:0/4177616935 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754017070 con 0x7f375c104a30 2023-07-21T22:27:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.942+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c1a2660 con 0x7f375c104a30 2023-07-21T22:27:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.942+0000 7f374b7fe700 1 -- 172.21.15.93:0/4177616935 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f375400f040 con 0x7f375c104a30 2023-07-21T22:27:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.942+0000 7f374b7fe700 1 -- 172.21.15.93:0/4177616935 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754005580 con 0x7f375c104a30 2023-07-21T22:27:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.942+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c1a2b20 con 0x7f375c104a30 2023-07-21T22:27:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.945+0000 7f374b7fe700 1 -- 172.21.15.93:0/4177616935 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f375400db20 con 0x7f375c104a30 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.945+0000 7f376cf19b80 1 Processor -- start 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.945+0000 7f376cf19b80 1 -- start start 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570210 0x55c6a6569f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a656a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a646fc50 con 0x55c6a63c66b0 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a64c29e0 con 0x55c6a6570210 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f376cf19b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a64bbe70 con 0x55c6a6570660 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f374b7fe700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 0x7f37440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47828/0 (socket says 172.21.15.93:47828) 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 -- 172.21.15.93:0/2995436394 learned_addr learned my addr 172.21.15.93:0/2995436394 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a656a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f375a59c700 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570210 0x55c6a6569f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f374b7fe700 1 -- 172.21.15.93:0/4177616935 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f37540cc450 con 0x7f375c104a30 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 -- 172.21.15.93:0/2995436394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 msgr2=0x55c6a656a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a656a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 -- 172.21.15.93:0/2995436394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570210 msgr2=0x55c6a6569f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570210 0x55c6a6569f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 -- 172.21.15.93:0/2995436394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a62e9d80 con 0x55c6a63c66b0 2023-07-21T22:27:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f375ad9d700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 0x7f37440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f37500125d0 tx=0x7f375000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998b1b011109ba54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.946+0000 7f3748ff9700 1 -- 172.21.15.93:0/2995436394 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750004030 con 0x55c6a63c66b0 2023-07-21T22:27:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f3748ff9700 1 -- 172.21.15.93:0/2995436394 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3750004830 con 0x55c6a63c66b0 2023-07-21T22:27:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f3748ff9700 1 -- 172.21.15.93:0/2995436394 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750005260 con 0x55c6a63c66b0 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 -- 172.21.15.93:0/2995436394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 msgr2=0x55c6a656fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f375ad9d700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 0x7f37440a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3754095a20 tx=0x7f375400b040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f37500125d0 tx=0x7f375000b8c0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 -- 172.21.15.93:0/2995436394 shutdown_connections 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a656a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a656fcd0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2995436394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570210 0x55c6a6569f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 -- 172.21.15.93:0/2995436394 >> 172.21.15.93:0/2995436394 conn(0x55c6a655fae0 msgr2=0x55c6a65650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 -- 172.21.15.93:0/2995436394 shutdown_connections 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.947+0000 7f376cf19b80 1 -- 172.21.15.93:0/2995436394 wait complete. 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 -- start start 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 Processor -- start 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 0x55c6a6576640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a6574070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a645ba60 con 0x55c6a6570210 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a645b760 con 0x55c6a63c66b0 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a633f8c0 con 0x55c6a6570660 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f376cf19b80 10 client.0 _ll_register_callbacks cb 0x55c6a6459700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f375a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f375a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47838/0 (socket says 172.21.15.93:47838) 2023-07-21T22:27:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 0x55c6a6576640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f375a59c700 1 -- 172.21.15.93:0/2248512615 learned_addr learned my addr 172.21.15.93:0/2248512615 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a6574070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 -- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 msgr2=0x55c6a6574070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a6574070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 -- 172.21.15.93:0/2248512615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 msgr2=0x55c6a6573930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.948+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3759d9b700 1 -- 172.21.15.93:0/2248512615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375000bde0 con 0x55c6a63c66b0 2023-07-21T22:27:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f375a59c700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 0x55c6a6576640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f375000b470 tx=0x7f3750005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f092997a903f50 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750026030 con 0x55c6a63c66b0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a6415490 con 0x55c6a63c66b0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3750027070 con 0x55c6a63c66b0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3750005210 con 0x55c6a63c66b0 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 10 client.26426 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55c6a62d2c00 con 0x55c6a63c66b0 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6a6415490 con 0x55c6a63c66b0 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f375002d020 con 0x55c6a63c66b0 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 10 client.26426 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 20 client.26426 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a6583b70 0x55c6a6585f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 4 client.26426 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.949+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a64b4e50 con 0x55c6a6583b70 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.950+0000 7f375a59c700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a6583b70 0x55c6a6585f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.950+0000 7f3741ffb700 10 client.26426 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.950+0000 7f375a59c700 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a6583b70 0x55c6a6585f80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d256bc0caf50295b server_cookie=d9004087db1bae64 in_seq=0 out_seq=0 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f3741ffb700 1 -- 172.21.15.93:0/2248512615 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6a64b4e50 con 0x55c6a6583b70 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f3741ffb700 10 client.26426 handle_command_reply: tid=0 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 client.26426 shutdown 2023-07-21T22:27:41.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 msgr2=0x55c6a6576640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 0x55c6a6576640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f375000b470 tx=0x7f3750005950 comp rx=0 tx=0).stop 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 shutdown_connections 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a6583b70 0x55c6a6585f80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570660 0x55c6a6574070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f3741ffb700 0 client.26426 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a63c66b0 0x55c6a6576640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2248512615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6573930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 >> 172.21.15.93:0/2248512615 conn(0x55c6a655fae0 msgr2=0x55c6a65643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 shutdown_connections 2023-07-21T22:27:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 1 -- 172.21.15.93:0/2248512615 wait complete. 2023-07-21T22:27:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.951+0000 7f376cf19b80 20 client.26426 trim_cache size 0 max 16384 2023-07-21T22:27:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.966+0000 7f376cf19b80 1 Processor -- start 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.966+0000 7f376cf19b80 1 -- start start 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a63c66b0 0x55c6a6579200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a6564b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a65748b0 con 0x55c6a6570210 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a656f920 con 0x55c6a6570660 2023-07-21T22:27:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f376cf19b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a656aab0 con 0x55c6a63c66b0 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47844/0 (socket says 172.21.15.93:47844) 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 -- 172.21.15.93:0/3696294222 learned_addr learned my addr 172.21.15.93:0/3696294222 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f3759d9b700 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a63c66b0 0x55c6a6579200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f3759d9b700 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a6564b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 -- 172.21.15.93:0/3696294222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a63c66b0 msgr2=0x55c6a6579200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a63c66b0 0x55c6a6579200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 -- 172.21.15.93:0/3696294222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 msgr2=0x55c6a6564b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a6564b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 -- 172.21.15.93:0/3696294222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a63101f0 con 0x55c6a6570210 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f375a59c700 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b061116027f5e7dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.967+0000 7f37497fa700 1 -- 172.21.15.93:0/3696294222 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c01d040 con 0x55c6a6570210 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f37497fa700 1 -- 172.21.15.93:0/3696294222 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f374c01e040 con 0x55c6a6570210 2023-07-21T22:27:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f37497fa700 1 -- 172.21.15.93:0/3696294222 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c005370 con 0x55c6a6570210 2023-07-21T22:27:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 -- 172.21.15.93:0/3696294222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 msgr2=0x55c6a6572ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c005960 comp rx=0 tx=0).stop 2023-07-21T22:27:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 -- 172.21.15.93:0/3696294222 shutdown_connections 2023-07-21T22:27:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a6564b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a63c66b0 0x55c6a6579200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 --2- 172.21.15.93:0/3696294222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a6570210 0x55c6a6572ea0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 -- 172.21.15.93:0/3696294222 >> 172.21.15.93:0/3696294222 conn(0x55c6a655fae0 msgr2=0x55c6a656da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 -- 172.21.15.93:0/3696294222 shutdown_connections 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.968+0000 7f376cf19b80 1 -- 172.21.15.93:0/3696294222 wait complete. 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 -- start start 2023-07-21T22:27:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 Processor -- start 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 0x55c6a65cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a65cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a65d6af0 con 0x55c6a63c66b0 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a65d6c60 con 0x55c6a6570660 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a65d6e20 con 0x55c6a6570210 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f376cf19b80 10 client.0 _ll_register_callbacks cb 0x7f37500049e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47850/0 (socket says 172.21.15.93:47850) 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 -- 172.21.15.93:0/2690179446 learned_addr learned my addr 172.21.15.93:0/2690179446 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f375a59c700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 0x55c6a65cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a65cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 -- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 msgr2=0x55c6a65cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 0x55c6a65cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 -- 172.21.15.93:0/2690179446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 msgr2=0x55c6a65cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a65cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f3759d9b700 1 -- 172.21.15.93:0/2690179446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c10c160 con 0x55c6a63c66b0 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.969+0000 7f375a59c700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 0x55c6a65cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3759d9b700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3750012450 tx=0x7f3750005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db587b1faebbcc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750026040 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3750000e40 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3750004830 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f375001fbd0 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 10 client.17037 fetch_fsmap learned FSMap version 55 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55c6a62e8250 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3750004830 con 0x55c6a63c66b0 2023-07-21T22:27:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 55) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3750041070 con 0x55c6a63c66b0 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 20 client.17037 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a65d81a0 0x55c6a65da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 4 client.17037 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.970+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6a6476ff0 con 0x55c6a65d81a0 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.971+0000 7f375a59c700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a65d81a0 0x55c6a65da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.971+0000 7f3741ffb700 10 client.17037 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:41.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:40.971+0000 7f375a59c700 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a65d81a0 0x55c6a65da5b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881c98517a381338 server_cookie=62defe16307c6205 in_seq=0 out_seq=0 2023-07-21T22:27:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.061+0000 7f3741ffb700 1 -- 172.21.15.93:0/2690179446 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6a6476ff0 con 0x55c6a65d81a0 2023-07-21T22:27:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.061+0000 7f3741ffb700 10 client.17037 handle_command_reply: tid=0 2023-07-21T22:27:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.061+0000 7f376cf19b80 1 client.17037 shutdown 2023-07-21T22:27:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 msgr2=0x55c6a65cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3750012450 tx=0x7f3750005950 comp rx=0 tx=0).stop 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 shutdown_connections 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c6a65d81a0 0x55c6a65da5b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c6a6570660 0x55c6a65cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6a63c66b0 0x55c6a65cb980 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 --2- 172.21.15.93:0/2690179446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c6a6570210 0x55c6a65cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 >> 172.21.15.93:0/2690179446 conn(0x55c6a655fae0 msgr2=0x55c6a656e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f3741ffb700 0 client.17037 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 shutdown_connections 2023-07-21T22:27:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.062+0000 7f376cf19b80 1 -- 172.21.15.93:0/2690179446 wait complete. 2023-07-21T22:27:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f376cf19b80 20 client.17037 trim_cache size 0 max 16384 2023-07-21T22:27:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:27:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:27:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:27:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:27:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 msgr2=0x7f37440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 0x7f37440a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3754095a20 tx=0x7f375400b040 comp rx=0 tx=0).stop 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 msgr2=0x7f375c1a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3754007ee0 tx=0x7f375400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 shutdown_connections 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37440a12c0 0x7f37440a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f375c104a30 0x7f375c1a1a80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f375c0fea30 0x7f375c196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 --2- 172.21.15.93:0/4177616935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f375c0ff3e0 0x7f375c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.063+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 >> 172.21.15.93:0/4177616935 conn(0x7f375c074760 msgr2=0x7f375c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.064+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 shutdown_connections 2023-07-21T22:27:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:41.064+0000 7f3760d94700 1 -- 172.21.15.93:0/4177616935 wait complete. 2023-07-21T22:27:41.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 676458 6.47 MB/sec execute 498 sec latency 61.170 ms 2023-07-21T22:27:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:41 smithi194 ceph-mon[98288]: pgmap v354: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 232 op/s 2023-07-21T22:27:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:41 smithi134 ceph-mon[97873]: pgmap v354: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 232 op/s 2023-07-21T22:27:41.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:27:41 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:27:41.631 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:27:41 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:27:41.632 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:27:41 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/720313430 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602dbc7b1e0 con 0x5602dbc51000 2023-07-21T22:27:41.632 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:27:41 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:27:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:41 smithi093 ceph-mon[96579]: pgmap v354: 97 pgs: 97 active+clean; 305 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 232 op/s 2023-07-21T22:27:42.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 678533 6.48 MB/sec execute 499 sec latency 16.314 ms 2023-07-21T22:27:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:42 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:27:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:42 smithi194 ceph-mon[98288]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:27:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:42 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/2243907757,v1:172.21.15.134:6837/2243907757] up:boot 2023-07-21T22:27:42.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:42 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:42.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:42 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:42 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:27:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:42 smithi134 ceph-mon[97873]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:27:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:42 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/2243907757,v1:172.21.15.134:6837/2243907757] up:boot 2023-07-21T22:27:42.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:42 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:42.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:42 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:42 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:27:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:42 smithi093 ceph-mon[96579]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:27:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:42 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/2243907757,v1:172.21.15.134:6837/2243907757] up:boot 2023-07-21T22:27:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:42 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:42.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:42 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:43.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 679541 6.47 MB/sec execute 500 sec latency 22.322 ms 2023-07-21T22:27:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:43 smithi194 ceph-mon[98288]: pgmap v355: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 355 op/s 2023-07-21T22:27:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:27:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:43 smithi134 ceph-mon[97873]: pgmap v355: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 355 op/s 2023-07-21T22:27:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:27:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:43 smithi093 ceph-mon[96579]: pgmap v355: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 355 op/s 2023-07-21T22:27:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:27:44.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 681482 6.48 MB/sec execute 501 sec latency 23.701 ms 2023-07-21T22:27:45.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 682770 6.48 MB/sec execute 502 sec latency 16.353 ms 2023-07-21T22:27:45.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:45 smithi194 ceph-mon[98288]: pgmap v357: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:45.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:45 smithi134 ceph-mon[97873]: pgmap v357: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:45 smithi093 ceph-mon[96579]: pgmap v357: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:46.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 684736 6.49 MB/sec execute 503 sec latency 17.323 ms 2023-07-21T22:27:47.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 686189 6.49 MB/sec execute 504 sec latency 11.766 ms 2023-07-21T22:27:47.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:47 smithi194 ceph-mon[98288]: pgmap v358: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:47.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:47 smithi134 ceph-mon[97873]: pgmap v358: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:47 smithi093 ceph-mon[96579]: pgmap v358: 97 pgs: 97 active+clean; 301 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 274 op/s 2023-07-21T22:27:48.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 688121 6.50 MB/sec execute 505 sec latency 18.226 ms 2023-07-21T22:27:49.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 689506 6.50 MB/sec execute 506 sec latency 49.036 ms 2023-07-21T22:27:49.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:49 smithi194 ceph-mon[98288]: pgmap v359: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:49 smithi134 ceph-mon[97873]: pgmap v359: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:49 smithi093 ceph-mon[96579]: pgmap v359: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:50.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 691366 6.51 MB/sec execute 507 sec latency 20.512 ms 2023-07-21T22:27:51.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 693000 6.52 MB/sec execute 508 sec latency 14.379 ms 2023-07-21T22:27:51.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:51 smithi194 ceph-mon[98288]: pgmap v360: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:51.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:51 smithi134 ceph-mon[97873]: pgmap v360: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:51.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:51 smithi093 ceph-mon[96579]: pgmap v360: 97 pgs: 97 active+clean; 308 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2023-07-21T22:27:52.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 694900 6.52 MB/sec execute 509 sec latency 16.242 ms 2023-07-21T22:27:53.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 696480 6.53 MB/sec execute 510 sec latency 12.915 ms 2023-07-21T22:27:53.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:53 smithi194 ceph-mon[98288]: pgmap v361: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:27:53.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:53 smithi134 ceph-mon[97873]: pgmap v361: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:27:53.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:53 smithi093 ceph-mon[96579]: pgmap v361: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 268 op/s 2023-07-21T22:27:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 -- 172.21.15.93:0/2904539106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540ff3e0 msgr2=0x7f375410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 --2- 172.21.15.93:0/2904539106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540ff3e0 0x7f375410b860 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f374c00b0a0 tx=0x7f374c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 -- 172.21.15.93:0/2904539106 shutdown_connections 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 --2- 172.21.15.93:0/2904539106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540ff3e0 0x7f375410b860 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 --2- 172.21.15.93:0/2904539106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3754104a30 0x7f3754104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 --2- 172.21.15.93:0/2904539106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37540fea30 0x7f37540feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 -- 172.21.15.93:0/2904539106 >> 172.21.15.93:0/2904539106 conn(0x7f3754074760 msgr2=0x7f3754074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 -- 172.21.15.93:0/2904539106 shutdown_connections 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 -- 172.21.15.93:0/2904539106 wait complete. 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.193+0000 7f375c62e700 1 Processor -- start 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 -- start start 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 0x7f3754100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3754104a30 0x7f37541a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a43d0 con 0x7f37540fea30 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a4510 con 0x7f3754104a30 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a4680 con 0x7f37540ff3e0 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f3759bc9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 0x7f3754100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47864/0 (socket says 172.21.15.93:47864) 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 -- 172.21.15.93:0/3185946267 learned_addr learned my addr 172.21.15.93:0/3185946267 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375abcb700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3754104a30 0x7f37541a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 -- 172.21.15.93:0/3185946267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 msgr2=0x7f3754100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 0x7f3754100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 -- 172.21.15.93:0/3185946267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3754104a30 msgr2=0x7f37541a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3754104a30 0x7f37541a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 -- 172.21.15.93:0/3185946267 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c009d00 con 0x7f37540fea30 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f3759bc9700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 0x7f3754100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f375000e970 tx=0x7f375000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f374b7fe700 1 -- 172.21.15.93:0/3185946267 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750009790 con 0x7f37540fea30 2023-07-21T22:27:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f374b7fe700 1 -- 172.21.15.93:0/3185946267 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f375000c7e0 con 0x7f37540fea30 2023-07-21T22:27:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f374b7fe700 1 -- 172.21.15.93:0/3185946267 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3750003c70 con 0x7f37540fea30 2023-07-21T22:27:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.194+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37541a4960 con 0x7f37540fea30 2023-07-21T22:27:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.195+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37541a4e70 con 0x7f37540fea30 2023-07-21T22:27:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.197+0000 7f374b7fe700 1 -- 172.21.15.93:0/3185946267 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f375000cc50 con 0x7f37540fea30 2023-07-21T22:27:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.197+0000 7f374b7fe700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 0x7f37400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.198+0000 7f3759bc9700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 0x7f37400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.198+0000 7f374b7fe700 1 -- 172.21.15.93:0/3185946267 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f3750014070 con 0x7f37540fea30 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.198+0000 7f3759bc9700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 0x7f37400a37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3754102030 tx=0x7f3744009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 Processor -- start 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 -- start start 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85660 0x56450dc7f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450db84c50 con 0x56450dadb6b0 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dbd79e0 con 0x56450dc85660 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f37687b3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dbd0e70 con 0x56450dc85210 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47872/0 (socket says 172.21.15.93:47872) 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f375a3ca700 1 -- 172.21.15.93:0/3768981307 learned_addr learned my addr 172.21.15.93:0/3768981307 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f3759bc9700 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.199+0000 7f375abcb700 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85660 0x56450dc7f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 -- 172.21.15.93:0/3768981307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 msgr2=0x56450dc7ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc7ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 -- 172.21.15.93:0/3768981307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85660 msgr2=0x56450dc7f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85660 0x56450dc7f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 -- 172.21.15.93:0/3768981307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450d9fed80 con 0x56450dadb6b0 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f375a3ca700 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f37500a1df0 tx=0x7f3750099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdda619a21151618 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f3748ff9700 1 -- 172.21.15.93:0/3768981307 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375009df00 con 0x56450dadb6b0 2023-07-21T22:27:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f3748ff9700 1 -- 172.21.15.93:0/3768981307 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37500a2320 con 0x56450dadb6b0 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.200+0000 7f3748ff9700 1 -- 172.21.15.93:0/3768981307 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375009a390 con 0x56450dadb6b0 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- 172.21.15.93:0/3768981307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 msgr2=0x56450dc84cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f37500a1df0 tx=0x7f3750099920 comp rx=0 tx=0).stop 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- 172.21.15.93:0/3768981307 shutdown_connections 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85660 0x56450dc7f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc84cd0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- 172.21.15.93:0/3768981307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- 172.21.15.93:0/3768981307 >> 172.21.15.93:0/3768981307 conn(0x56450dc74ae0 msgr2=0x56450dc7a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- 172.21.15.93:0/3768981307 shutdown_connections 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- 172.21.15.93:0/3768981307 wait complete. 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- start start 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 Processor -- start 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc88750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450db70a60 con 0x56450dc85660 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450db70760 con 0x56450dadb6b0 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450da548c0 con 0x56450dc85210 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.201+0000 7f37687b3b80 10 client.0 _ll_register_callbacks cb 0x56450db6e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47882/0 (socket says 172.21.15.93:47882) 2023-07-21T22:27:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 -- 172.21.15.93:0/1670634626 learned_addr learned my addr 172.21.15.93:0/1670634626 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f3759bc9700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc88750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 -- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 msgr2=0x56450dc88750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc88750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 -- 172.21.15.93:0/1670634626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 msgr2=0x56450dc8b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 -- 172.21.15.93:0/1670634626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37541a4e70 con 0x56450dc85660 2023-07-21T22:27:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f375abcb700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f374c00bf60 tx=0x7f374c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab328866d86adf92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f373dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00e040 con 0x56450dc85660 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f374c0076e0 con 0x56450dc85660 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450db2a490 con 0x56450dc85660 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374c004e80 con 0x56450dc85660 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f37687b3b80 10 client.17055 fetch_fsmap learned FSMap version 56 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56450d9e7c00 con 0x56450dc85660 2023-07-21T22:27:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.202+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56450db2a490 con 0x56450dc85660 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 56) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f374c025030 con 0x56450dc85660 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 10 client.17055 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 20 client.17055 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dc9af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 4 client.17055 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56450dbc9e50 con 0x56450dc98b00 2023-07-21T22:27:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dc9af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f373dffb700 10 client.17055 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.203+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dc9af10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3062ce5a8c5d3c9 server_cookie=7ff677a61c7a9522 in_seq=0 out_seq=0 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f373dffb700 1 -- 172.21.15.93:0/1670634626 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56450dbc9e50 con 0x56450dc98b00 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f373dffb700 10 client.17055 handle_command_reply: tid=0 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 client.17055 shutdown 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 msgr2=0x56450dc88d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f374c00bf60 tx=0x7f374c00bf90 comp rx=0 tx=0).stop 2023-07-21T22:27:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 shutdown_connections 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dc9af10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85660 0x56450dc88d30 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f373dffb700 0 client.17055 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1670634626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85210 0x56450dc88750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 >> 172.21.15.93:0/1670634626 conn(0x56450dc74ae0 msgr2=0x56450dc793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 shutdown_connections 2023-07-21T22:27:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 1 -- 172.21.15.93:0/1670634626 wait complete. 2023-07-21T22:27:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.204+0000 7f37687b3b80 20 client.17055 trim_cache size 0 max 16384 2023-07-21T22:27:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 Processor -- start 2023-07-21T22:27:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 -- start start 2023-07-21T22:27:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 0x56450dc8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dbc9e50 con 0x56450dadb6b0 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450db2a490 con 0x56450dc85210 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f37687b3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dc850d0 con 0x56450dc85660 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f375abcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f375a3ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47888/0 (socket says 172.21.15.93:47888) 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f375a3ca700 1 -- 172.21.15.93:0/1105586867 learned_addr learned my addr 172.21.15.93:0/1105586867 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f3759bc9700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 0x56450dc8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.220+0000 7f375a3ca700 1 -- 172.21.15.93:0/1105586867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 msgr2=0x56450dc8b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375a3ca700 1 -- 172.21.15.93:0/1105586867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 msgr2=0x56450dc8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 0x56450dc8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375a3ca700 1 -- 172.21.15.93:0/1105586867 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450dc9fb40 con 0x56450dadb6b0 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375abcb700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f3759bc9700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 0x56450dc8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f375a3ca700 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f37500963d0 tx=0x7f3750096300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c66efbc42a1d6d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f37497fa700 1 -- 172.21.15.93:0/1105586867 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500b82b0 con 0x56450dadb6b0 2023-07-21T22:27:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f37497fa700 1 -- 172.21.15.93:0/1105586867 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37500ca7d0 con 0x56450dadb6b0 2023-07-21T22:27:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.221+0000 7f37497fa700 1 -- 172.21.15.93:0/1105586867 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37500993c0 con 0x56450dadb6b0 2023-07-21T22:27:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- 172.21.15.93:0/1105586867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 msgr2=0x56450dc862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f37500963d0 tx=0x7f3750096300 comp rx=0 tx=0).stop 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- 172.21.15.93:0/1105586867 shutdown_connections 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dadb6b0 0x56450dc862c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 --2- 172.21.15.93:0/1105586867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dc85210 0x56450dc8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- 172.21.15.93:0/1105586867 >> 172.21.15.93:0/1105586867 conn(0x56450dc74ae0 msgr2=0x56450dc83830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- 172.21.15.93:0/1105586867 shutdown_connections 2023-07-21T22:27:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- 172.21.15.93:0/1105586867 wait complete. 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 -- start start 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.222+0000 7f37687b3b80 1 Processor -- start 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dcebba0 con 0x56450dc85210 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dcebd10 con 0x56450dadb6b0 2023-07-21T22:27:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450dcebed0 con 0x56450dc85660 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 10 client.0 _ll_register_callbacks cb 0x7f374c00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47892/0 (socket says 172.21.15.93:47892) 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 -- 172.21.15.93:0/163435832 learned_addr learned my addr 172.21.15.93:0/163435832 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f375a3ca700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f375abcb700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 -- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 msgr2=0x56450dc8e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 -- 172.21.15.93:0/163435832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 msgr2=0x56450dc8dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 -- 172.21.15.93:0/163435832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3744017070 con 0x56450dc85210 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f375a3ca700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f375abcb700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f3759bc9700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56450dc89d00 tx=0x7f37440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c494fd99143483 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f373dffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744025070 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f374401f610 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.223+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37500a2320 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374401f7d0 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 10 client.17061 fetch_fsmap learned FSMap version 56 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56450d9e7c00 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37500a2320 con 0x56450dc85210 2023-07-21T22:27:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 56) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f374402aa30 con 0x56450dc85210 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 20 client.17061 populate_metadata read hostname 'smithi093' 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dcef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 4 client.17061 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56450db8bff0 con 0x56450dc98b00 2023-07-21T22:27:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.224+0000 7f375a3ca700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dcef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.225+0000 7f373dffb700 10 client.17061 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.225+0000 7f375a3ca700 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dcef700 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c5e077c110b523 server_cookie=fa38362a97fb7c3b in_seq=0 out_seq=0 2023-07-21T22:27:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.273+0000 7f373dffb700 1 -- 172.21.15.93:0/163435832 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56450db8bff0 con 0x56450dc98b00 2023-07-21T22:27:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.273+0000 7f373dffb700 10 client.17061 handle_command_reply: tid=0 2023-07-21T22:27:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.273+0000 7f37687b3b80 1 client.17061 shutdown 2023-07-21T22:27:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 msgr2=0x56450dc8e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56450dc89d00 tx=0x7f37440047c0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 shutdown_connections 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56450dc98b00 0x56450dcef700 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56450dc85660 0x56450dc8e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56450dadb6b0 0x56450dc8dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f373dffb700 0 client.17061 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:27:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 --2- 172.21.15.93:0/163435832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56450dc85210 0x56450dc8e160 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.274+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 >> 172.21.15.93:0/163435832 conn(0x56450dc74ae0 msgr2=0x56450dc83830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 shutdown_connections 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f37687b3b80 1 -- 172.21.15.93:0/163435832 wait complete. 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f37687b3b80 20 client.17061 trim_cache size 0 max 16384 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:27:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:27:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:27:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:27:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 msgr2=0x7f37400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 0x7f37400a37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3754102030 tx=0x7f3744009260 comp rx=0 tx=0).stop 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 msgr2=0x7f3754100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.275+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f375000e970 tx=0x7f375000ec80 comp rx=0 tx=0).stop 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 shutdown_connections 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3754104a30 0x7f37541a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f37400a1340 0x7f37400a37f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37540fea30 0x7f3754100810 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 --2- 172.21.15.93:0/3185946267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37540ff3e0 0x7f3754100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 >> 172.21.15.93:0/3185946267 conn(0x7f3754074760 msgr2=0x7f37540fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 shutdown_connections 2023-07-21T22:27:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:27:54.276+0000 7f375c62e700 1 -- 172.21.15.93:0/3185946267 wait complete. 2023-07-21T22:27:54.343 INFO:tasks.workunit.client.0.smithi093.stdout: 1 697996 6.53 MB/sec execute 511 sec latency 44.984 ms 2023-07-21T22:27:55.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 699837 6.54 MB/sec execute 512 sec latency 32.651 ms 2023-07-21T22:27:55.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:55 smithi194 ceph-mon[98288]: pgmap v362: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2023-07-21T22:27:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:55 smithi134 ceph-mon[97873]: pgmap v362: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2023-07-21T22:27:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:55 smithi093 ceph-mon[96579]: pgmap v362: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2023-07-21T22:27:56.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 701113 6.54 MB/sec execute 513 sec latency 18.730 ms 2023-07-21T22:27:57.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 703260 6.55 MB/sec execute 514 sec latency 15.082 ms 2023-07-21T22:27:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:57 smithi194 ceph-mon[98288]: pgmap v363: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-07-21T22:27:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:57 smithi134 ceph-mon[97873]: pgmap v363: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-07-21T22:27:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:57 smithi093 ceph-mon[96579]: pgmap v363: 97 pgs: 97 active+clean; 315 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2023-07-21T22:27:58.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 704555 6.55 MB/sec execute 515 sec latency 16.248 ms 2023-07-21T22:27:59.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 706682 6.56 MB/sec execute 516 sec latency 19.092 ms 2023-07-21T22:27:59.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:27:59 smithi194 ceph-mon[98288]: pgmap v364: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 349 op/s 2023-07-21T22:27:59.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:27:59 smithi134 ceph-mon[97873]: pgmap v364: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 349 op/s 2023-07-21T22:27:59.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:27:59 smithi093 ceph-mon[96579]: pgmap v364: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 349 op/s 2023-07-21T22:28:00.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 707653 6.55 MB/sec execute 517 sec latency 62.400 ms 2023-07-21T22:28:01.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 708873 6.55 MB/sec execute 518 sec latency 57.705 ms 2023-07-21T22:28:01.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:01 smithi194 ceph-mon[98288]: pgmap v365: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2023-07-21T22:28:01.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:01 smithi134 ceph-mon[97873]: pgmap v365: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2023-07-21T22:28:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:01 smithi093 ceph-mon[96579]: pgmap v365: 97 pgs: 97 active+clean; 314 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2023-07-21T22:28:02.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 710560 6.56 MB/sec execute 519 sec latency 22.282 ms 2023-07-21T22:28:03.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 711919 6.56 MB/sec execute 520 sec latency 18.589 ms 2023-07-21T22:28:03.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:03 smithi194 ceph-mon[98288]: pgmap v366: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2023-07-21T22:28:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:03 smithi134 ceph-mon[97873]: pgmap v366: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2023-07-21T22:28:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:03 smithi093 ceph-mon[96579]: pgmap v366: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2023-07-21T22:28:04.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 713981 6.57 MB/sec execute 521 sec latency 41.216 ms 2023-07-21T22:28:05.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 714363 6.56 MB/sec execute 522 sec latency 422.193 ms 2023-07-21T22:28:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:05 smithi194 ceph-mon[98288]: pgmap v367: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2023-07-21T22:28:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:05 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:05 smithi194 ceph-mon[98288]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:28:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:05 smithi134 ceph-mon[97873]: pgmap v367: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2023-07-21T22:28:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:05 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:05 smithi134 ceph-mon[97873]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:28:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:05 smithi093 ceph-mon[96579]: pgmap v367: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2023-07-21T22:28:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:05 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:05 smithi093 ceph-mon[96579]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:28:06.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 714363 6.55 MB/sec execute 523 sec latency 1422.251 ms 2023-07-21T22:28:07.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 714363 6.54 MB/sec execute 524 sec latency 2422.305 ms 2023-07-21T22:28:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 -- 172.21.15.93:0/2785203298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 msgr2=0x7f154410c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 --2- 172.21.15.93:0/2785203298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f154410c940 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f153c00b0a0 tx=0x7f153c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 -- 172.21.15.93:0/2785203298 shutdown_connections 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 --2- 172.21.15.93:0/2785203298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f154410c940 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 --2- 172.21.15.93:0/2785203298 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 0x7f1544105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 --2- 172.21.15.93:0/2785203298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 0x7f15440fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 -- 172.21.15.93:0/2785203298 >> 172.21.15.93:0/2785203298 conn(0x7f15440fb600 msgr2=0x7f15440fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 -- 172.21.15.93:0/2785203298 shutdown_connections 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.392+0000 7f154a15e700 1 -- 172.21.15.93:0/2785203298 wait complete. 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 Processor -- start 2023-07-21T22:28:07.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 -- start start 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 0x7f1544195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 0x7f15441997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1544199de0 con 0x7f15441004b0 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1544199f50 con 0x7f15440ffb00 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f154a15e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419a110 con 0x7f1544105b10 2023-07-21T22:28:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47906/0 (socket says 172.21.15.93:47906) 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 -- 172.21.15.93:0/112314683 learned_addr learned my addr 172.21.15.93:0/112314683 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1543fff700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 0x7f15441997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f15437fe700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 0x7f1544195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 -- 172.21.15.93:0/112314683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 msgr2=0x7f15441997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 0x7f15441997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 -- 172.21.15.93:0/112314683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 msgr2=0x7f1544195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.393+0000 7f1542ffd700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 0x7f1544195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f1542ffd700 1 -- 172.21.15.93:0/112314683 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c009d00 con 0x7f15441004b0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f1542ffd700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f153400d950 tx=0x7f153400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f1540ff9700 1 -- 172.21.15.93:0/112314683 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400f890 con 0x7f15441004b0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f1540ff9700 1 -- 172.21.15.93:0/112314683 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1534010460 con 0x7f15441004b0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154419a3f0 con 0x7f15441004b0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f1540ff9700 1 -- 172.21.15.93:0/112314683 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400e610 con 0x7f15441004b0 2023-07-21T22:28:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.394+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15441a2d50 con 0x7f15441004b0 2023-07-21T22:28:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.397+0000 7f1540ff9700 1 -- 172.21.15.93:0/112314683 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f15340108d0 con 0x7f15441004b0 2023-07-21T22:28:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.397+0000 7f1540ff9700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 0x7f152c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.397+0000 7f1540ff9700 1 -- 172.21.15.93:0/112314683 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f15340ccb30 con 0x7f15441004b0 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.397+0000 7f15437fe700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 0x7f152c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15437fe700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 0x7f152c0a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1538005950 tx=0x7f153800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 Processor -- start 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 -- start start 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e7158bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e715864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7148bc50 con 0x557e7158c210 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e714de9e0 con 0x557e7158c660 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15562e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e714d7e70 con 0x557e713e26b0 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47914/0 (socket says 172.21.15.93:47914) 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f1542ffd700 1 -- 172.21.15.93:0/1652097354 learned_addr learned my addr 172.21.15.93:0/1652097354 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f15437fe700 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e7158bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.398+0000 7f1543fff700 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e715864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 -- 172.21.15.93:0/1652097354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 msgr2=0x557e7158bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e7158bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 -- 172.21.15.93:0/1652097354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 msgr2=0x557e715864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e715864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 -- 172.21.15.93:0/1652097354 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e71305d80 con 0x557e7158c210 2023-07-21T22:28:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1542ffd700 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f15340960d0 tx=0x7f1534099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab89eee8ab22c5da server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1529ffb700 1 -- 172.21.15.93:0/1652097354 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340962e0 con 0x557e7158c210 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1529ffb700 1 -- 172.21.15.93:0/1652097354 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1534096440 con 0x557e7158c210 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f1529ffb700 1 -- 172.21.15.93:0/1652097354 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340a1b90 con 0x557e7158c210 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f15562e3b80 1 -- 172.21.15.93:0/1652097354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 msgr2=0x557e71585f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.399+0000 7f15562e3b80 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f15340960d0 tx=0x7f1534099bd0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- 172.21.15.93:0/1652097354 shutdown_connections 2023-07-21T22:28:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e715864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e7158bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- 172.21.15.93:0/1652097354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- 172.21.15.93:0/1652097354 >> 172.21.15.93:0/1652097354 conn(0x557e7157bae0 msgr2=0x557e715810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- 172.21.15.93:0/1652097354 shutdown_connections 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- 172.21.15.93:0/1652097354 wait complete. 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- start start 2023-07-21T22:28:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 Processor -- start 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 0x557e71595b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 0x557e7158f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71477a60 con 0x557e713e26b0 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71477760 con 0x557e7158c210 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.400+0000 7f15562e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7135b8c0 con 0x557e7158c660 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15562e3b80 10 client.0 _ll_register_callbacks cb 0x557e71475700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47918/0 (socket says 172.21.15.93:47918) 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 -- 172.21.15.93:0/4103151155 learned_addr learned my addr 172.21.15.93:0/4103151155 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1542ffd700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 0x557e71595b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 -- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 msgr2=0x557e7158f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1543fff700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 0x557e7158f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 0x557e7158f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 -- 172.21.15.93:0/4103151155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 msgr2=0x557e71595b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 0x557e71595b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 -- 172.21.15.93:0/4103151155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15441a2d50 con 0x557e713e26b0 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1542ffd700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 0x557e71595b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1543fff700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 0x557e7158f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15437fe700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f15380114e0 tx=0x7f15380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803fdf9341e12896 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1522ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538004a30 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1538017070 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e71431490 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1538004b90 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15562e3b80 10 client.17073 fetch_fsmap learned FSMap version 57 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.401+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x557e712eec00 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e71431490 con 0x557e713e26b0 2023-07-21T22:28:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 57) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f153801f830 con 0x557e713e26b0 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 10 client.17073 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 20 client.17073 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e7159f900 0x557e715a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 4 client.17073 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e714d0e50 con 0x557e7159f900 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f1542ffd700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e7159f900 0x557e715a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f1522ffd700 10 client.17073 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.402+0000 7f1542ffd700 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e7159f900 0x557e715a1d10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511f868014f1bacf server_cookie=7cb3c0340752b83 in_seq=0 out_seq=0 2023-07-21T22:28:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f1522ffd700 1 -- 172.21.15.93:0/4103151155 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e714d0e50 con 0x557e7159f900 2023-07-21T22:28:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f1522ffd700 10 client.17073 handle_command_reply: tid=0 2023-07-21T22:28:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 client.17073 shutdown 2023-07-21T22:28:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 msgr2=0x557e715955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f15380114e0 tx=0x7f15380047c0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 shutdown_connections 2023-07-21T22:28:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e7159f900 0x557e715a1d10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c660 0x557e7158f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f1522ffd700 0 client.17073 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e713e26b0 0x557e715955f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 --2- 172.21.15.93:0/4103151155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c210 0x557e71595b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 >> 172.21.15.93:0/4103151155 conn(0x557e7157bae0 msgr2=0x557e715803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 shutdown_connections 2023-07-21T22:28:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.403+0000 7f15562e3b80 1 -- 172.21.15.93:0/4103151155 wait complete. 2023-07-21T22:28:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.404+0000 7f15562e3b80 20 client.17073 trim_cache size 0 max 16384 2023-07-21T22:28:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 Processor -- start 2023-07-21T22:28:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 -- start start 2023-07-21T22:28:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 0x557e7158b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71586a00 con 0x557e7158c660 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7158a510 con 0x557e713e26b0 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15562e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7158b8a0 con 0x557e7158c210 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47930/0 (socket says 172.21.15.93:47930) 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 -- 172.21.15.93:0/380188748 learned_addr learned my addr 172.21.15.93:0/380188748 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1542ffd700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15437fe700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 0x557e7158b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 -- 172.21.15.93:0/380188748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 msgr2=0x557e71585f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 -- 172.21.15.93:0/380188748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 msgr2=0x557e7158b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 0x557e7158b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1543fff700 1 -- 172.21.15.93:0/380188748 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7132c1f0 con 0x557e7158c660 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f1542ffd700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.418+0000 7f15437fe700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 0x557e7158b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f1543fff700 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f153c00b0a0 tx=0x7f153c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf32ea1452aed2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f152a7fc700 1 -- 172.21.15.93:0/380188748 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00e040 con 0x557e7158c660 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f152a7fc700 1 -- 172.21.15.93:0/380188748 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f153c0093d0 con 0x557e7158c660 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f152a7fc700 1 -- 172.21.15.93:0/380188748 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c025700 con 0x557e7158c660 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 -- 172.21.15.93:0/380188748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 msgr2=0x557e71586480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f153c00b0a0 tx=0x7f153c00ceb0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 -- 172.21.15.93:0/380188748 shutdown_connections 2023-07-21T22:28:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c660 0x557e71586480 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e713e26b0 0x557e7158b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 --2- 172.21.15.93:0/380188748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e7158c210 0x557e71585f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.419+0000 7f15562e3b80 1 -- 172.21.15.93:0/380188748 >> 172.21.15.93:0/380188748 conn(0x557e7157bae0 msgr2=0x557e7158f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- 172.21.15.93:0/380188748 shutdown_connections 2023-07-21T22:28:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- 172.21.15.93:0/380188748 wait complete. 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- start start 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 Processor -- start 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e71595810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e71594080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71594650 con 0x557e7158c210 2023-07-21T22:28:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71594790 con 0x557e7158c660 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e71594920 con 0x557e713e26b0 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f15562e3b80 10 client.0 _ll_register_callbacks cb 0x7f1538010c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f1542ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47934/0 (socket says 172.21.15.93:47934) 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.420+0000 7f1542ffd700 1 -- 172.21.15.93:0/576126522 learned_addr learned my addr 172.21.15.93:0/576126522 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1543fff700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e71594080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15437fe700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e71595810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 -- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 msgr2=0x557e71595810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e71595810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 -- 172.21.15.93:0/576126522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 msgr2=0x557e71594080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e71594080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 -- 172.21.15.93:0/576126522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154410d240 con 0x557e7158c210 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1542ffd700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f15340998d0 tx=0x7f15340a21a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fb071817351714 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340962e0 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f15340a1b90 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1534096440 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15340999a0 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 10 client.17079 fetch_fsmap learned FSMap version 57 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x557e71304250 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1534096440 con 0x557e7158c210 2023-07-21T22:28:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 57) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1534098b30 con 0x557e7158c210 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 20 client.17079 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e715f4940 0x557e715f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 4 client.17079 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.421+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e71492ff0 con 0x557e715f4940 2023-07-21T22:28:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.422+0000 7f15437fe700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e715f4940 0x557e715f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.422+0000 7f1522ffd700 10 client.17079 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.422+0000 7f15437fe700 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e715f4940 0x557e715f6d50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45db14c2ccf4d5f9 server_cookie=23fc550f0677eb8c in_seq=0 out_seq=0 2023-07-21T22:28:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f1522ffd700 1 -- 172.21.15.93:0/576126522 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e71492ff0 con 0x557e715f4940 2023-07-21T22:28:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f1522ffd700 10 client.17079 handle_command_reply: tid=0 2023-07-21T22:28:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 client.17079 shutdown 2023-07-21T22:28:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 msgr2=0x557e71593b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f15340998d0 tx=0x7f15340a21a0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 shutdown_connections 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557e715f4940 0x557e715f6d50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e7158c660 0x557e71594080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e713e26b0 0x557e71595810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 --2- 172.21.15.93:0/576126522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557e7158c210 0x557e71593b40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 >> 172.21.15.93:0/576126522 conn(0x557e7157bae0 msgr2=0x557e7158e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.475+0000 7f1522ffd700 0 client.17079 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.476+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 shutdown_connections 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.476+0000 7f15562e3b80 1 -- 172.21.15.93:0/576126522 wait complete. 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.476+0000 7f15562e3b80 20 client.17079 trim_cache size 0 max 16384 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:28:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:28:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:28:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 msgr2=0x7f152c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 0x7f152c0a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1538005950 tx=0x7f153800b410 comp rx=0 tx=0).stop 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 msgr2=0x7f1544196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f153400d950 tx=0x7f153400dc60 comp rx=0 tx=0).stop 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 shutdown_connections 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544105b10 0x7f15441997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15440ffb00 0x7f1544195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f152c0a12c0 0x7f152c0a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 --2- 172.21.15.93:0/112314683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15441004b0 0x7f1544196220 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.477+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 >> 172.21.15.93:0/112314683 conn(0x7f15440fb600 msgr2=0x7f15440fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.478+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 shutdown_connections 2023-07-21T22:28:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:07.478+0000 7f154a15e700 1 -- 172.21.15.93:0/112314683 wait complete. 2023-07-21T22:28:07.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:07 smithi194 ceph-mon[98288]: pgmap v369: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T22:28:07.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:07 smithi134 ceph-mon[97873]: pgmap v369: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T22:28:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:07 smithi093 ceph-mon[96579]: pgmap v369: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 319 op/s 2023-07-21T22:28:08.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 714363 6.52 MB/sec execute 525 sec latency 3422.391 ms 2023-07-21T22:28:09.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 714545 6.51 MB/sec execute 526 sec latency 4300.377 ms 2023-07-21T22:28:09.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:09 smithi194 ceph-mon[98288]: pgmap v370: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:09 smithi134 ceph-mon[97873]: pgmap v370: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:09 smithi093 ceph-mon[96579]: pgmap v370: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:10.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 715457 6.51 MB/sec execute 527 sec latency 349.804 ms 2023-07-21T22:28:11.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 715457 6.50 MB/sec execute 528 sec latency 1349.847 ms 2023-07-21T22:28:11.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:11 smithi194 ceph-mon[98288]: pgmap v371: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:11.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:11 smithi134 ceph-mon[97873]: pgmap v371: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:11 smithi093 ceph-mon[96579]: pgmap v371: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 247 op/s 2023-07-21T22:28:12.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 716374 6.49 MB/sec execute 529 sec latency 1853.897 ms 2023-07-21T22:28:13.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 717933 6.50 MB/sec execute 530 sec latency 12.865 ms 2023-07-21T22:28:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:13 smithi194 ceph-mon[98288]: pgmap v372: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:13 smithi194 ceph-mon[98288]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:28:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:13 smithi134 ceph-mon[97873]: pgmap v372: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:13 smithi134 ceph-mon[97873]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:28:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:13 smithi093 ceph-mon[96579]: pgmap v372: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:13 smithi093 ceph-mon[96579]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:28:14.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 719513 6.50 MB/sec execute 531 sec latency 157.457 ms 2023-07-21T22:28:15.344 INFO:tasks.workunit.client.0.smithi093.stdout: 1 721158 6.51 MB/sec execute 532 sec latency 43.147 ms 2023-07-21T22:28:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:15 smithi194 ceph-mon[98288]: pgmap v374: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-21T22:28:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:15 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:15 smithi134 ceph-mon[97873]: pgmap v374: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-21T22:28:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:15 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:15 smithi093 ceph-mon[96579]: pgmap v374: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-21T22:28:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:15 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:16.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 722415 6.51 MB/sec execute 533 sec latency 24.613 ms 2023-07-21T22:28:17.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 722548 6.50 MB/sec execute 534 sec latency 885.609 ms 2023-07-21T22:28:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:17 smithi194 ceph-mon[98288]: pgmap v375: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:17 smithi134 ceph-mon[97873]: pgmap v375: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:17 smithi093 ceph-mon[96579]: pgmap v375: 97 pgs: 97 active+clean; 324 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.3 MiB/s wr, 159 op/s 2023-07-21T22:28:18.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 723630 6.49 MB/sec execute 535 sec latency 1383.691 ms 2023-07-21T22:28:19.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 725065 6.50 MB/sec execute 536 sec latency 12.474 ms 2023-07-21T22:28:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:19 smithi194 ceph-mon[98288]: pgmap v376: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:19 smithi134 ceph-mon[97873]: pgmap v376: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:19 smithi093 ceph-mon[96579]: pgmap v376: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:20.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 726096 6.49 MB/sec execute 537 sec latency 315.659 ms 2023-07-21T22:28:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.586+0000 7fe281b39700 1 -- 172.21.15.93:0/1359605781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 msgr2=0x7fe27c109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.586+0000 7fe281b39700 1 --2- 172.21.15.93:0/1359605781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c109bb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 -- 172.21.15.93:0/1359605781 shutdown_connections 2023-07-21T22:28:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 --2- 172.21.15.93:0/1359605781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c10a180 0x7fe27c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 --2- 172.21.15.93:0/1359605781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c100db0 0x7fe27c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 --2- 172.21.15.93:0/1359605781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c109bb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 -- 172.21.15.93:0/1359605781 >> 172.21.15.93:0/1359605781 conn(0x7fe27c075bd0 msgr2=0x7fe27c075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 -- 172.21.15.93:0/1359605781 shutdown_connections 2023-07-21T22:28:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 -- 172.21.15.93:0/1359605781 wait complete. 2023-07-21T22:28:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 Processor -- start 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.587+0000 7fe281b39700 1 -- start start 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c100db0 0x7fe27c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c10a180 0x7fe27c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c196070 con 0x7fe27c1016c0 2023-07-21T22:28:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c1961e0 con 0x7fe27c10a180 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c1963a0 con 0x7fe27c100db0 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47948/0 (socket says 172.21.15.93:47948) 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 -- 172.21.15.93:0/3649700312 learned_addr learned my addr 172.21.15.93:0/3649700312 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c100db0 0x7fe27c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27bfff700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c10a180 0x7fe27c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 -- 172.21.15.93:0/3649700312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c100db0 msgr2=0x7fe27c19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c100db0 0x7fe27c19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 -- 172.21.15.93:0/3649700312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c10a180 msgr2=0x7fe27c195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c10a180 0x7fe27c195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 -- 172.21.15.93:0/3649700312 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0095f0 con 0x7fe27c1016c0 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe27affd700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0095c0 tx=0x7fe26c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe278ff9700 1 -- 172.21.15.93:0/3649700312 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c01d070 con 0x7fe27c1016c0 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe278ff9700 1 -- 172.21.15.93:0/3649700312 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c0054f0 con 0x7fe27c1016c0 2023-07-21T22:28:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c196620 con 0x7fe27c1016c0 2023-07-21T22:28:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe278ff9700 1 -- 172.21.15.93:0/3649700312 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c004a70 con 0x7fe27c1016c0 2023-07-21T22:28:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.588+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c1a2b90 con 0x7fe27c1016c0 2023-07-21T22:28:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.591+0000 7fe278ff9700 1 -- 172.21.15.93:0/3649700312 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe26c004bd0 con 0x7fe27c1016c0 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.591+0000 7fe278ff9700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 0x7fe2640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.591+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 0x7fe2640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.591+0000 7fe278ff9700 1 -- 172.21.15.93:0/3649700312 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe26c0cdaf0 con 0x7fe27c1016c0 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 0x7fe2640a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe270005950 tx=0x7fe27000b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 Processor -- start 2023-07-21T22:28:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 -- start start 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7ceaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1660 0x55fbb7ceb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7bf0c50 con 0x55fbb7b476b0 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.592+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7c439e0 con 0x55fbb7cf1660 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7c3ce70 con 0x55fbb7cf1210 2023-07-21T22:28:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47954/0 (socket says 172.21.15.93:47954) 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 -- 172.21.15.93:0/2586132494 learned_addr learned my addr 172.21.15.93:0/2586132494 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27bfff700 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1660 0x55fbb7ceb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 -- 172.21.15.93:0/2586132494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 msgr2=0x55fbb7ceaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27affd700 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7ceaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7ceaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 -- 172.21.15.93:0/2586132494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1660 msgr2=0x55fbb7ceb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1660 0x55fbb7ceb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 -- 172.21.15.93:0/2586132494 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb7a6ad80 con 0x55fbb7b476b0 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe270010fa0 tx=0x7fe270013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e76ecef69cae41 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe261ffb700 1 -- 172.21.15.93:0/2586132494 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27001e620 con 0x55fbb7b476b0 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe261ffb700 1 -- 172.21.15.93:0/2586132494 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe270015070 con 0x55fbb7b476b0 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.593+0000 7fe261ffb700 1 -- 172.21.15.93:0/2586132494 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27001ca20 con 0x55fbb7b476b0 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2586132494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 msgr2=0x55fbb7cf0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe270010fa0 tx=0x7fe270013000 comp rx=0 tx=0).stop 2023-07-21T22:28:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2586132494 shutdown_connections 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1660 0x55fbb7ceb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf0cd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2586132494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7ceaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2586132494 >> 172.21.15.93:0/2586132494 conn(0x55fbb7ce0ae0 msgr2=0x55fbb7ce60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2586132494 shutdown_connections 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2586132494 wait complete. 2023-07-21T22:28:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.594+0000 7fe28dcbeb80 1 -- start start 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 Processor -- start 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7bdca60 con 0x55fbb7cf1660 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7bdc760 con 0x55fbb7cf1210 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7ac08c0 con 0x55fbb7b476b0 2023-07-21T22:28:20.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe28dcbeb80 10 client.0 _ll_register_callbacks cb 0x55fbb7bda700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47964/0 (socket says 172.21.15.93:47964) 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 -- 172.21.15.93:0/4214490941 learned_addr learned my addr 172.21.15.93:0/4214490941 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27affd700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 -- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7b476b0 msgr2=0x55fbb7cf74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 -- 172.21.15.93:0/4214490941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 msgr2=0x55fbb7cf4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 -- 172.21.15.93:0/4214490941 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe270010c50 con 0x55fbb7cf1660 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27affd700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe27bfff700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe274009f30 tx=0x7fe274004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5718ba82d5569a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe25affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.595+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274007910 con 0x55fbb7cf1660 2023-07-21T22:28:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fbb7b96490 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe274007a70 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe274007c30 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 10 client.17091 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55fbb7a53c00 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fbb7b96490 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe274014040 con 0x55fbb7cf1660 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 10 client.17091 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 20 client.17091 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7d04b30 0x55fbb7d06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 4 client.17091 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fbb7c35e50 con 0x55fbb7d04b30 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.596+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7d04b30 0x55fbb7d06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.597+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7d04b30 0x55fbb7d06f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e3e0e991f7db9e server_cookie=f3abbdae51b8ec06 in_seq=0 out_seq=0 2023-07-21T22:28:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.597+0000 7fe25affd700 10 client.17091 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.597+0000 7fe25affd700 1 -- 172.21.15.93:0/4214490941 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fbb7c35e50 con 0x55fbb7d04b30 2023-07-21T22:28:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.597+0000 7fe25affd700 10 client.17091 handle_command_reply: tid=0 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.597+0000 7fe28dcbeb80 1 client.17091 shutdown 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 msgr2=0x55fbb7cf4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe274009f30 tx=0x7fe274004320 comp rx=0 tx=0).stop 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 shutdown_connections 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7d04b30 0x55fbb7d06f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7cf1660 0x55fbb7cf4cf0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7b476b0 0x55fbb7cf74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/4214490941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe25affd700 0 client.17091 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 >> 172.21.15.93:0/4214490941 conn(0x55fbb7ce0ae0 msgr2=0x55fbb7ce53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 shutdown_connections 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/4214490941 wait complete. 2023-07-21T22:28:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.598+0000 7fe28dcbeb80 20 client.17091 trim_cache size 0 max 16384 2023-07-21T22:28:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.612+0000 7fe28dcbeb80 1 Processor -- start 2023-07-21T22:28:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.612+0000 7fe28dcbeb80 1 -- start start 2023-07-21T22:28:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cf2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7ce60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cf0cd0 con 0x55fbb7b476b0 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cf0920 con 0x55fbb7cf1210 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cf2c90 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47970/0 (socket says 172.21.15.93:47970) 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 -- 172.21.15.93:0/3163341264 learned_addr learned my addr 172.21.15.93:0/3163341264 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27bfff700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7ce60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cf2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 -- 172.21.15.93:0/3163341264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 msgr2=0x55fbb7cf2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cf2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 -- 172.21.15.93:0/3163341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 msgr2=0x55fbb7ce60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7ce60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 -- 172.21.15.93:0/3163341264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbb7a911f0 con 0x55fbb7b476b0 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cf2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe27affd700 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe26c09ab80 tx=0x7fe26c09a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc0e0cf20cf02f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe2627fc700 1 -- 172.21.15.93:0/3163341264 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c09abf0 con 0x55fbb7b476b0 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe2627fc700 1 -- 172.21.15.93:0/3163341264 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe26c09ec20 con 0x55fbb7b476b0 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.613+0000 7fe2627fc700 1 -- 172.21.15.93:0/3163341264 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c09a0e0 con 0x55fbb7b476b0 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/3163341264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 msgr2=0x55fbb7ce5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe26c09ab80 tx=0x7fe26c09a780 comp rx=0 tx=0).stop 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/3163341264 shutdown_connections 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7cf1210 0x55fbb7ce60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cf2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/3163341264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7b476b0 0x55fbb7ce5b60 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/3163341264 >> 172.21.15.93:0/3163341264 conn(0x55fbb7d01070 msgr2=0x55fbb7cf1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/3163341264 shutdown_connections 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/3163341264 wait complete. 2023-07-21T22:28:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 -- start start 2023-07-21T22:28:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.614+0000 7fe28dcbeb80 1 Processor -- start 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 0x55fbb7cfb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cfa1f0 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cfa360 con 0x55fbb7b476b0 2023-07-21T22:28:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbb7cfa520 con 0x55fbb7cf1210 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe28dcbeb80 10 client.0 _ll_register_callbacks cb 0x7fe274003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47976/0 (socket says 172.21.15.93:47976) 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 -- 172.21.15.93:0/2094102318 learned_addr learned my addr 172.21.15.93:0/2094102318 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 0x55fbb7cfb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27bfff700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 -- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 msgr2=0x55fbb7cf9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 -- 172.21.15.93:0/2094102318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 msgr2=0x55fbb7cfb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 0x55fbb7cfb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 -- 172.21.15.93:0/2094102318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c1a2b90 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 0x55fbb7cfb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe27affd700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005d90 tx=0x7fe26c0cb180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722ebc10ebe4f5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe25affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.615+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0b7070 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe26c0ba040 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe270015070 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe26c0a2fb0 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 10 client.17097 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55fbb7a69250 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe270015070 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe26c0a2bb0 con 0x55fbb7ce0ae0 2023-07-21T22:28:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 10 client.17097 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 20 client.17097 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7cf1660 0x55fbb7d5b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 4 client.17097 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fbb7bf7ff0 con 0x55fbb7cf1660 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.616+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7cf1660 0x55fbb7d5b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.617+0000 7fe25affd700 10 client.17097 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.617+0000 7fe27b7fe700 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7cf1660 0x55fbb7d5b800 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293b8c8a64222d21 server_cookie=3d334712ee3413d in_seq=0 out_seq=0 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe25affd700 1 -- 172.21.15.93:0/2094102318 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fbb7bf7ff0 con 0x55fbb7cf1660 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe25affd700 10 client.17097 handle_command_reply: tid=0 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 client.17097 shutdown 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 msgr2=0x55fbb7cfbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005d90 tx=0x7fe26c0cb180 comp rx=0 tx=0).stop 2023-07-21T22:28:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 shutdown_connections 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55fbb7cf1660 0x55fbb7d5b800 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fbb7cf1210 0x55fbb7cf9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fbb7b476b0 0x55fbb7cfb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe25affd700 0 client.17097 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 --2- 172.21.15.93:0/2094102318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fbb7ce0ae0 0x55fbb7cfbb80 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 >> 172.21.15.93:0/2094102318 conn(0x55fbb7d01070 msgr2=0x55fbb7cf6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 shutdown_connections 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.656+0000 7fe28dcbeb80 1 -- 172.21.15.93:0/2094102318 wait complete. 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe28dcbeb80 20 client.17097 trim_cache size 0 max 16384 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:28:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:28:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 msgr2=0x7fe2640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 0x7fe2640a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe270005950 tx=0x7fe27000b410 comp rx=0 tx=0).stop 2023-07-21T22:28:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 msgr2=0x7fe27c19c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0095c0 tx=0x7fe26c00f710 comp rx=0 tx=0).stop 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 shutdown_connections 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe27c10a180 0x7fe27c195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe27c100db0 0x7fe27c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe2640a12c0 0x7fe2640a3770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 --2- 172.21.15.93:0/3649700312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe27c1016c0 0x7fe27c19c0e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 >> 172.21.15.93:0/3649700312 conn(0x7fe27c075bd0 msgr2=0x7fe27c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 shutdown_connections 2023-07-21T22:28:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:20.657+0000 7fe281b39700 1 -- 172.21.15.93:0/3649700312 wait complete. 2023-07-21T22:28:21.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 726096 6.48 MB/sec execute 538 sec latency 1315.744 ms 2023-07-21T22:28:21.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:21 smithi194 ceph-mon[98288]: pgmap v377: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:21.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:21 smithi134 ceph-mon[97873]: pgmap v377: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:21 smithi093 ceph-mon[96579]: pgmap v377: 97 pgs: 97 active+clean; 326 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.6 MiB/s wr, 197 op/s 2023-07-21T22:28:22.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 727930 6.49 MB/sec execute 539 sec latency 1483.459 ms 2023-07-21T22:28:23.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 729054 6.49 MB/sec execute 540 sec latency 13.701 ms 2023-07-21T22:28:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:23 smithi194 ceph-mon[98288]: pgmap v378: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2023-07-21T22:28:23.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:23 smithi134 ceph-mon[97873]: pgmap v378: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2023-07-21T22:28:23.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:23 smithi093 ceph-mon[96579]: pgmap v378: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2023-07-21T22:28:24.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 730087 6.48 MB/sec execute 541 sec latency 467.418 ms 2023-07-21T22:28:25.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 731763 6.49 MB/sec execute 542 sec latency 37.601 ms 2023-07-21T22:28:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:25 smithi194 ceph-mon[98288]: pgmap v379: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:25 smithi134 ceph-mon[97873]: pgmap v379: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:25 smithi093 ceph-mon[96579]: pgmap v379: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:26.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 732948 6.49 MB/sec execute 543 sec latency 21.094 ms 2023-07-21T22:28:27.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 733187 6.48 MB/sec execute 544 sec latency 850.828 ms 2023-07-21T22:28:27.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:27 smithi194 ceph-mon[98288]: pgmap v380: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:27.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:27 smithi134 ceph-mon[97873]: pgmap v380: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:27 smithi093 ceph-mon[96579]: pgmap v380: 97 pgs: 97 active+clean; 325 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.3 MiB/s wr, 185 op/s 2023-07-21T22:28:28.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 733187 6.47 MB/sec execute 545 sec latency 1850.920 ms 2023-07-21T22:28:29.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 733587 6.46 MB/sec execute 546 sec latency 2688.854 ms 2023-07-21T22:28:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:29 smithi194 ceph-mon[98288]: pgmap v381: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2023-07-21T22:28:29.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:29 smithi134 ceph-mon[97873]: pgmap v381: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2023-07-21T22:28:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:29 smithi093 ceph-mon[96579]: pgmap v381: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2023-07-21T22:28:30.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 735317 6.47 MB/sec execute 547 sec latency 35.386 ms 2023-07-21T22:28:31.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 736556 6.47 MB/sec execute 548 sec latency 17.092 ms 2023-07-21T22:28:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:31 smithi194 ceph-mon[98288]: pgmap v382: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.3 MiB/s wr, 180 op/s 2023-07-21T22:28:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:31 smithi134 ceph-mon[97873]: pgmap v382: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.3 MiB/s wr, 180 op/s 2023-07-21T22:28:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:31 smithi093 ceph-mon[96579]: pgmap v382: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.3 MiB/s wr, 180 op/s 2023-07-21T22:28:32.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 736729 6.46 MB/sec execute 549 sec latency 913.035 ms 2023-07-21T22:28:33.345 INFO:tasks.workunit.client.0.smithi093.stdout: 1 736729 6.45 MB/sec execute 550 sec latency 1913.154 ms 2023-07-21T22:28:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:33 smithi194 ceph-mon[98288]: pgmap v383: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-21T22:28:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:33 smithi134 ceph-mon[97873]: pgmap v383: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-21T22:28:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:33 smithi093 ceph-mon[96579]: pgmap v383: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-21T22:28:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- 172.21.15.93:0/1958723484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 msgr2=0x7fa45c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 --2- 172.21.15.93:0/1958723484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c10b900 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b0a0 tx=0x7fa45400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- 172.21.15.93:0/1958723484 shutdown_connections 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 --2- 172.21.15.93:0/1958723484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c10b900 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 --2- 172.21.15.93:0/1958723484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa45c104ad0 0x7fa45c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 --2- 172.21.15.93:0/1958723484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- 172.21.15.93:0/1958723484 >> 172.21.15.93:0/1958723484 conn(0x7fa45c075bd0 msgr2=0x7fa45c075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- 172.21.15.93:0/1958723484 shutdown_connections 2023-07-21T22:28:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- 172.21.15.93:0/1958723484 wait complete. 2023-07-21T22:28:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 Processor -- start 2023-07-21T22:28:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.774+0000 7fa463621700 1 -- start start 2023-07-21T22:28:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa45c104ad0 0x7fa45c1949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c195020 con 0x7fa45c0ff470 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c195190 con 0x7fa45c0feac0 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c195350 con 0x7fa45c104ad0 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47990/0 (socket says 172.21.15.93:47990) 2023-07-21T22:28:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 -- 172.21.15.93:0/4047401477 learned_addr learned my addr 172.21.15.93:0/4047401477 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 -- 172.21.15.93:0/4047401477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa45c104ad0 msgr2=0x7fa45c1949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa45c104ad0 0x7fa45c1949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 -- 172.21.15.93:0/4047401477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 msgr2=0x7fa45c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 -- 172.21.15.93:0/4047401477 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454009d00 con 0x7fa45c0ff470 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d950 tx=0x7fa44c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa4527fc700 1 -- 172.21.15.93:0/4047401477 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c009b40 con 0x7fa45c0ff470 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa4527fc700 1 -- 172.21.15.93:0/4047401477 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c009ca0 con 0x7fa45c0ff470 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c195630 con 0x7fa45c0ff470 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa4527fc700 1 -- 172.21.15.93:0/4047401477 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00f620 con 0x7fa45c0ff470 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.775+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c1a1b40 con 0x7fa45c0ff470 2023-07-21T22:28:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.777+0000 7fa4527fc700 1 -- 172.21.15.93:0/4047401477 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa44c010460 con 0x7fa45c0ff470 2023-07-21T22:28:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.778+0000 7fa4527fc700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 0x7fa4480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.778+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 0x7fa4480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.779+0000 7fa4527fc700 1 -- 172.21.15.93:0/4047401477 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa44c0cd020 con 0x7fa45c0ff470 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.779+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 0x7fa4480a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa458005fd0 tx=0x7fa458005dc0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 Processor -- start 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 -- start start 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x556592497cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498210 0x556592491f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556592397c50 con 0x556592498660 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565923ea9e0 con 0x5565922ee6b0 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565923e3e70 con 0x556592498210 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48000/0 (socket says 172.21.15.93:48000) 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 -- 172.21.15.93:0/2410323811 learned_addr learned my addr 172.21.15.93:0/2410323811 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa4613bd700 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x556592497cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa460bbc700 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498210 0x556592491f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 -- 172.21.15.93:0/2410323811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498210 msgr2=0x556592491f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498210 0x556592491f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 -- 172.21.15.93:0/2410323811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 msgr2=0x556592497cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x556592497cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 -- 172.21.15.93:0/2410323811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556592211d80 con 0x556592498660 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.780+0000 7fa461bbe700 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b070 tx=0x7fa454009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baa9043e7223657 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa4477fe700 1 -- 172.21.15.93:0/2410323811 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454021070 con 0x556592498660 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa4477fe700 1 -- 172.21.15.93:0/2410323811 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45400cec0 con 0x556592498660 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa4477fe700 1 -- 172.21.15.93:0/2410323811 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400e040 con 0x556592498660 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/2410323811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 msgr2=0x5565924924a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b070 tx=0x7fa454009c70 comp rx=0 tx=0).stop 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/2410323811 shutdown_connections 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498660 0x5565924924a0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x556592497cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/2410323811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498210 0x556592491f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/2410323811 >> 172.21.15.93:0/2410323811 conn(0x556592487ae0 msgr2=0x55659248d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/2410323811 shutdown_connections 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.781+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/2410323811 wait complete. 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 -- start start 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 Processor -- start 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x5565924a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556592383a60 con 0x556592498210 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556592383760 con 0x556592498660 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565922678c0 con 0x5565922ee6b0 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa46f7a6b80 10 client.0 _ll_register_callbacks cb 0x556592381700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48004/0 (socket says 172.21.15.93:48004) 2023-07-21T22:28:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa460bbc700 1 -- 172.21.15.93:0/1107625084 learned_addr learned my addr 172.21.15.93:0/1107625084 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa461bbe700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.782+0000 7fa4613bd700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x5565924a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 -- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 msgr2=0x5565924a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x5565924a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 -- 172.21.15.93:0/1107625084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 msgr2=0x55659249b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 -- 172.21.15.93:0/1107625084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c10c200 con 0x556592498210 2023-07-21T22:28:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa460bbc700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa44c09a080 tx=0x7fa44c09a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f611e2de485fdca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c010910 con 0x556592498210 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa44c09a350 con 0x556592498210 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565923dce50 con 0x556592498210 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c09a510 con 0x556592498210 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa46f7a6b80 10 client.17109 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x5565921fac00 con 0x556592498210 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa44c0c8080 con 0x556592498210 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa44c0a2140 con 0x556592498210 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa46f7a6b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.783+0000 7fa46f7a6b80 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa46f7a6b80 20 client.17109 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565924ab8b0 0x5565924adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa46f7a6b80 4 client.17109 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55659239eff0 con 0x5565924ab8b0 2023-07-21T22:28:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa4613bd700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565924ab8b0 0x5565924adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa444ff9700 10 client.17109 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.784+0000 7fa4613bd700 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565924ab8b0 0x5565924adcc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0fcb3665916795 server_cookie=b54a1aa45310a833 in_seq=0 out_seq=0 2023-07-21T22:28:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa444ff9700 1 -- 172.21.15.93:0/1107625084 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55659239eff0 con 0x5565924ab8b0 2023-07-21T22:28:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa444ff9700 10 client.17109 handle_command_reply: tid=0 2023-07-21T22:28:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 client.17109 shutdown 2023-07-21T22:28:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 msgr2=0x5565924a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa44c09a080 tx=0x7fa44c09a230 comp rx=0 tx=0).stop 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 shutdown_connections 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565924ab8b0 0x5565924adcc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x5565924a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa444ff9700 0 client.17109 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/1107625084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924a1aa0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 >> 172.21.15.93:0/1107625084 conn(0x556592487ae0 msgr2=0x55659248c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 shutdown_connections 2023-07-21T22:28:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.785+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/1107625084 wait complete. 2023-07-21T22:28:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.786+0000 7fa46f7a6b80 20 client.17109 trim_cache size 0 max 16384 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 Processor -- start 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 -- start start 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x556592499330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659249bc50 con 0x556592498210 2023-07-21T22:28:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556592492cd0 con 0x556592498660 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556592492870 con 0x5565922ee6b0 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48012/0 (socket says 172.21.15.93:48012) 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa460bbc700 1 -- 172.21.15.93:0/4103986260 learned_addr learned my addr 172.21.15.93:0/4103986260 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.800+0000 7fa461bbe700 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 -- 172.21.15.93:0/4103986260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 msgr2=0x556592499330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x556592499330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 -- 172.21.15.93:0/4103986260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 msgr2=0x55659249a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 -- 172.21.15.93:0/4103986260 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565922381f0 con 0x556592498210 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa461bbe700 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0cad70 tx=0x7fa44c099ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931b1866e3542536 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa447fff700 1 -- 172.21.15.93:0/4103986260 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c014040 con 0x556592498210 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa447fff700 1 -- 172.21.15.93:0/4103986260 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565923dce50 con 0x556592498210 2023-07-21T22:28:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa447fff700 1 -- 172.21.15.93:0/4103986260 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c09a350 con 0x556592498210 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4103986260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 msgr2=0x55659249b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0cad70 tx=0x7fa44c099ff0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4103986260 shutdown_connections 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556592498660 0x55659249a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5565922ee6b0 0x556592499330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4103986260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x55659249b710 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.801+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4103986260 >> 172.21.15.93:0/4103986260 conn(0x556592487ae0 msgr2=0x556592499fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4103986260 shutdown_connections 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4103986260 wait complete. 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- start start 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 Processor -- start 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x5565924f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498660 0x5565924f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565924fa070 con 0x556592498210 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565924fa1e0 con 0x5565922ee6b0 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565924fa3a0 con 0x556592498660 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa46f7a6b80 10 client.0 _ll_register_callbacks cb 0x7fa44c099d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa460bbc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48018/0 (socket says 172.21.15.93:48018) 2023-07-21T22:28:33.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.802+0000 7fa460bbc700 1 -- 172.21.15.93:0/4208275234 learned_addr learned my addr 172.21.15.93:0/4208275234 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x5565924f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa461bbe700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498660 0x5565924f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 -- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498660 msgr2=0x5565924f9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498660 0x5565924f9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 -- 172.21.15.93:0/4208275234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 msgr2=0x5565924f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x5565924f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 -- 172.21.15.93:0/4208275234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565921fac00 con 0x556592498210 2023-07-21T22:28:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x5565924f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa460bbc700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005950 tx=0x7fa44c0a26c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aa923dea256178 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0997e0 con 0x556592498210 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa44c09e120 con 0x556592498210 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55659233d490 con 0x556592498210 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa44c09a350 con 0x556592498210 2023-07-21T22:28:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa46f7a6b80 10 client.17115 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x556592210250 con 0x556592498210 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55659233d490 con 0x556592498210 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa44c0833d0 con 0x556592498210 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.803+0000 7fa46f7a6b80 10 client.17115 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa46f7a6b80 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa46f7a6b80 20 client.17115 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565925027d0 0x556592504bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa46f7a6b80 4 client.17115 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:28:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556592232f10 con 0x5565925027d0 2023-07-21T22:28:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565925027d0 0x556592504bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa4613bd700 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565925027d0 0x556592504bb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39df3c8df6856d1 server_cookie=bd29078f2cbd6d34 in_seq=0 out_seq=0 2023-07-21T22:28:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.804+0000 7fa444ff9700 10 client.17115 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.845+0000 7fa444ff9700 1 -- 172.21.15.93:0/4208275234 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556592232f10 con 0x5565925027d0 2023-07-21T22:28:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.845+0000 7fa444ff9700 10 client.17115 handle_command_reply: tid=0 2023-07-21T22:28:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.845+0000 7fa46f7a6b80 1 client.17115 shutdown 2023-07-21T22:28:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 msgr2=0x5565924f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005950 tx=0x7fa44c0a26c0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:28:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:28:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:28:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:28:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 shutdown_connections 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565925027d0 0x556592504bb0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556592498660 0x5565924f9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565922ee6b0 0x5565924f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 --2- 172.21.15.93:0/4208275234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556592498210 0x5565924f5e40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 >> 172.21.15.93:0/4208275234 conn(0x556592487ae0 msgr2=0x556592498aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa444ff9700 0 client.17115 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 shutdown_connections 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 1 -- 172.21.15.93:0/4208275234 wait complete. 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.846+0000 7fa46f7a6b80 20 client.17115 trim_cache size 0 max 16384 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 msgr2=0x7fa4480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 0x7fa4480a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa458005fd0 tx=0x7fa458005dc0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 msgr2=0x7fa45c19b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d950 tx=0x7fa44c00dd10 comp rx=0 tx=0).stop 2023-07-21T22:28:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 shutdown_connections 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa45c104ad0 0x7fa45c1949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa45c0feac0 0x7fa45c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa4480a1340 0x7fa4480a37f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 --2- 172.21.15.93:0/4047401477 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa45c0ff470 0x7fa45c19b090 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 >> 172.21.15.93:0/4047401477 conn(0x7fa45c075bd0 msgr2=0x7fa45c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 shutdown_connections 2023-07-21T22:28:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:33.847+0000 7fa463621700 1 -- 172.21.15.93:0/4047401477 wait complete. 2023-07-21T22:28:34.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 737178 6.43 MB/sec execute 551 sec latency 2757.437 ms 2023-07-21T22:28:35.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 738958 6.45 MB/sec execute 552 sec latency 35.919 ms 2023-07-21T22:28:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:35 smithi194 ceph-mon[98288]: pgmap v384: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:35 smithi134 ceph-mon[97873]: pgmap v384: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:35 smithi093 ceph-mon[96579]: pgmap v384: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:36.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 740278 6.44 MB/sec execute 553 sec latency 79.740 ms 2023-07-21T22:28:37.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 741479 6.45 MB/sec execute 554 sec latency 577.534 ms 2023-07-21T22:28:37.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:37 smithi194 ceph-mon[98288]: pgmap v385: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:37 smithi134 ceph-mon[97873]: pgmap v385: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:37 smithi093 ceph-mon[96579]: pgmap v385: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.2 MiB/s wr, 157 op/s 2023-07-21T22:28:38.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 742918 6.45 MB/sec execute 555 sec latency 13.749 ms 2023-07-21T22:28:39.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 744271 6.45 MB/sec execute 556 sec latency 287.661 ms 2023-07-21T22:28:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:39 smithi194 ceph-mon[98288]: pgmap v386: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2023-07-21T22:28:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:39 smithi134 ceph-mon[97873]: pgmap v386: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2023-07-21T22:28:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:39 smithi093 ceph-mon[96579]: pgmap v386: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 233 op/s 2023-07-21T22:28:40.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 746065 6.46 MB/sec execute 557 sec latency 12.764 ms 2023-07-21T22:28:41.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 747368 6.46 MB/sec execute 558 sec latency 112.178 ms 2023-07-21T22:28:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:41 smithi194 ceph-mon[98288]: pgmap v387: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2023-07-21T22:28:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:41 smithi134 ceph-mon[97873]: pgmap v387: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2023-07-21T22:28:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:41 smithi093 ceph-mon[96579]: pgmap v387: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2023-07-21T22:28:42.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 747368 6.44 MB/sec execute 559 sec latency 1112.262 ms 2023-07-21T22:28:43.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 748443 6.44 MB/sec execute 560 sec latency 1610.186 ms 2023-07-21T22:28:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:43 smithi194 ceph-mon[98288]: pgmap v388: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 219 op/s 2023-07-21T22:28:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:43 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:43 smithi134 ceph-mon[97873]: pgmap v388: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 219 op/s 2023-07-21T22:28:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:43 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:43 smithi093 ceph-mon[96579]: pgmap v388: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 219 op/s 2023-07-21T22:28:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:43 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:44.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 749772 6.45 MB/sec execute 561 sec latency 18.335 ms 2023-07-21T22:28:44.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:44 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:44.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:44 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:44.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:44 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:28:44.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:44 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:44.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:44 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:44.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:44 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:28:44.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:44 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:44.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:44 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:44.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:44 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:28:45.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 750911 6.44 MB/sec execute 562 sec latency 311.405 ms 2023-07-21T22:28:45.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:45 smithi194 ceph-mon[98288]: pgmap v389: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:45.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:45 smithi134 ceph-mon[97873]: pgmap v389: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:45 smithi093 ceph-mon[96579]: pgmap v389: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:46.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 750911 6.43 MB/sec execute 563 sec latency 1311.479 ms 2023-07-21T22:28:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 -- 172.21.15.93:0/1427319260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c0ff3e0 msgr2=0x7f529c10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 --2- 172.21.15.93:0/1427319260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c0ff3e0 0x7f529c10b860 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f529400b0a0 tx=0x7f529400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 -- 172.21.15.93:0/1427319260 shutdown_connections 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 --2- 172.21.15.93:0/1427319260 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c0ff3e0 0x7f529c10b860 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 --2- 172.21.15.93:0/1427319260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c104a30 0x7f529c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 --2- 172.21.15.93:0/1427319260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0fea30 0x7f529c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 -- 172.21.15.93:0/1427319260 >> 172.21.15.93:0/1427319260 conn(0x7f529c074760 msgr2=0x7f529c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 -- 172.21.15.93:0/1427319260 shutdown_connections 2023-07-21T22:28:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 -- 172.21.15.93:0/1427319260 wait complete. 2023-07-21T22:28:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.962+0000 7f52a3a14700 1 Processor -- start 2023-07-21T22:28:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 -- start start 2023-07-21T22:28:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c0fea30 0x7f529c100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0ff3e0 0x7f529c100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a4380 con 0x7f529c104a30 2023-07-21T22:28:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a44c0 con 0x7f529c0ff3e0 2023-07-21T22:28:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a3a14700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a4630 con 0x7f529c0fea30 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48034/0 (socket says 172.21.15.93:48034) 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 -- 172.21.15.93:0/981571774 learned_addr learned my addr 172.21.15.93:0/981571774 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a0faf700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0ff3e0 0x7f529c100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a17b0700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c0fea30 0x7f529c100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 -- 172.21.15.93:0/981571774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c0fea30 msgr2=0x7f529c100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c0fea30 0x7f529c100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 -- 172.21.15.93:0/981571774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0ff3e0 msgr2=0x7f529c100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0ff3e0 0x7f529c100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 -- 172.21.15.93:0/981571774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5294009d00 con 0x7f529c104a30 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.963+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f529400b830 tx=0x7f52940079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.964+0000 7f52927fc700 1 -- 172.21.15.93:0/981571774 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529400e040 con 0x7f529c104a30 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.964+0000 7f52927fc700 1 -- 172.21.15.93:0/981571774 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294004ce0 con 0x7f529c104a30 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.964+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c1a48b0 con 0x7f529c104a30 2023-07-21T22:28:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.964+0000 7f52927fc700 1 -- 172.21.15.93:0/981571774 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529401d480 con 0x7f529c104a30 2023-07-21T22:28:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.964+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a4d30 con 0x7f529c104a30 2023-07-21T22:28:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.966+0000 7f52927fc700 1 -- 172.21.15.93:0/981571774 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5294003a00 con 0x7f529c104a30 2023-07-21T22:28:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.966+0000 7f52927fc700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 0x7f52880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.966+0000 7f52a17b0700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 0x7f52880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.966+0000 7f52927fc700 1 -- 172.21.15.93:0/981571774 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f52940ce3e0 con 0x7f529c104a30 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.967+0000 7f52a17b0700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 0x7f52880a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ffba0 tx=0x7f529800b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.967+0000 7f52afb99b80 1 Processor -- start 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.967+0000 7f52afb99b80 1 -- start start 2023-07-21T22:28:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631067b7c50 con 0x5631068b8660 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56310680a9e0 con 0x56310670e6b0 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52afb99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563106803e70 con 0x5631068b8210 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48042/0 (socket says 172.21.15.93:48042) 2023-07-21T22:28:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2205784993 learned_addr learned my addr 172.21.15.93:0/2205784993 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a17b0700 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2205784993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 msgr2=0x5631068b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2205784993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 msgr2=0x5631068b7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2205784993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563106631d80 con 0x5631068b8660 2023-07-21T22:28:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a17b0700 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f52940cc130 tx=0x7f5294097280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14c5e33344a0662 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52877fe700 1 -- 172.21.15.93:0/2205784993 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294004e50 con 0x5631068b8660 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52877fe700 1 -- 172.21.15.93:0/2205784993 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294024360 con 0x5631068b8660 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.968+0000 7f52877fe700 1 -- 172.21.15.93:0/2205784993 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940861f0 con 0x5631068b8660 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- 172.21.15.93:0/2205784993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 msgr2=0x5631068b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f52940cc130 tx=0x7f5294097280 comp rx=0 tx=0).stop 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- 172.21.15.93:0/2205784993 shutdown_connections 2023-07-21T22:28:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068b24a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2205784993 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- 172.21.15.93:0/2205784993 >> 172.21.15.93:0/2205784993 conn(0x5631068a7ae0 msgr2=0x5631068ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- 172.21.15.93:0/2205784993 shutdown_connections 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- 172.21.15.93:0/2205784993 wait complete. 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 -- start start 2023-07-21T22:28:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.969+0000 7f52afb99b80 1 Processor -- start 2023-07-21T22:28:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631067a3a60 con 0x5631068b8660 2023-07-21T22:28:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631067a3760 con 0x56310670e6b0 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631066878c0 con 0x5631068b8210 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52afb99b80 10 client.0 _ll_register_callbacks cb 0x5631067a1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48048/0 (socket says 172.21.15.93:48048) 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 -- 172.21.15.93:0/3758653886 learned_addr learned my addr 172.21.15.93:0/3758653886 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a0faf700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a17b0700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 -- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 msgr2=0x5631068c18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068c18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 -- 172.21.15.93:0/3758653886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 msgr2=0x5631068c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a1fb1700 1 -- 172.21.15.93:0/3758653886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a4d30 con 0x5631068b8660 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.970+0000 7f52a17b0700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f52940062a0 tx=0x7f5294097ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c8581284fda624 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529409f570 con 0x5631068b8660 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f529409f330 con 0x5631068b8660 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56310675d490 con 0x5631068b8660 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52940a3900 con 0x5631068b8660 2023-07-21T22:28:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 10 client.17127 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x56310661ac00 con 0x5631068b8660 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56310675d490 con 0x5631068b8660 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f52940b9020 con 0x5631068b8660 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 20 client.17127 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5631068cb310 0x5631068cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 4 client.17127 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631067fce50 con 0x5631068cb310 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.971+0000 7f52a17b0700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5631068cb310 0x5631068cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.972+0000 7f5284ff9700 10 client.17127 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.972+0000 7f52a17b0700 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5631068cb310 0x5631068cd6f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54ad64c734e82e3 server_cookie=323d3bce5b4eac06 in_seq=0 out_seq=0 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.972+0000 7f5284ff9700 1 -- 172.21.15.93:0/3758653886 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5631067fce50 con 0x5631068cb310 2023-07-21T22:28:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f5284ff9700 10 client.17127 handle_command_reply: tid=0 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 client.17127 shutdown 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 msgr2=0x5631068bb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f52940062a0 tx=0x7f5294097ee0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 shutdown_connections 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5631068cb310 0x5631068cd6f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bb690 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f5284ff9700 0 client.17127 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 --2- 172.21.15.93:0/3758653886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068c18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 >> 172.21.15.93:0/3758653886 conn(0x5631068a7ae0 msgr2=0x5631068ac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 shutdown_connections 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 1 -- 172.21.15.93:0/3758653886 wait complete. 2023-07-21T22:28:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.973+0000 7f52afb99b80 20 client.17127 trim_cache size 0 max 16384 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 Processor -- start 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 -- start start 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8660 0x5631068ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631068bc800 con 0x5631068b8210 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631068b6180 con 0x56310670e6b0 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52afb99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631067fce50 con 0x5631068b8660 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52a0faf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52a0faf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48054/0 (socket says 172.21.15.93:48054) 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52a0faf700 1 -- 172.21.15.93:0/465067350 learned_addr learned my addr 172.21.15.93:0/465067350 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.988+0000 7f52a17b0700 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8660 0x5631068ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 -- 172.21.15.93:0/465067350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8660 msgr2=0x5631068ba5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8660 0x5631068ba5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 -- 172.21.15.93:0/465067350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 msgr2=0x5631068b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 -- 172.21.15.93:0/465067350 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631066581f0 con 0x5631068b8210 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52a0faf700 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f528c005950 tx=0x7f528c00bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27561e9c70c21f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f5287fff700 1 -- 172.21.15.93:0/465067350 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c016070 con 0x5631068b8210 2023-07-21T22:28:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f5287fff700 1 -- 172.21.15.93:0/465067350 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f528c004570 con 0x5631068b8210 2023-07-21T22:28:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f5287fff700 1 -- 172.21.15.93:0/465067350 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c003680 con 0x5631068b8210 2023-07-21T22:28:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 -- 172.21.15.93:0/465067350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 msgr2=0x5631068bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f528c005950 tx=0x7f528c00bae0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 -- 172.21.15.93:0/465067350 shutdown_connections 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8660 0x5631068ba5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 --2- 172.21.15.93:0/465067350 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8210 0x5631068bb600 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.989+0000 7f52afb99b80 1 -- 172.21.15.93:0/465067350 >> 172.21.15.93:0/465067350 conn(0x5631068a7ae0 msgr2=0x5631068b9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 -- 172.21.15.93:0/465067350 shutdown_connections 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 -- 172.21.15.93:0/465067350 wait complete. 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 -- start start 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 Processor -- start 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.990+0000 7f52afb99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631068c1cd0 con 0x5631068b8660 2023-07-21T22:28:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631068c0170 con 0x56310670e6b0 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631068c02e0 con 0x5631068b8210 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 10 client.0 _ll_register_callbacks cb 0x7f5294097160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48062/0 (socket says 172.21.15.93:48062) 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2755408495 learned_addr learned my addr 172.21.15.93:0/2755408495 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a17b0700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a0faf700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 msgr2=0x5631068bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2755408495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 msgr2=0x5631068c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 -- 172.21.15.93:0/2755408495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c004250 con 0x5631068b8660 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52a1fb1700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f529409b010 tx=0x7f52940bb0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596018fc02918f1e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f5284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294004e50 con 0x5631068b8660 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f529409f330 con 0x5631068b8660 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5294024360 con 0x5631068b8660 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52940a34c0 con 0x5631068b8660 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 10 client.17133 fetch_fsmap learned FSMap version 58 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.991+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x563106630250 con 0x5631068b8660 2023-07-21T22:28:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5294024360 con 0x5631068b8660 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 58) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f52940a1110 con 0x5631068b8660 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 10 client.17133 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 10 client.17133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 20 client.17133 populate_metadata read hostname 'smithi093' 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56310691fff0 0x5631069223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 4 client.17133 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:28:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5631067beff0 con 0x56310691fff0 2023-07-21T22:28:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52a17b0700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56310691fff0 0x5631069223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f5284ff9700 10 client.17133 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:46.992+0000 7f52a17b0700 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56310691fff0 0x5631069223d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12f10a0cd20fb12 server_cookie=f161a11abbad2c8a in_seq=0 out_seq=0 2023-07-21T22:28:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f5284ff9700 1 -- 172.21.15.93:0/2755408495 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5631067beff0 con 0x56310691fff0 2023-07-21T22:28:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f5284ff9700 10 client.17133 handle_command_reply: tid=0 2023-07-21T22:28:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 client.17133 shutdown 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 msgr2=0x5631068bfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f529409b010 tx=0x7f52940bb0b0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 shutdown_connections 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56310691fff0 0x5631069223d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631068b8660 0x5631068bfc30 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56310670e6b0 0x5631068c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 --2- 172.21.15.93:0/2755408495 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5631068b8210 0x5631068bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f5284ff9700 0 client.17133 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:28:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 >> 172.21.15.93:0/2755408495 conn(0x5631068a7ae0 msgr2=0x5631068ba720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 shutdown_connections 2023-07-21T22:28:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 1 -- 172.21.15.93:0/2755408495 wait complete. 2023-07-21T22:28:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.036+0000 7f52afb99b80 20 client.17133 trim_cache size 0 max 16384 2023-07-21T22:28:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:28:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:28:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:28:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:28:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 msgr2=0x7f52880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 0x7f52880a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f529c0ffba0 tx=0x7f529800b410 comp rx=0 tx=0).stop 2023-07-21T22:28:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 msgr2=0x7f529c1a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f529400b830 tx=0x7f52940079b0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 shutdown_connections 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52880a12c0 0x7f52880a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f529c104a30 0x7f529c1a3e40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f529c0fea30 0x7f529c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 --2- 172.21.15.93:0/981571774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f529c0ff3e0 0x7f529c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 >> 172.21.15.93:0/981571774 conn(0x7f529c074760 msgr2=0x7f529c0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 shutdown_connections 2023-07-21T22:28:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:28:47.037+0000 7f52a3a14700 1 -- 172.21.15.93:0/981571774 wait complete. 2023-07-21T22:28:47.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 750911 6.42 MB/sec execute 564 sec latency 2311.618 ms 2023-07-21T22:28:47.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:47 smithi194 ceph-mon[98288]: pgmap v390: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:47.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:47 smithi134 ceph-mon[97873]: pgmap v390: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:47 smithi093 ceph-mon[96579]: pgmap v390: 97 pgs: 97 active+clean; 336 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2023-07-21T22:28:48.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 750911 6.41 MB/sec execute 565 sec latency 3311.698 ms 2023-07-21T22:28:49.346 INFO:tasks.workunit.client.0.smithi093.stdout: 1 751310 6.40 MB/sec execute 566 sec latency 4152.551 ms 2023-07-21T22:28:49.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:49 smithi194 ceph-mon[98288]: pgmap v391: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2023-07-21T22:28:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:49 smithi134 ceph-mon[97873]: pgmap v391: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2023-07-21T22:28:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:49 smithi093 ceph-mon[96579]: pgmap v391: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2023-07-21T22:28:50.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 753043 6.41 MB/sec execute 567 sec latency 53.869 ms 2023-07-21T22:28:50.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:50 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:50.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:50 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:50.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:50 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:51.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 754287 6.41 MB/sec execute 568 sec latency 19.750 ms 2023-07-21T22:28:51.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:51 smithi194 ceph-mon[98288]: pgmap v392: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-21T22:28:51.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:51 smithi134 ceph-mon[97873]: pgmap v392: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-21T22:28:51.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:51 smithi093 ceph-mon[96579]: pgmap v392: 97 pgs: 97 active+clean; 345 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-21T22:28:52.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 754461 6.40 MB/sec execute 569 sec latency 910.644 ms 2023-07-21T22:28:53.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 754461 6.39 MB/sec execute 570 sec latency 1910.742 ms 2023-07-21T22:28:53.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:53 smithi194 ceph-mon[98288]: pgmap v393: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T22:28:53.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:53 smithi134 ceph-mon[97873]: pgmap v393: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T22:28:53.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:53 smithi093 ceph-mon[96579]: pgmap v393: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2023-07-21T22:28:54.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 754860 6.38 MB/sec execute 571 sec latency 2747.675 ms 2023-07-21T22:28:55.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 756681 6.39 MB/sec execute 572 sec latency 13.836 ms 2023-07-21T22:28:55.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:55 smithi194 ceph-mon[98288]: pgmap v394: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:55.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:55 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:55 smithi134 ceph-mon[97873]: pgmap v394: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:55.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:55 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:55 smithi093 ceph-mon[96579]: pgmap v394: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:55 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:28:56.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 758009 6.39 MB/sec execute 573 sec latency 20.306 ms 2023-07-21T22:28:57.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 758009 6.38 MB/sec execute 574 sec latency 1018.455 ms 2023-07-21T22:28:57.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:57 smithi194 ceph-mon[98288]: pgmap v395: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:57 smithi134 ceph-mon[97873]: pgmap v395: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:57 smithi093 ceph-mon[96579]: pgmap v395: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.5 MiB/s wr, 132 op/s 2023-07-21T22:28:58.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 758009 6.36 MB/sec execute 575 sec latency 2018.508 ms 2023-07-21T22:28:59.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 758459 6.35 MB/sec execute 576 sec latency 2853.120 ms 2023-07-21T22:28:59.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:28:59 smithi194 ceph-mon[98288]: pgmap v396: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2023-07-21T22:28:59.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:28:59 smithi134 ceph-mon[97873]: pgmap v396: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2023-07-21T22:28:59.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:28:59 smithi093 ceph-mon[96579]: pgmap v396: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2023-07-21T22:29:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 -- 172.21.15.93:0/2799709538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc0ff3e0 msgr2=0x7fbfbc10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2799709538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc10b860 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400b0a0 tx=0x7fbfb400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 -- 172.21.15.93:0/2799709538 shutdown_connections 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2799709538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc10b860 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2799709538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc104a30 0x7fbfbc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2799709538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 -- 172.21.15.93:0/2799709538 >> 172.21.15.93:0/2799709538 conn(0x7fbfbc074760 msgr2=0x7fbfbc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 -- 172.21.15.93:0/2799709538 shutdown_connections 2023-07-21T22:29:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.149+0000 7fbfc1505700 1 -- 172.21.15.93:0/2799709538 wait complete. 2023-07-21T22:29:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 Processor -- start 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 -- start start 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 0x7fbfbc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc19af80 con 0x7fbfbc104a30 2023-07-21T22:29:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc19b0f0 con 0x7fbfbc0ff3e0 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfc1505700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc19b2b0 con 0x7fbfbc0fea30 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 0x7fbfbc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35152/0 (socket says 172.21.15.93:35152) 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2579856031 learned_addr learned my addr 172.21.15.93:0/2579856031 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.150+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2579856031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc0ff3e0 msgr2=0x7fbfbc1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 msgr2=0x7fbfbc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 0x7fbfbc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2579856031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009d00 con 0x7fbfbc0fea30 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 0x7fbfbc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00e970 tx=0x7fbfac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfa3fff700 1 -- 172.21.15.93:0/2579856031 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac009790 con 0x7fbfbc0fea30 2023-07-21T22:29:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc19b590 con 0x7fbfbc0fea30 2023-07-21T22:29:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfa3fff700 1 -- 172.21.15.93:0/2579856031 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfac00c7e0 con 0x7fbfbc0fea30 2023-07-21T22:29:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfa3fff700 1 -- 172.21.15.93:0/2579856031 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac003c70 con 0x7fbfbc0fea30 2023-07-21T22:29:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.151+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc1a2dd0 con 0x7fbfbc0fea30 2023-07-21T22:29:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.153+0000 7fbfa3fff700 1 -- 172.21.15.93:0/2579856031 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbfac00cc50 con 0x7fbfbc0fea30 2023-07-21T22:29:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfa3fff700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 0x7fbfa40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfa3fff700 1 -- 172.21.15.93:0/2579856031 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfac014070 con 0x7fbfbc0fea30 2023-07-21T22:29:00.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 0x7fbfa40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 0x7fbfa40a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1975b0 tx=0x7fbfb0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfcd68ab80 1 Processor -- start 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.154+0000 7fbfcd68ab80 1 -- start start 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c48fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c38fc50 con 0x56301c490660 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c3e29e0 con 0x56301c2e66b0 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c3dbe70 con 0x56301c490210 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48084/0 (socket says 172.21.15.93:48084) 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c48fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 -- 172.21.15.93:0/3568291869 learned_addr learned my addr 172.21.15.93:0/3568291869 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 -- 172.21.15.93:0/3568291869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 msgr2=0x56301c489f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 -- 172.21.15.93:0/3568291869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 msgr2=0x56301c48fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c48fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 -- 172.21.15.93:0/3568291869 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301c209d80 con 0x56301c490660 2023-07-21T22:29:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c48fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009c30 tx=0x7fbfb4009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3235b46efedbabe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfa17fa700 1 -- 172.21.15.93:0/3568291869 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400e040 con 0x56301c490660 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfa17fa700 1 -- 172.21.15.93:0/3568291869 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4009460 con 0x56301c490660 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.155+0000 7fbfa17fa700 1 -- 172.21.15.93:0/3568291869 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4005040 con 0x56301c490660 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3568291869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 msgr2=0x56301c48a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009c30 tx=0x7fbfb4009c60 comp rx=0 tx=0).stop 2023-07-21T22:29:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3568291869 shutdown_connections 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490660 0x56301c48a4a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c48fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3568291869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3568291869 >> 172.21.15.93:0/3568291869 conn(0x56301c47fae0 msgr2=0x56301c4850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3568291869 shutdown_connections 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3568291869 wait complete. 2023-07-21T22:29:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.156+0000 7fbfcd68ab80 1 -- start start 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 Processor -- start 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c4999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490660 0x56301c493740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c37ba60 con 0x56301c2e66b0 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c37b760 con 0x56301c490660 2023-07-21T22:29:00.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c25f8c0 con 0x56301c490210 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 10 client.0 _ll_register_callbacks cb 0x56301c379700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48086/0 (socket says 172.21.15.93:48086) 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfba7fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c4999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 -- 172.21.15.93:0/3877515977 learned_addr learned my addr 172.21.15.93:0/3877515977 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490660 0x56301c493740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 -- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 msgr2=0x56301c4999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c4999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 -- 172.21.15.93:0/3877515977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490660 msgr2=0x56301c493740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490660 0x56301c493740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 -- 172.21.15.93:0/3877515977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc1a2dd0 con 0x56301c2e66b0 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c4999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbfac099ea0 tx=0x7fbfac099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a76c697edf799ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbf9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0b6070 con 0x56301c2e66b0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfac0a2360 con 0x56301c2e66b0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.157+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56301c335490 con 0x56301c2e66b0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfac0b82b0 con 0x56301c2e66b0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 10 client.17145 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fbfbc10c160 con 0x56301c2e66b0 2023-07-21T22:29:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfac063080 con 0x56301c2e66b0 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbfac09a170 con 0x56301c2e66b0 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 10 client.17145 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 20 client.17145 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4a3a60 0x56301c4a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 4 client.17145 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56301c3d4e50 con 0x56301c4a3a60 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.158+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4a3a60 0x56301c4a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4a3a60 0x56301c4a5e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73538ed3186c645 server_cookie=97942d04a3e0f979 in_seq=0 out_seq=0 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbf9eff5700 10 client.17145 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbf9eff5700 1 -- 172.21.15.93:0/3877515977 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56301c3d4e50 con 0x56301c4a3a60 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbf9eff5700 10 client.17145 handle_command_reply: tid=0 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 client.17145 shutdown 2023-07-21T22:29:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 msgr2=0x56301c4994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbfac099ea0 tx=0x7fbfac099ce0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 shutdown_connections 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4a3a60 0x56301c4a5e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490660 0x56301c493740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbf9eff5700 0 client.17145 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c4994b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/3877515977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490210 0x56301c4999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 >> 172.21.15.93:0/3877515977 conn(0x56301c47fae0 msgr2=0x56301c4843e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 shutdown_connections 2023-07-21T22:29:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.159+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/3877515977 wait complete. 2023-07-21T22:29:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.160+0000 7fbfcd68ab80 20 client.17145 trim_cache size 0 max 16384 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.174+0000 7fbfcd68ab80 1 Processor -- start 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.174+0000 7fbfcd68ab80 1 -- start start 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c48eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c48ad00 con 0x56301c2e66b0 2023-07-21T22:29:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c48e180 con 0x56301c490210 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c48a480 con 0x56301c490660 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48094/0 (socket says 172.21.15.93:48094) 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2207159847 learned_addr learned my addr 172.21.15.93:0/2207159847 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c48eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2207159847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 msgr2=0x56301c48eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c48eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2207159847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490210 msgr2=0x56301c489f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 -- 172.21.15.93:0/2207159847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301c1f2c00 con 0x56301c2e66b0 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbfac096980 tx=0x7fbfac096380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca37c6d6ffb734 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfa1ffb700 1 -- 172.21.15.93:0/2207159847 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0b6070 con 0x56301c2e66b0 2023-07-21T22:29:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfa1ffb700 1 -- 172.21.15.93:0/2207159847 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56301c335490 con 0x56301c2e66b0 2023-07-21T22:29:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.175+0000 7fbfa1ffb700 1 -- 172.21.15.93:0/2207159847 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0b8340 con 0x56301c2e66b0 2023-07-21T22:29:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/2207159847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 msgr2=0x56301c491330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbfac096980 tx=0x7fbfac096380 comp rx=0 tx=0).stop 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/2207159847 shutdown_connections 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c48eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c2e66b0 0x56301c491330 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/2207159847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c490210 0x56301c489f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/2207159847 >> 172.21.15.93:0/2207159847 conn(0x56301c47fae0 msgr2=0x56301c48e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/2207159847 shutdown_connections 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/2207159847 wait complete. 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.176+0000 7fbfcd68ab80 1 -- start start 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 Processor -- start 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c4976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c4f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c4f2e00 con 0x56301c490210 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c4f2f40 con 0x56301c2e66b0 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301c4f3100 con 0x56301c490660 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfcd68ab80 10 client.0 _ll_register_callbacks cb 0x7fbfac016330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48102/0 (socket says 172.21.15.93:48102) 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 -- 172.21.15.93:0/1706698273 learned_addr learned my addr 172.21.15.93:0/1706698273 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c4976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfbb7fe700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c4f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 -- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 msgr2=0x56301c4f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c4f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 -- 172.21.15.93:0/1706698273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 msgr2=0x56301c4976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c4976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 -- 172.21.15.93:0/1706698273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc10c160 con 0x56301c490210 2023-07-21T22:29:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbfba7fc700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00097c0 tx=0x7fbfb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eceabcf29bdff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbf9eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.177+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0022070 con 0x56301c490210 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbfb002b3b0 con 0x56301c490210 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfb4009460 con 0x56301c490210 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb002b570 con 0x56301c490210 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 10 client.17151 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56301c2301f0 con 0x56301c490210 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4009460 con 0x56301c490210 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbfb001d5d0 con 0x56301c490210 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 10 client.17151 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 20 client.17151 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4fcca0 0x56301c4ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 4 client.17151 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56301c396ff0 con 0x56301c4fcca0 2023-07-21T22:29:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.178+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4fcca0 0x56301c4ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.179+0000 7fbf9eff5700 10 client.17151 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:00.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.179+0000 7fbfbaffd700 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4fcca0 0x56301c4ff0b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b36f85d7f2964e server_cookie=1e7806833a078d6b in_seq=0 out_seq=0 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbf9eff5700 1 -- 172.21.15.93:0/1706698273 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56301c396ff0 con 0x56301c4fcca0 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbf9eff5700 10 client.17151 handle_command_reply: tid=0 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 client.17151 shutdown 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 msgr2=0x56301c497c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00097c0 tx=0x7fbfb00047c0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 shutdown_connections 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56301c4fcca0 0x56301c4ff0b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56301c490660 0x56301c4f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56301c2e66b0 0x56301c4976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbf9eff5700 0 client.17151 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 --2- 172.21.15.93:0/1706698273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56301c490210 0x56301c497c00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 >> 172.21.15.93:0/1706698273 conn(0x56301c47fae0 msgr2=0x56301c48f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 shutdown_connections 2023-07-21T22:29:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.208+0000 7fbfcd68ab80 1 -- 172.21.15.93:0/1706698273 wait complete. 2023-07-21T22:29:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfcd68ab80 20 client.17151 trim_cache size 0 max 16384 2023-07-21T22:29:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:29:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:29:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:29:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 msgr2=0x7fbfa40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 0x7fbfa40a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc1975b0 tx=0x7fbfb0009260 comp rx=0 tx=0).stop 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 msgr2=0x7fbfbc195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00e970 tx=0x7fbfac00ec80 comp rx=0 tx=0).stop 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 shutdown_connections 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbfa40a12c0 0x7fbfa40a3770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbfbc104a30 0x7fbfbc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfbc0fea30 0x7fbfbc195d90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 --2- 172.21.15.93:0/2579856031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfbc0ff3e0 0x7fbfbc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 >> 172.21.15.93:0/2579856031 conn(0x7fbfbc074760 msgr2=0x7fbfbc067980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 shutdown_connections 2023-07-21T22:29:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:00.209+0000 7fbfc1505700 1 -- 172.21.15.93:0/2579856031 wait complete. 2023-07-21T22:29:00.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 760192 6.37 MB/sec execute 577 sec latency 14.442 ms 2023-07-21T22:29:01.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 760904 6.36 MB/sec execute 578 sec latency 33.919 ms 2023-07-21T22:29:01.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:01 smithi194 ceph-mon[98288]: pgmap v397: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-07-21T22:29:01.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:01 smithi134 ceph-mon[97873]: pgmap v397: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-07-21T22:29:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:01 smithi093 ceph-mon[96579]: pgmap v397: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2023-07-21T22:29:02.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 762584 6.36 MB/sec execute 579 sec latency 78.847 ms 2023-07-21T22:29:03.347 INFO:tasks.workunit.client.0.smithi093.stdout: 1 763962 6.37 MB/sec execute 580 sec latency 12.974 ms 2023-07-21T22:29:03.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:03 smithi194 ceph-mon[98288]: pgmap v398: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2023-07-21T22:29:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:03 smithi134 ceph-mon[97873]: pgmap v398: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2023-07-21T22:29:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:03 smithi093 ceph-mon[96579]: pgmap v398: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 2.3 MiB/s wr, 227 op/s 2023-07-21T22:29:04.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 765501 6.37 MB/sec execute 581 sec latency 95.174 ms 2023-07-21T22:29:05.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 767194 6.38 MB/sec execute 582 sec latency 44.020 ms 2023-07-21T22:29:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:05 smithi194 ceph-mon[98288]: pgmap v399: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-07-21T22:29:05.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:05 smithi194 ceph-mon[98288]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:29:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:05 smithi134 ceph-mon[97873]: pgmap v399: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-07-21T22:29:05.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:05 smithi134 ceph-mon[97873]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:29:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:05 smithi093 ceph-mon[96579]: pgmap v399: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2023-07-21T22:29:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:05 smithi093 ceph-mon[96579]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:29:06.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 768342 6.38 MB/sec execute 583 sec latency 25.720 ms 2023-07-21T22:29:07.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 768649 6.36 MB/sec execute 584 sec latency 773.355 ms 2023-07-21T22:29:07.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:07 smithi194 ceph-mon[98288]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2023-07-21T22:29:07.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:07 smithi134 ceph-mon[97873]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2023-07-21T22:29:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:07 smithi093 ceph-mon[96579]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2023-07-21T22:29:08.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 769670 6.36 MB/sec execute 585 sec latency 1271.443 ms 2023-07-21T22:29:09.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 771066 6.37 MB/sec execute 586 sec latency 17.808 ms 2023-07-21T22:29:09.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:09 smithi194 ceph-mon[98288]: pgmap v402: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:09 smithi134 ceph-mon[97873]: pgmap v402: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:09 smithi093 ceph-mon[96579]: pgmap v402: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:10.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 772191 6.36 MB/sec execute 587 sec latency 282.657 ms 2023-07-21T22:29:11.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 773510 6.37 MB/sec execute 588 sec latency 601.116 ms 2023-07-21T22:29:11.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:11 smithi194 ceph-mon[98288]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:11.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:11 smithi134 ceph-mon[97873]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:11 smithi093 ceph-mon[96579]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:29:12.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 774833 6.37 MB/sec execute 589 sec latency 16.099 ms 2023-07-21T22:29:13.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 775740 6.36 MB/sec execute 590 sec latency 433.462 ms 2023-07-21T22:29:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 -- 172.21.15.93:0/1677294875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5260101700 msgr2=0x7f5260109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 --2- 172.21.15.93:0/1677294875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5260101700 0x7f5260109bf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5250009990 tx=0x7f5250009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 -- 172.21.15.93:0/1677294875 shutdown_connections 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 --2- 172.21.15.93:0/1677294875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f526010a130 0x7f526010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 --2- 172.21.15.93:0/1677294875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260100df0 0x7f52601011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 --2- 172.21.15.93:0/1677294875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5260101700 0x7f5260109bf0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 -- 172.21.15.93:0/1677294875 >> 172.21.15.93:0/1677294875 conn(0x7f5260075bd0 msgr2=0x7f5260075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 -- 172.21.15.93:0/1677294875 shutdown_connections 2023-07-21T22:29:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 -- 172.21.15.93:0/1677294875 wait complete. 2023-07-21T22:29:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.317+0000 7f5265178700 1 Processor -- start 2023-07-21T22:29:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 -- start start 2023-07-21T22:29:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5260100df0 0x7f526019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260101700 0x7f526019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5260196050 con 0x7f526010a130 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52601961c0 con 0x7f5260100df0 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f5265178700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5260196380 con 0x7f5260101700 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48118/0 (socket says 172.21.15.93:48118) 2023-07-21T22:29:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 -- 172.21.15.93:0/1851084679 learned_addr learned my addr 172.21.15.93:0/1851084679 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5260100df0 0x7f526019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525e59c700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260101700 0x7f526019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 -- 172.21.15.93:0/1851084679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260101700 msgr2=0x7f526019c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260101700 0x7f526019c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 -- 172.21.15.93:0/1851084679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5260100df0 msgr2=0x7f526019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5260100df0 0x7f526019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 -- 172.21.15.93:0/1851084679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500095f0 con 0x7f526010a130 2023-07-21T22:29:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.318+0000 7f525f59e700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5258007ee0 tx=0x7f525800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.319+0000 7f5247fff700 1 -- 172.21.15.93:0/1851084679 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258017070 con 0x7f526010a130 2023-07-21T22:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.319+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5260196660 con 0x7f526010a130 2023-07-21T22:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.319+0000 7f5247fff700 1 -- 172.21.15.93:0/1851084679 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525800f040 con 0x7f526010a130 2023-07-21T22:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.319+0000 7f5247fff700 1 -- 172.21.15.93:0/1851084679 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258005740 con 0x7f526010a130 2023-07-21T22:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.319+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52601a2b70 con 0x7f526010a130 2023-07-21T22:29:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.321+0000 7f5247fff700 1 -- 172.21.15.93:0/1851084679 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f525800dc60 con 0x7f526010a130 2023-07-21T22:29:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.321+0000 7f5247fff700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 0x7f52480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.321+0000 7f5247fff700 1 -- 172.21.15.93:0/1851084679 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f52580cc470 con 0x7f526010a130 2023-07-21T22:29:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.321+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 0x7f52480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 0x7f52480a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5254005950 tx=0x7f525400b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f52712fdb80 1 Processor -- start 2023-07-21T22:29:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f52712fdb80 1 -- start start 2023-07-21T22:29:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca7ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.322+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f660 0x55b05ca794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52712fdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c97ec50 con 0x55b05ca7f210 2023-07-21T22:29:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52712fdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c9d19e0 con 0x55b05ca7f660 2023-07-21T22:29:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52712fdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c9cae70 con 0x55b05c8d56b0 2023-07-21T22:29:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48124/0 (socket says 172.21.15.93:48124) 2023-07-21T22:29:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 -- 172.21.15.93:0/4236332532 learned_addr learned my addr 172.21.15.93:0/4236332532 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 -- 172.21.15.93:0/4236332532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 msgr2=0x55b05ca7ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525ed9d700 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca7ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525f59e700 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f660 0x55b05ca794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca7ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 -- 172.21.15.93:0/4236332532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f660 msgr2=0x55b05ca794a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f660 0x55b05ca794a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 -- 172.21.15.93:0/4236332532 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05c7f8d80 con 0x55b05ca7f210 2023-07-21T22:29:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f525e59c700 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5250006010 tx=0x7f5250005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e35d15aa5b12a7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52457fa700 1 -- 172.21.15.93:0/4236332532 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000b7d0 con 0x55b05ca7f210 2023-07-21T22:29:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52457fa700 1 -- 172.21.15.93:0/4236332532 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525001e040 con 0x55b05ca7f210 2023-07-21T22:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.323+0000 7f52457fa700 1 -- 172.21.15.93:0/4236332532 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250005490 con 0x55b05ca7f210 2023-07-21T22:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- 172.21.15.93:0/4236332532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 msgr2=0x55b05ca78f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5250006010 tx=0x7f5250005e80 comp rx=0 tx=0).stop 2023-07-21T22:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- 172.21.15.93:0/4236332532 shutdown_connections 2023-07-21T22:29:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f660 0x55b05ca794a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca7ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 --2- 172.21.15.93:0/4236332532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca78f40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- 172.21.15.93:0/4236332532 >> 172.21.15.93:0/4236332532 conn(0x55b05ca6eae0 msgr2=0x55b05ca740e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- 172.21.15.93:0/4236332532 shutdown_connections 2023-07-21T22:29:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- 172.21.15.93:0/4236332532 wait complete. 2023-07-21T22:29:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 -- start start 2023-07-21T22:29:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.324+0000 7f52712fdb80 1 Processor -- start 2023-07-21T22:29:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca88550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 0x55b05ca88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c96aa60 con 0x55b05ca7f660 2023-07-21T22:29:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c96a760 con 0x55b05ca7f210 2023-07-21T22:29:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05c84e8c0 con 0x55b05c8d56b0 2023-07-21T22:29:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 10 client.0 _ll_register_callbacks cb 0x55b05c968700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48132/0 (socket says 172.21.15.93:48132) 2023-07-21T22:29:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 -- 172.21.15.93:0/3173816881 learned_addr learned my addr 172.21.15.93:0/3173816881 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525e59c700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 0x55b05ca88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525ed9d700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca88550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 -- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 msgr2=0x55b05ca88550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca88550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 -- 172.21.15.93:0/3173816881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 msgr2=0x55b05ca88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 0x55b05ca88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 -- 172.21.15.93:0/3173816881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526010d250 con 0x55b05ca7f660 2023-07-21T22:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525ed9d700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca88550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525e59c700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 0x55b05ca88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f525f59e700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5258099240 tx=0x7f52580993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95c55507463a9a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f5242ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525809d370 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f525809d570 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b05c924490 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52580a1990 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 10 client.17163 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.325+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55b05c7e1c00 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52580b5080 con 0x55b05ca7f660 2023-07-21T22:29:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f52580997a0 con 0x55b05ca7f660 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 10 client.17163 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 20 client.17163 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05ca92aa0 0x55b05ca94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 4 client.17163 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b05c9c3e50 con 0x55b05ca92aa0 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f525ed9d700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05ca92aa0 0x55b05ca94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f5242ff5700 10 client.17163 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.326+0000 7f525ed9d700 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05ca92aa0 0x55b05ca94eb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f0584ee0ad6e22 server_cookie=777ff3820499ac66 in_seq=0 out_seq=0 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f5242ff5700 1 -- 172.21.15.93:0/3173816881 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b05c9c3e50 con 0x55b05ca92aa0 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f5242ff5700 10 client.17163 handle_command_reply: tid=0 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 client.17163 shutdown 2023-07-21T22:29:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 msgr2=0x55b05ca827e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5258099240 tx=0x7f52580993e0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 shutdown_connections 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05ca92aa0 0x55b05ca94eb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05ca827e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f5242ff5700 0 client.17163 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05c8d56b0 0x55b05ca88550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 --2- 172.21.15.93:0/3173816881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05ca7f210 0x55b05ca88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 >> 172.21.15.93:0/3173816881 conn(0x55b05ca6eae0 msgr2=0x55b05ca733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 shutdown_connections 2023-07-21T22:29:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 1 -- 172.21.15.93:0/3173816881 wait complete. 2023-07-21T22:29:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.327+0000 7f52712fdb80 20 client.17163 trim_cache size 0 max 16384 2023-07-21T22:29:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 Processor -- start 2023-07-21T22:29:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 -- start start 2023-07-21T22:29:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca80140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f660 0x55b05ca740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ca900d0 con 0x55b05ca7f210 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ca83c70 con 0x55b05c8d56b0 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f52712fdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ca7d240 con 0x55b05ca7f660 2023-07-21T22:29:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48138/0 (socket says 172.21.15.93:48138) 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 -- 172.21.15.93:0/309615000 learned_addr learned my addr 172.21.15.93:0/309615000 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525ed9d700 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca80140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 -- 172.21.15.93:0/309615000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f660 msgr2=0x55b05ca740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f660 0x55b05ca740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 -- 172.21.15.93:0/309615000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 msgr2=0x55b05ca80140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca80140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 -- 172.21.15.93:0/309615000 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05c81f1f0 con 0x55b05ca7f210 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525ed9d700 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca80140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f525e59c700 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5250025040 tx=0x7f5250019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b4a38d2be9271e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.342+0000 7f5245ffb700 1 -- 172.21.15.93:0/309615000 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250003a90 con 0x55b05ca7f210 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f5245ffb700 1 -- 172.21.15.93:0/309615000 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5250005570 con 0x55b05ca7f210 2023-07-21T22:29:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f5245ffb700 1 -- 172.21.15.93:0/309615000 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000f990 con 0x55b05ca7f210 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- 172.21.15.93:0/309615000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 msgr2=0x55b05ca73b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5250025040 tx=0x7f5250019040 comp rx=0 tx=0).stop 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- 172.21.15.93:0/309615000 shutdown_connections 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f660 0x55b05ca740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca80140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 --2- 172.21.15.93:0/309615000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f210 0x55b05ca73b60 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- 172.21.15.93:0/309615000 >> 172.21.15.93:0/309615000 conn(0x55b05ca6eae0 msgr2=0x55b05ca7fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- 172.21.15.93:0/309615000 shutdown_connections 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- 172.21.15.93:0/309615000 wait complete. 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 -- start start 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.343+0000 7f52712fdb80 1 Processor -- start 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 0x55b05ca86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ca87140 con 0x55b05ca7f660 2023-07-21T22:29:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05cae5ea0 con 0x55b05c8d56b0 2023-07-21T22:29:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05cae6030 con 0x55b05ca7f210 2023-07-21T22:29:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 10 client.0 _ll_register_callbacks cb 0x7f52580958a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48146/0 (socket says 172.21.15.93:48146) 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 -- 172.21.15.93:0/1897723568 learned_addr learned my addr 172.21.15.93:0/1897723568 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525e59c700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 0x55b05ca86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 -- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 msgr2=0x55b05ca86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 0x55b05ca86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 -- 172.21.15.93:0/1897723568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 msgr2=0x55b05ca864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 -- 172.21.15.93:0/1897723568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b05c7f8d80 con 0x55b05ca7f660 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525e59c700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 0x55b05ca86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f525f59e700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f525800d7b0 tx=0x7f5258000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d48d8d61e41c92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f5242ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525809d370 con 0x55b05ca7f660 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f525809d570 con 0x55b05ca7f660 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f525001e040 con 0x55b05ca7f660 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.344+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5258098ea0 con 0x55b05ca7f660 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 10 client.17169 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55b05c7f7250 con 0x55b05ca7f660 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525001e040 con 0x55b05ca7f660 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f52580058a0 con 0x55b05ca7f660 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 10 client.17169 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 20 client.17169 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05cae7450 0x55b05cae9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 4 client.17169 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b05c985ff0 con 0x55b05cae7450 2023-07-21T22:29:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05cae7450 0x55b05cae9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f5242ff5700 10 client.17169 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.345+0000 7f525ed9d700 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05cae7450 0x55b05cae9860 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b51e5e62e62a24 server_cookie=a115623c0175cbc4 in_seq=0 out_seq=0 2023-07-21T22:29:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f5242ff5700 1 -- 172.21.15.93:0/1897723568 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b05c985ff0 con 0x55b05cae7450 2023-07-21T22:29:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f5242ff5700 10 client.17169 handle_command_reply: tid=0 2023-07-21T22:29:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 client.17169 shutdown 2023-07-21T22:29:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 msgr2=0x55b05cae5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f525800d7b0 tx=0x7f5258000f40 comp rx=0 tx=0).stop 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 shutdown_connections 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05cae7450 0x55b05cae9860 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b05ca7f660 0x55b05cae5960 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05c8d56b0 0x55b05ca864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f5242ff5700 0 client.17169 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 --2- 172.21.15.93:0/1897723568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05ca7f210 0x55b05ca86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 >> 172.21.15.93:0/1897723568 conn(0x55b05ca6eae0 msgr2=0x55b05ca82780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 shutdown_connections 2023-07-21T22:29:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.387+0000 7f52712fdb80 1 -- 172.21.15.93:0/1897723568 wait complete. 2023-07-21T22:29:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f52712fdb80 20 client.17169 trim_cache size 0 max 16384 2023-07-21T22:29:13.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:29:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:29:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:29:13.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:29:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 msgr2=0x7f52480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 0x7f52480a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5254005950 tx=0x7f525400b410 comp rx=0 tx=0).stop 2023-07-21T22:29:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 msgr2=0x7f52601959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5258007ee0 tx=0x7f525800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 shutdown_connections 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f52480a12c0 0x7f52480a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f526010a130 0x7f52601959e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5260100df0 0x7f526019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 --2- 172.21.15.93:0/1851084679 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260101700 0x7f526019c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 >> 172.21.15.93:0/1851084679 conn(0x7f5260075bd0 msgr2=0x7f526010ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 shutdown_connections 2023-07-21T22:29:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:13.388+0000 7f5265178700 1 -- 172.21.15.93:0/1851084679 wait complete. 2023-07-21T22:29:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:13 smithi194 ceph-mon[98288]: pgmap v404: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:13 smithi194 ceph-mon[98288]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:29:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:13 smithi134 ceph-mon[97873]: pgmap v404: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:13 smithi134 ceph-mon[97873]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:29:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:13 smithi093 ceph-mon[96579]: pgmap v404: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:13 smithi093 ceph-mon[96579]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:29:14.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 776112 6.35 MB/sec execute 591 sec latency 1287.573 ms 2023-07-21T22:29:15.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 777870 6.37 MB/sec execute 592 sec latency 15.059 ms 2023-07-21T22:29:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:15 smithi194 ceph-mon[98288]: pgmap v406: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-07-21T22:29:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:15 smithi134 ceph-mon[97873]: pgmap v406: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-07-21T22:29:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:15 smithi093 ceph-mon[96579]: pgmap v406: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2023-07-21T22:29:16.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 779149 6.36 MB/sec execute 593 sec latency 18.670 ms 2023-07-21T22:29:17.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 779282 6.35 MB/sec execute 594 sec latency 922.137 ms 2023-07-21T22:29:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:17 smithi194 ceph-mon[98288]: pgmap v407: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:17 smithi134 ceph-mon[97873]: pgmap v407: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:17 smithi093 ceph-mon[96579]: pgmap v407: 97 pgs: 97 active+clean; 357 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 204 op/s 2023-07-21T22:29:18.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 779282 6.34 MB/sec execute 595 sec latency 1922.268 ms 2023-07-21T22:29:19.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 779573 6.33 MB/sec execute 596 sec latency 2769.094 ms 2023-07-21T22:29:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:19 smithi194 ceph-mon[98288]: pgmap v408: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:19 smithi134 ceph-mon[97873]: pgmap v408: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:19 smithi093 ceph-mon[96579]: pgmap v408: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:20.348 INFO:tasks.workunit.client.0.smithi093.stdout: 1 781411 6.34 MB/sec execute 597 sec latency 15.688 ms 2023-07-21T22:29:21.349 INFO:tasks.workunit.client.0.smithi093.stdout: 1 782645 6.34 MB/sec execute 598 sec latency 51.376 ms 2023-07-21T22:29:21.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:21 smithi194 ceph-mon[98288]: pgmap v409: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:21.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:21 smithi134 ceph-mon[97873]: pgmap v409: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:21 smithi093 ceph-mon[96579]: pgmap v409: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2023-07-21T22:29:22.349 INFO:tasks.workunit.client.0.smithi093.stdout: 1 782831 6.33 MB/sec execute 599 sec latency 846.341 ms 2023-07-21T22:29:23.349 INFO:tasks.workunit.client.0.smithi093.stdout: 1 cleanup 600 sec 2023-07-21T22:29:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:23 smithi194 ceph-mon[98288]: pgmap v410: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-07-21T22:29:23.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:23 smithi134 ceph-mon[97873]: pgmap v410: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-07-21T22:29:23.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:23 smithi093 ceph-mon[96579]: pgmap v410: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 1.9 MiB/s wr, 169 op/s 2023-07-21T22:29:24.349 INFO:tasks.workunit.client.0.smithi093.stdout: 1 cleanup 601 sec 2023-07-21T22:29:24.382 INFO:tasks.workunit.client.0.smithi093.stdout: 0 cleanup 601 sec 2023-07-21T22:29:24.382 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:29:24.382 INFO:tasks.workunit.client.0.smithi093.stdout: Operation Count AvgLat MaxLat 2023-07-21T22:29:24.382 INFO:tasks.workunit.client.0.smithi093.stdout: ---------------------------------------- 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: NTCreateX 121197 2.540 4994.844 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Close 89008 0.002 0.064 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Rename 5134 14.098 4969.718 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Unlink 24499 1.914 4483.030 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Deltree 2 96.773 133.982 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Mkdir 1 0.004 0.004 2023-07-21T22:29:24.383 INFO:tasks.workunit.client.0.smithi093.stdout: Qpathinfo 109876 1.192 262.016 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: Qfileinfo 19212 0.001 0.039 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: Qfsinfo 20140 0.205 9.441 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: Sfileinfo 9888 0.011 5.959 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: Find 42463 0.181 92.152 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: WriteX 60224 0.064 57.699 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: ReadX 189901 0.006 2.394 2023-07-21T22:29:24.384 INFO:tasks.workunit.client.0.smithi093.stdout: LockX 394 0.548 1.540 2023-07-21T22:29:24.385 INFO:tasks.workunit.client.0.smithi093.stdout: UnlockX 394 0.545 1.515 2023-07-21T22:29:24.385 INFO:tasks.workunit.client.0.smithi093.stdout: Flush 8504 2.793 57.051 2023-07-21T22:29:24.385 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:29:24.385 INFO:tasks.workunit.client.0.smithi093.stdout:Throughput 6.33294 MB/sec 1 clients 1 procs max_latency=4994.855 ms 2023-07-21T22:29:24.385 INFO:tasks.workunit.client.0.smithi093.stderr:+ dbench 10 2023-07-21T22:29:24.386 INFO:tasks.workunit.client.0.smithi093.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T22:29:24.386 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:29:24.386 INFO:tasks.workunit.client.0.smithi093.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T22:29:24.387 INFO:tasks.workunit.client.0.smithi093.stdout:8 of 10 processes prepared for launch 0 sec 2023-07-21T22:29:24.484 INFO:tasks.workunit.client.0.smithi093.stdout:10 of 10 processes prepared for launch 0 sec 2023-07-21T22:29:24.484 INFO:tasks.workunit.client.0.smithi093.stdout:releasing clients 2023-07-21T22:29:25.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326 118.75 MB/sec warmup 1 sec latency 47.193 ms 2023-07-21T22:29:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:25 smithi194 ceph-mon[98288]: pgmap v411: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:25 smithi134 ceph-mon[97873]: pgmap v411: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:25 smithi093 ceph-mon[96579]: pgmap v411: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:26.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 678 102.19 MB/sec warmup 2 sec latency 604.645 ms 2023-07-21T22:29:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 -- 172.21.15.93:0/413314720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 msgr2=0x7f54f4104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/413314720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f4104770 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f54e40099c0 tx=0x7f54e4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 -- 172.21.15.93:0/413314720 shutdown_connections 2023-07-21T22:29:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/413314720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4104e50 0x7f54f410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/413314720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4067c10 0x7f54f4067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/413314720 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f4104770 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 -- 172.21.15.93:0/413314720 >> 172.21.15.93:0/413314720 conn(0x7f54f4074760 msgr2=0x7f54f4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 -- 172.21.15.93:0/413314720 shutdown_connections 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.509+0000 7f54fc4ce700 1 -- 172.21.15.93:0/413314720 wait complete. 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 Processor -- start 2023-07-21T22:29:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 -- start start 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 0x7f54f40fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 0x7f54f41a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a42e0 con 0x7f54f40685b0 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a4420 con 0x7f54f4067c10 2023-07-21T22:29:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fc4ce700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a4590 con 0x7f54f4104e50 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54f9a69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54f9a69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48158/0 (socket says 172.21.15.93:48158) 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54f9a69700 1 -- 172.21.15.93:0/801230544 learned_addr learned my addr 172.21.15.93:0/801230544 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 0x7f54f40fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54faa6b700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 0x7f54f41a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 -- 172.21.15.93:0/801230544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 msgr2=0x7f54f41a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 0x7f54f41a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 -- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 msgr2=0x7f54f40ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54fa26a700 1 -- 172.21.15.93:0/801230544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40095f0 con 0x7f54f4067c10 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54f9a69700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.510+0000 7f54faa6b700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 0x7f54f41a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54fa26a700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 0x7f54f40fec40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9f0 tx=0x7f54f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54eb7fe700 1 -- 172.21.15.93:0/801230544 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f000c9c0 con 0x7f54f4067c10 2023-07-21T22:29:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f41a4870 con 0x7f54f4067c10 2023-07-21T22:29:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54eb7fe700 1 -- 172.21.15.93:0/801230544 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f0010430 con 0x7f54f4067c10 2023-07-21T22:29:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54eb7fe700 1 -- 172.21.15.93:0/801230544 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0018680 con 0x7f54f4067c10 2023-07-21T22:29:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.511+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f41a4d50 con 0x7f54f4067c10 2023-07-21T22:29:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.513+0000 7f54eb7fe700 1 -- 172.21.15.93:0/801230544 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f54f0010860 con 0x7f54f4067c10 2023-07-21T22:29:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.514+0000 7f54eb7fe700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 0x7f54e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.514+0000 7f54f9a69700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 0x7f54e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.514+0000 7f54eb7fe700 1 -- 172.21.15.93:0/801230544 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f54f0014070 con 0x7f54f4067c10 2023-07-21T22:29:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.514+0000 7f54f9a69700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 0x7f54e00a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54e40036c0 tx=0x7f54e401a040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 Processor -- start 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 -- start start 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 0x55ca557cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca556cbc50 con 0x55ca557cc210 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5571e9e0 con 0x55ca556226b0 2023-07-21T22:29:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f5508653b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55717e70 con 0x55ca557cc660 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f54f9a69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f54f9a69700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48166/0 (socket says 172.21.15.93:48166) 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f54f9a69700 1 -- 172.21.15.93:0/3738813946 learned_addr learned my addr 172.21.15.93:0/3738813946 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 0x55ca557cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.515+0000 7f54faa6b700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 -- 172.21.15.93:0/3738813946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 msgr2=0x55ca557c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 -- 172.21.15.93:0/3738813946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 msgr2=0x55ca557c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 -- 172.21.15.93:0/3738813946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca55545d80 con 0x55ca556226b0 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54f9a69700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 0x55ca557cbcd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ca590 tx=0x7f54f0096380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f659ee773b09e3e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54e8ff9700 1 -- 172.21.15.93:0/3738813946 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a2360 con 0x55ca556226b0 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54e8ff9700 1 -- 172.21.15.93:0/3738813946 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f00105a0 con 0x55ca556226b0 2023-07-21T22:29:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.516+0000 7f54e8ff9700 1 -- 172.21.15.93:0/3738813946 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a1ed0 con 0x55ca556226b0 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 -- 172.21.15.93:0/3738813946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 msgr2=0x55ca557cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 0x55ca557cbcd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ca590 tx=0x7f54f0096380 comp rx=0 tx=0).stop 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 -- 172.21.15.93:0/3738813946 shutdown_connections 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca556226b0 0x55ca557cbcd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 --2- 172.21.15.93:0/3738813946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca557cc210 0x55ca557c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 -- 172.21.15.93:0/3738813946 >> 172.21.15.93:0/3738813946 conn(0x55ca557bbae0 msgr2=0x55ca557c10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 -- 172.21.15.93:0/3738813946 shutdown_connections 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.517+0000 7f5508653b80 1 -- 172.21.15.93:0/3738813946 wait complete. 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 -- start start 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 Processor -- start 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca556b7a60 con 0x55ca556226b0 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca556b7760 con 0x55ca557cc210 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5559b8c0 con 0x55ca557cc660 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f5508653b80 10 client.0 _ll_register_callbacks cb 0x55ca556b5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48170/0 (socket says 172.21.15.93:48170) 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 -- 172.21.15.93:0/3685380555 learned_addr learned my addr 172.21.15.93:0/3685380555 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54faa6b700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54f9a69700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 -- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 msgr2=0x55ca557cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 -- 172.21.15.93:0/3685380555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 msgr2=0x55ca557cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.518+0000 7f54fa26a700 1 -- 172.21.15.93:0/3685380555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f410d210 con 0x55ca556226b0 2023-07-21T22:29:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54fa26a700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ca640 tx=0x7f54f0099630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437a11caf866db7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a2360 con 0x55ca556226b0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f54f00ca310 con 0x55ca556226b0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca55671490 con 0x55ca556226b0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f00a20e0 con 0x55ca556226b0 2023-07-21T22:29:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 10 client.17181 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f54f41a4d50 con 0x55ca556226b0 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca55671490 con 0x55ca556226b0 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f54f000cc00 con 0x55ca556226b0 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 10 client.17181 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 20 client.17181 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca557df680 0x55ca557e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 4 client.17181 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca55710e50 con 0x55ca557df680 2023-07-21T22:29:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.519+0000 7f54f9a69700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca557df680 0x55ca557e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.520+0000 7f54ddffb700 10 client.17181 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.520+0000 7f54f9a69700 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca557df680 0x55ca557e1a90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1dcdb1ace2a99e server_cookie=fa2991cb15a2d86b in_seq=0 out_seq=0 2023-07-21T22:29:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f54ddffb700 1 -- 172.21.15.93:0/3685380555 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca55710e50 con 0x55ca557df680 2023-07-21T22:29:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f54ddffb700 10 client.17181 handle_command_reply: tid=0 2023-07-21T22:29:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 client.17181 shutdown 2023-07-21T22:29:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 msgr2=0x55ca557d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ca640 tx=0x7f54f0099630 comp rx=0 tx=0).stop 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 shutdown_connections 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca557df680 0x55ca557e1a90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f54ddffb700 0 client.17181 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d25e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 --2- 172.21.15.93:0/3685380555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 >> 172.21.15.93:0/3685380555 conn(0x55ca557bbae0 msgr2=0x55ca557c03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 shutdown_connections 2023-07-21T22:29:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 1 -- 172.21.15.93:0/3685380555 wait complete. 2023-07-21T22:29:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.522+0000 7f5508653b80 20 client.17181 trim_cache size 0 max 16384 2023-07-21T22:29:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 Processor -- start 2023-07-21T22:29:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 -- start start 2023-07-21T22:29:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55710e50 con 0x55ca556226b0 2023-07-21T22:29:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55671490 con 0x55ca557cc210 2023-07-21T22:29:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f5508653b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca557bd4f0 con 0x55ca557cc660 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48178/0 (socket says 172.21.15.93:48178) 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f54fa26a700 1 -- 172.21.15.93:0/795656244 learned_addr learned my addr 172.21.15.93:0/795656244 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f54f9a69700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.537+0000 7f54faa6b700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 -- 172.21.15.93:0/795656244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 msgr2=0x55ca557cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 -- 172.21.15.93:0/795656244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 msgr2=0x55ca557d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 -- 172.21.15.93:0/795656244 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca557e66c0 con 0x55ca556226b0 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54f9a69700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54fa26a700 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f54f0096410 tx=0x7f54f00107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c508ebfc85b347 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54e97fa700 1 -- 172.21.15.93:0/795656244 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00996a0 con 0x55ca556226b0 2023-07-21T22:29:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54e97fa700 1 -- 172.21.15.93:0/795656244 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f0096250 con 0x55ca556226b0 2023-07-21T22:29:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.538+0000 7f54e97fa700 1 -- 172.21.15.93:0/795656244 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a2150 con 0x55ca556226b0 2023-07-21T22:29:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- 172.21.15.93:0/795656244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 msgr2=0x55ca557cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f54f0096410 tx=0x7f54f00107d0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- 172.21.15.93:0/795656244 shutdown_connections 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc660 0x55ca557cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557cb180 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- 172.21.15.93:0/795656244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc210 0x55ca557d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- 172.21.15.93:0/795656244 >> 172.21.15.93:0/795656244 conn(0x55ca557bbae0 msgr2=0x55ca557ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- 172.21.15.93:0/795656244 shutdown_connections 2023-07-21T22:29:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- 172.21.15.93:0/795656244 wait complete. 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 -- start start 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 Processor -- start 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.539+0000 7f5508653b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc210 0x55ca557d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc660 0x55ca55832530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55832a70 con 0x55ca556226b0 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55832bb0 con 0x55ca557cc660 2023-07-21T22:29:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca55832d70 con 0x55ca557cc210 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 10 client.0 _ll_register_callbacks cb 0x7f54e4005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48184/0 (socket says 172.21.15.93:48184) 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 -- 172.21.15.93:0/2727368967 learned_addr learned my addr 172.21.15.93:0/2727368967 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54f9a69700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc210 0x55ca557d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54faa6b700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc660 0x55ca55832530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 -- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc210 msgr2=0x55ca557d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc210 0x55ca557d3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 -- 172.21.15.93:0/2727368967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc660 msgr2=0x55ca55832530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc660 0x55ca55832530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 -- 172.21.15.93:0/2727368967 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e4003d10 con 0x55ca556226b0 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54fa26a700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f54f00963c0 tx=0x7f54f0005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fef07ac4c06423e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0096090 con 0x55ca556226b0 2023-07-21T22:29:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f54f0095ea0 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54f00105a0 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f00a2360 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 10 client.17187 fetch_fsmap learned FSMap version 60 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f54f41a4d50 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.540+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f00105a0 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 60) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f54f007f020 con 0x55ca556226b0 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 10 client.17187 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T22:29:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 20 client.17187 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca55834030 0x55ca55836410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 4 client.17187 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca556d2ff0 con 0x55ca55834030 2023-07-21T22:29:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f54f9a69700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca55834030 0x55ca55836410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f54ddffb700 10 client.17187 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.541+0000 7f54f9a69700 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca55834030 0x55ca55836410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5745ac28e13e1ef8 server_cookie=1087d7674a7e8d07 in_seq=0 out_seq=0 2023-07-21T22:29:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f54ddffb700 1 -- 172.21.15.93:0/2727368967 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca556d2ff0 con 0x55ca55834030 2023-07-21T22:29:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f54ddffb700 10 client.17187 handle_command_reply: tid=0 2023-07-21T22:29:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 client.17187 shutdown 2023-07-21T22:29:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 msgr2=0x55ca557d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f54f00963c0 tx=0x7f54f0005b40 comp rx=0 tx=0).stop 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 shutdown_connections 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca55834030 0x55ca55836410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca557cc660 0x55ca55832530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f54ddffb700 0 client.17187 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca556226b0 0x55ca557d3300 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 --2- 172.21.15.93:0/2727368967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca557cc210 0x55ca557d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 >> 172.21.15.93:0/2727368967 conn(0x55ca557bbae0 msgr2=0x55ca557caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 shutdown_connections 2023-07-21T22:29:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.854+0000 7f5508653b80 1 -- 172.21.15.93:0/2727368967 wait complete. 2023-07-21T22:29:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f5508653b80 20 client.17187 trim_cache size 0 max 16384 2023-07-21T22:29:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:29:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:29:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:29:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:29:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 msgr2=0x7f54e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 0x7f54e00a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f54e40036c0 tx=0x7f54e401a040 comp rx=0 tx=0).stop 2023-07-21T22:29:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 msgr2=0x7f54f40fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 0x7f54f40fec40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9f0 tx=0x7f54f000ed00 comp rx=0 tx=0).stop 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 shutdown_connections 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54f4104e50 0x7f54f41a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4067c10 0x7f54f40fec40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54e00a12c0 0x7f54e00a3770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 --2- 172.21.15.93:0/801230544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54f40685b0 0x7f54f40ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 >> 172.21.15.93:0/801230544 conn(0x7f54f4074760 msgr2=0x7f54f4103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 shutdown_connections 2023-07-21T22:29:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:26.855+0000 7f54fc4ce700 1 -- 172.21.15.93:0/801230544 wait complete. 2023-07-21T22:29:27.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 929 71.04 MB/sec warmup 3 sec latency 983.895 ms 2023-07-21T22:29:27.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:27 smithi194 ceph-mon[98288]: pgmap v412: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:27.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:27 smithi134 ceph-mon[97873]: pgmap v412: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:27 smithi093 ceph-mon[96579]: pgmap v412: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2023-07-21T22:29:28.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 1412 63.89 MB/sec warmup 4 sec latency 183.555 ms 2023-07-21T22:29:28.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:28 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:28.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:28 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:28.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:28 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:29.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 2188 54.44 MB/sec warmup 5 sec latency 1183.583 ms 2023-07-21T22:29:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:29 smithi194 ceph-mon[98288]: pgmap v413: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2023-07-21T22:29:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:29 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:29.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:29 smithi134 ceph-mon[97873]: pgmap v413: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2023-07-21T22:29:29.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:29 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:29 smithi093 ceph-mon[96579]: pgmap v413: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2023-07-21T22:29:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:29 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:30.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 2639 52.22 MB/sec warmup 6 sec latency 2183.610 ms 2023-07-21T22:29:31.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 3208 51.91 MB/sec warmup 7 sec latency 3183.648 ms 2023-07-21T22:29:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:31 smithi194 ceph-mon[98288]: pgmap v414: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.6 MiB/s wr, 267 op/s 2023-07-21T22:29:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:31 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:31 smithi134 ceph-mon[97873]: pgmap v414: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.6 MiB/s wr, 267 op/s 2023-07-21T22:29:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:31 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:31 smithi093 ceph-mon[96579]: pgmap v414: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.6 MiB/s wr, 267 op/s 2023-07-21T22:29:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:31 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:32.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 3482 47.37 MB/sec warmup 8 sec latency 4183.748 ms 2023-07-21T22:29:33.484 INFO:tasks.workunit.client.0.smithi093.stdout: 10 3643 42.47 MB/sec warmup 9 sec latency 4574.641 ms 2023-07-21T22:29:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:33 smithi194 ceph-mon[98288]: pgmap v415: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T22:29:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:33 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:33 smithi134 ceph-mon[97873]: pgmap v415: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T22:29:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:33 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:33 smithi093 ceph-mon[96579]: pgmap v415: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 358 op/s 2023-07-21T22:29:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:33 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:34.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 3826 38.39 MB/sec warmup 10 sec latency 73.836 ms 2023-07-21T22:29:35.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 4069 35.52 MB/sec warmup 11 sec latency 131.323 ms 2023-07-21T22:29:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:35 smithi194 ceph-mon[98288]: pgmap v416: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:35 smithi134 ceph-mon[97873]: pgmap v416: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:35 smithi093 ceph-mon[96579]: pgmap v416: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:36.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 4443 33.79 MB/sec warmup 12 sec latency 110.578 ms 2023-07-21T22:29:36.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:36.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:36.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:37.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 4908 33.43 MB/sec warmup 13 sec latency 119.572 ms 2023-07-21T22:29:37.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:37 smithi194 ceph-mon[98288]: pgmap v417: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:37 smithi134 ceph-mon[97873]: pgmap v417: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:37 smithi093 ceph-mon[96579]: pgmap v417: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 2.1 MiB/s wr, 304 op/s 2023-07-21T22:29:38.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 5600 32.32 MB/sec warmup 14 sec latency 115.559 ms 2023-07-21T22:29:39.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 6045 33.02 MB/sec warmup 15 sec latency 51.296 ms 2023-07-21T22:29:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:39 smithi194 ceph-mon[98288]: pgmap v418: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2023-07-21T22:29:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:39 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:39 smithi134 ceph-mon[97873]: pgmap v418: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2023-07-21T22:29:39.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:39 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:39 smithi093 ceph-mon[96579]: pgmap v418: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 403 op/s 2023-07-21T22:29:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:39 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/3416083677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a4104a60 msgr2=0x7fc2a4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/3416083677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a4104a60 0x7fc2a4104e30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc294009990 tx=0x7fc294009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/3416083677 shutdown_connections 2023-07-21T22:29:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/3416083677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/3416083677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a4104a60 0x7fc2a4104e30 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/3416083677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a40fea50 0x7fc2a40feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/3416083677 >> 172.21.15.93:0/3416083677 conn(0x7fc2a40fa570 msgr2=0x7fc2a40fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/3416083677 shutdown_connections 2023-07-21T22:29:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.979+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/3416083677 wait complete. 2023-07-21T22:29:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 Processor -- start 2023-07-21T22:29:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 -- start start 2023-07-21T22:29:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a4104a60 0x7fc2a419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a419cf70 con 0x7fc2a40fea50 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a419d0e0 con 0x7fc2a40ff400 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a9d2b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a419d2a0 con 0x7fc2a4104a60 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59362/0 (socket says 172.21.15.93:59362) 2023-07-21T22:29:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48198/0 (socket says 172.21.15.93:48198) 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.980+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/676658501 learned_addr learned my addr 172.21.15.93:0/676658501 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a4104a60 0x7fc2a419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/676658501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a4104a60 msgr2=0x7fc2a419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a4104a60 0x7fc2a419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 msgr2=0x7fc2a4197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/676658501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2940095f0 con 0x7fc2a40ff400 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9a0 tx=0x7fc29800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a0ff9700 1 -- 172.21.15.93:0/676658501 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298009790 con 0x7fc2a40ff400 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a419d580 con 0x7fc2a40ff400 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41a4dc0 con 0x7fc2a40ff400 2023-07-21T22:29:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a0ff9700 1 -- 172.21.15.93:0/676658501 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc298010430 con 0x7fc2a40ff400 2023-07-21T22:29:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.981+0000 7fc2a0ff9700 1 -- 172.21.15.93:0/676658501 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298018680 con 0x7fc2a40ff400 2023-07-21T22:29:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.983+0000 7fc2a0ff9700 1 -- 172.21.15.93:0/676658501 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc29800ca10 con 0x7fc2a40ff400 2023-07-21T22:29:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.984+0000 7fc2a0ff9700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 0x7fc28c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.984+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 0x7fc28c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.984+0000 7fc2a0ff9700 1 -- 172.21.15.93:0/676658501 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc298014070 con 0x7fc2a40ff400 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.985+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 0x7fc28c0a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc29400b2e0 tx=0x7fc294005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 Processor -- start 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 -- start start 2023-07-21T22:29:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b087311cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b08730c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087211c50 con 0x55b087312210 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0872649e0 con 0x55b0871686b0 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08725de70 con 0x55b087312660 2023-07-21T22:29:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48208/0 (socket says 172.21.15.93:48208) 2023-07-21T22:29:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3542138279 learned_addr learned my addr 172.21.15.93:0/3542138279 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b08730c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b087311cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3542138279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 msgr2=0x55b08730c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b08730c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3542138279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 msgr2=0x55b087311cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b087311cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3542138279 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08708bd80 con 0x55b087312210 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.986+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc298096910 tx=0x7fc298099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dc76307837b1d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc289ffb700 1 -- 172.21.15.93:0/3542138279 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29800ccc0 con 0x55b087312210 2023-07-21T22:29:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc289ffb700 1 -- 172.21.15.93:0/3542138279 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29800ce20 con 0x55b087312210 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc289ffb700 1 -- 172.21.15.93:0/3542138279 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980a20c0 con 0x55b087312210 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3542138279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 msgr2=0x55b08730bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc298096910 tx=0x7fc298099910 comp rx=0 tx=0).stop 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3542138279 shutdown_connections 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b08730c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b087311cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3542138279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b08730bf40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3542138279 >> 172.21.15.93:0/3542138279 conn(0x55b087301ae0 msgr2=0x55b0873070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.987+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3542138279 shutdown_connections 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.988+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3542138279 wait complete. 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.988+0000 7fc2b5eb0b80 1 -- start start 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.988+0000 7fc2b5eb0b80 1 Processor -- start 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 0x55b087318630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b087315840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087315f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0871fda60 con 0x55b0871686b0 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0871fd760 con 0x55b087312210 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0870e18c0 con 0x55b087312660 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2b5eb0b80 10 client.0 _ll_register_callbacks cb 0x55b0871fb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 0x55b087318630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 0x55b087318630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48212/0 (socket says 172.21.15.93:48212) 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a37fe700 1 -- 172.21.15.93:0/1113068535 learned_addr learned my addr 172.21.15.93:0/1113068535 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087315f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b087315840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 -- 172.21.15.93:0/1113068535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 msgr2=0x55b087315840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b087315840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 -- 172.21.15.93:0/1113068535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 msgr2=0x55b087318630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 0x55b087318630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 -- 172.21.15.93:0/1113068535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41a4dc0 con 0x55b087312660 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b087315840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087315f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b087312a70 tx=0x7fc29c014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55e7486c2eebd394 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.989+0000 7fc282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00e040 con 0x55b087312660 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc29c014e70 con 0x55b087312660 2023-07-21T22:29:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0871b7490 con 0x55b087312660 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29c003a00 con 0x55b087312660 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 10 client.26509 fetch_fsmap learned FSMap version 66 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x55b087074c00 con 0x55b087312660 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29c02a080 con 0x55b087312660 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fc29c024030 con 0x55b087312660 2023-07-21T22:29:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 10 client.26509 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 10 client.26509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 20 client.26509 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b087325d10 0x55b087328120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 4 client.26509 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b087256e50 con 0x55b087325d10 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.990+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b087325d10 0x55b087328120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.991+0000 7fc282ffd700 10 client.26509 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.991+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b087325d10 0x55b087328120 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb6a31720408d7a server_cookie=cfb51138322a07bb in_seq=0 out_seq=0 2023-07-21T22:29:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc282ffd700 1 -- 172.21.15.93:0/1113068535 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b087256e50 con 0x55b087325d10 2023-07-21T22:29:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc282ffd700 10 client.26509 handle_command_reply: tid=0 2023-07-21T22:29:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 client.26509 shutdown 2023-07-21T22:29:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 msgr2=0x55b087315f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087315f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55b087312a70 tx=0x7fc29c014600 comp rx=0 tx=0).stop 2023-07-21T22:29:40.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 shutdown_connections 2023-07-21T22:29:40.051 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b087325d10 0x55b087328120 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087315f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b0871686b0 0x55b087318630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc282ffd700 0 client.26509 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/1113068535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b087315840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 >> 172.21.15.93:0/1113068535 conn(0x55b087301ae0 msgr2=0x55b0873063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 shutdown_connections 2023-07-21T22:29:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/1113068535 wait complete. 2023-07-21T22:29:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:39.993+0000 7fc2b5eb0b80 20 client.26509 trim_cache size 0 max 16384 2023-07-21T22:29:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 Processor -- start 2023-07-21T22:29:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 -- start start 2023-07-21T22:29:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b08731cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087314db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087308a20 con 0x55b087312210 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b087307cb0 con 0x55b0871686b0 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0873118a0 con 0x55b087312660 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48222/0 (socket says 172.21.15.93:48222) 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2a37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b08731cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3607233051 learned_addr learned my addr 172.21.15.93:0/3607233051 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.010+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087314db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3607233051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 msgr2=0x55b087314db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087314db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3607233051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 msgr2=0x55b08731cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b08731cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 -- 172.21.15.93:0/3607233051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0870b21f0 con 0x55b087312210 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b08731cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc2980cab20 tx=0x7fc29800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff147aab36b9ad9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc28a7fc700 1 -- 172.21.15.93:0/3607233051 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980961e0 con 0x55b087312210 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc28a7fc700 1 -- 172.21.15.93:0/3607233051 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc298099390 con 0x55b087312210 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.011+0000 7fc28a7fc700 1 -- 172.21.15.93:0/3607233051 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298094ab0 con 0x55b087312210 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3607233051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 msgr2=0x55b0873131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc2980cab20 tx=0x7fc29800cf90 comp rx=0 tx=0).stop 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3607233051 shutdown_connections 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b087312660 0x55b087314db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0871686b0 0x55b08731cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3607233051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312210 0x55b0873131b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3607233051 >> 172.21.15.93:0/3607233051 conn(0x55b087301ae0 msgr2=0x55b087312aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3607233051 shutdown_connections 2023-07-21T22:29:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3607233051 wait complete. 2023-07-21T22:29:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 -- start start 2023-07-21T22:29:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.012+0000 7fc2b5eb0b80 1 Processor -- start 2023-07-21T22:29:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0871686b0 0x55b08730ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b08730ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08730dc70 con 0x55b087312660 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08730dde0 con 0x55b087312210 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08730dfa0 con 0x55b0871686b0 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 10 client.0 _ll_register_callbacks cb 0x7fc29c011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48230/0 (socket says 172.21.15.93:48230) 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 -- 172.21.15.93:0/3267283685 learned_addr learned my addr 172.21.15.93:0/3267283685 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0871686b0 0x55b08730ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b08730ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 -- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0871686b0 msgr2=0x55b08730ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0871686b0 0x55b08730ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 -- 172.21.15.93:0/3267283685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 msgr2=0x55b08730ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b08730ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 -- 172.21.15.93:0/3267283685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a410c190 con 0x55b087312660 2023-07-21T22:29:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a2ffd700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b08730ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2a3fff700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc29c011b20 tx=0x7fc29c004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1257e0e045d813 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c00e070 con 0x55b087312660 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc29c01eac0 con 0x55b087312660 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc29800ce20 con 0x55b087312660 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc29c006070 con 0x55b087312660 2023-07-21T22:29:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 10 client.17205 fetch_fsmap learned FSMap version 66 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.013+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x55b08708bd80 con 0x55b087312660 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29800ce20 con 0x55b087312660 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fc29c02f030 con 0x55b087312660 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 20 client.17205 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b08737aa90 0x55b08737cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 4 client.17205 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b087218ff0 con 0x55b08737aa90 2023-07-21T22:29:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b08737aa90 0x55b08737cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc282ffd700 10 client.17205 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.014+0000 7fc2a37fe700 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b08737aa90 0x55b08737cea0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca110d4522bb84b4 server_cookie=879d72f9c55b3f45 in_seq=0 out_seq=0 2023-07-21T22:29:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc282ffd700 1 -- 172.21.15.93:0/3267283685 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b087218ff0 con 0x55b08737aa90 2023-07-21T22:29:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc282ffd700 10 client.17205 handle_command_reply: tid=0 2023-07-21T22:29:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 client.17205 shutdown 2023-07-21T22:29:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 msgr2=0x55b08730d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc29c011b20 tx=0x7fc29c004cb0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 shutdown_connections 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b08737aa90 0x55b08737cea0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b087312660 0x55b08730d730 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc282ffd700 0 client.17205 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0871686b0 0x55b08730ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 --2- 172.21.15.93:0/3267283685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b087312210 0x55b08730ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 >> 172.21.15.93:0/3267283685 conn(0x55b087301ae0 msgr2=0x55b087311120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 shutdown_connections 2023-07-21T22:29:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.369+0000 7fc2b5eb0b80 1 -- 172.21.15.93:0/3267283685 wait complete. 2023-07-21T22:29:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2b5eb0b80 20 client.17205 trim_cache size 0 max 16384 2023-07-21T22:29:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:29:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:29:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:29:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:29:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 msgr2=0x7fc28c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 0x7fc28c0a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc29400b2e0 tx=0x7fc294005fd0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 msgr2=0x7fc2a41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9a0 tx=0x7fc29800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 shutdown_connections 2023-07-21T22:29:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2a4104a60 0x7fc2a419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc28c0a1340 0x7fc28c0a37f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2a40fea50 0x7fc2a4197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 --2- 172.21.15.93:0/676658501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2a40ff400 0x7fc2a41982c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 >> 172.21.15.93:0/676658501 conn(0x7fc2a40fa570 msgr2=0x7fc2a40fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 shutdown_connections 2023-07-21T22:29:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:40.370+0000 7fc2a9d2b700 1 -- 172.21.15.93:0/676658501 wait complete. 2023-07-21T22:29:40.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 6410 32.36 MB/sec warmup 16 sec latency 212.376 ms 2023-07-21T22:29:41.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 6847 32.92 MB/sec warmup 17 sec latency 106.640 ms 2023-07-21T22:29:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:41 smithi194 ceph-mon[98288]: pgmap v419: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2023-07-21T22:29:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:41 smithi134 ceph-mon[97873]: pgmap v419: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2023-07-21T22:29:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:41 smithi093 ceph-mon[96579]: pgmap v419: 97 pgs: 97 active+clean; 382 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2023-07-21T22:29:42.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 6964 31.60 MB/sec warmup 18 sec latency 438.814 ms 2023-07-21T22:29:43.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 6971 29.93 MB/sec warmup 19 sec latency 1173.075 ms 2023-07-21T22:29:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:43 smithi194 ceph-mon[98288]: pgmap v420: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 368 op/s 2023-07-21T22:29:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:43 smithi134 ceph-mon[97873]: pgmap v420: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 368 op/s 2023-07-21T22:29:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:43 smithi093 ceph-mon[96579]: pgmap v420: 97 pgs: 97 active+clean; 474 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 368 op/s 2023-07-21T22:29:44.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 7067 28.50 MB/sec warmup 20 sec latency 252.104 ms 2023-07-21T22:29:44.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:44 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:44.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:44 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:44.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:44 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:45.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 7195 27.27 MB/sec warmup 21 sec latency 83.677 ms 2023-07-21T22:29:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: pgmap v421: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:45.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:45.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: pgmap v421: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:45.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:45.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: pgmap v421: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:45.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:29:46.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 7344 26.04 MB/sec warmup 22 sec latency 106.329 ms 2023-07-21T22:29:47.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 7629 25.23 MB/sec warmup 23 sec latency 81.527 ms 2023-07-21T22:29:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:47 smithi093 ceph-mon[96579]: pgmap v422: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:47.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:47 smithi194 ceph-mon[98288]: pgmap v422: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:47 smithi134 ceph-mon[97873]: pgmap v422: 97 pgs: 97 active+clean; 581 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 20 MiB/s wr, 485 op/s 2023-07-21T22:29:48.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 7974 24.86 MB/sec warmup 24 sec latency 98.848 ms 2023-07-21T22:29:49.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 8679 25.22 MB/sec warmup 25 sec latency 150.407 ms 2023-07-21T22:29:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:49 smithi093 ceph-mon[96579]: pgmap v423: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 695 op/s 2023-07-21T22:29:49.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:49 smithi194 ceph-mon[98288]: pgmap v423: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 695 op/s 2023-07-21T22:29:49.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:49 smithi134 ceph-mon[97873]: pgmap v423: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 25 MiB/s wr, 695 op/s 2023-07-21T22:29:50.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 9868 27.15 MB/sec warmup 26 sec latency 75.281 ms 2023-07-21T22:29:51.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 10590 28.01 MB/sec warmup 27 sec latency 31.909 ms 2023-07-21T22:29:51.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:51 smithi093 ceph-mon[96579]: pgmap v424: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 596 op/s 2023-07-21T22:29:51.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:51 smithi194 ceph-mon[98288]: pgmap v424: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 596 op/s 2023-07-21T22:29:51.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:51 smithi134 ceph-mon[97873]: pgmap v424: 97 pgs: 97 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 23 MiB/s wr, 596 op/s 2023-07-21T22:29:52.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 11230 27.80 MB/sec warmup 28 sec latency 38.446 ms 2023-07-21T22:29:53.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 12296 28.87 MB/sec warmup 29 sec latency 43.326 ms 2023-07-21T22:29:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 -- 172.21.15.93:0/2687222222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0108620 msgr2=0x7f1cd01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/2687222222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0108620 0x7f1cd01089f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009990 tx=0x7f1ccc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 -- 172.21.15.93:0/2687222222 shutdown_connections 2023-07-21T22:29:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/2687222222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0111bd0 0x7f1cd0113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/2687222222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0108620 0x7f1cd01089f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/2687222222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd0111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.523+0000 7f1cd7a14700 1 -- 172.21.15.93:0/2687222222 >> 172.21.15.93:0/2687222222 conn(0x7f1cd006c270 msgr2=0x7f1cd006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.524+0000 7f1cd7a14700 1 -- 172.21.15.93:0/2687222222 shutdown_connections 2023-07-21T22:29:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.524+0000 7f1cd7a14700 1 -- 172.21.15.93:0/2687222222 wait complete. 2023-07-21T22:29:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.524+0000 7f1cd7a14700 1 Processor -- start 2023-07-21T22:29:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.524+0000 7f1cd7a14700 1 -- start start 2023-07-21T22:29:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0108620 0x7f1cd01a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd01a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 0x7f1cd01a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a0af0 con 0x7f1cd0111bd0 2023-07-21T22:29:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a0c60 con 0x7f1cd0108620 2023-07-21T22:29:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd7a14700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01a0e20 con 0x7f1cd0108f30 2023-07-21T22:29:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd5fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 0x7f1cd01a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd5fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 0x7f1cd01a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48242/0 (socket says 172.21.15.93:48242) 2023-07-21T22:29:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd5fb1700 1 -- 172.21.15.93:0/3381698019 learned_addr learned my addr 172.21.15.93:0/3381698019 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd01a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.525+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0108620 0x7f1cd01a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3381698019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0108620 msgr2=0x7f1cd01a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0108620 0x7f1cd01a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 msgr2=0x7f1cd01a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 0x7f1cd01a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3381698019 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0095f0 con 0x7f1cd0108f30 2023-07-21T22:29:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd01a6b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8007ec0 tx=0x7f1cc800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cc67fc700 1 -- 172.21.15.93:0/3381698019 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8004db0 con 0x7f1cd0108f30 2023-07-21T22:29:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cc67fc700 1 -- 172.21.15.93:0/3381698019 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cc800f040 con 0x7f1cd0108f30 2023-07-21T22:29:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.526+0000 7f1cc67fc700 1 -- 172.21.15.93:0/3381698019 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8005910 con 0x7f1cd0108f30 2023-07-21T22:29:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.527+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd01a1100 con 0x7f1cd0108f30 2023-07-21T22:29:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.527+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd01ad5b0 con 0x7f1cd0108f30 2023-07-21T22:29:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.530+0000 7f1cc67fc700 1 -- 172.21.15.93:0/3381698019 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1cc800db20 con 0x7f1cd0108f30 2023-07-21T22:29:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.530+0000 7f1cc67fc700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 0x7f1cbc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.531+0000 7f1cc67fc700 1 -- 172.21.15.93:0/3381698019 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1cc80cc580 con 0x7f1cd0108f30 2023-07-21T22:29:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.531+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 0x7f1cbc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.531+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 0x7f1cbc0a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00b5c0 tx=0x7f1ccc005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 Processor -- start 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 -- start start 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 0x564a6b90ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 0x564a6b906f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d660 0x564a6b9074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b80cc50 con 0x564a6b7636b0 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b85f9e0 con 0x564a6b90d210 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b858e70 con 0x564a6b90d660 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.532+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 0x564a6b90ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 0x564a6b90ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48248/0 (socket says 172.21.15.93:48248) 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd57b0700 1 -- 172.21.15.93:0/3060216158 learned_addr learned my addr 172.21.15.93:0/3060216158 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 0x564a6b906f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd5fb1700 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d660 0x564a6b9074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3060216158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d660 msgr2=0x564a6b9074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d660 0x564a6b9074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3060216158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 msgr2=0x564a6b90ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 0x564a6b90ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 -- 172.21.15.93:0/3060216158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a6b686d80 con 0x564a6b90d210 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 0x564a6b906f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80c9cd0 tx=0x7f1cc80994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53228a4a34c174e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cbb7fe700 1 -- 172.21.15.93:0/3060216158 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc809d680 con 0x564a6b90d210 2023-07-21T22:29:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cbb7fe700 1 -- 172.21.15.93:0/3060216158 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cc80a1aa0 con 0x564a6b90d210 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.533+0000 7f1cbb7fe700 1 -- 172.21.15.93:0/3060216158 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8095d60 con 0x564a6b90d210 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/3060216158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 msgr2=0x564a6b906f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 0x564a6b906f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80c9cd0 tx=0x7f1cc80994f0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/3060216158 shutdown_connections 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d660 0x564a6b9074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b7636b0 0x564a6b90ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/3060216158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b90d210 0x564a6b906f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/3060216158 >> 172.21.15.93:0/3060216158 conn(0x564a6b8fcae0 msgr2=0x564a6b9020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/3060216158 shutdown_connections 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.534+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/3060216158 wait complete. 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 -- start start 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 Processor -- start 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d210 0x564a6b910830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b90d660 0x564a6b910f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b7f8a60 con 0x564a6b90d660 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b7f8760 con 0x564a6b7636b0 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.535+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b6dc8c0 con 0x564a6b90d210 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1ce3b99b80 10 client.0 _ll_register_callbacks cb 0x564a6b7f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59416/0 (socket says 172.21.15.93:59416) 2023-07-21T22:29:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd4faf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d210 0x564a6b910830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 -- 172.21.15.93:0/2211082280 learned_addr learned my addr 172.21.15.93:0/2211082280 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd5fb1700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b90d660 0x564a6b910f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 -- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d210 msgr2=0x564a6b910830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d210 0x564a6b910830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 -- 172.21.15.93:0/2211082280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b90d660 msgr2=0x564a6b910f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b90d660 0x564a6b910f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 -- 172.21.15.93:0/2211082280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd01ad5b0 con 0x564a6b7636b0 2023-07-21T22:29:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc003fc0 tx=0x7f1ccc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e7b00ffc14dee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc02a050 con 0x564a6b7636b0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a6b7b2490 con 0x564a6b7636b0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.536+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1ccc005690 con 0x564a6b7636b0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ccc023660 con 0x564a6b7636b0 2023-07-21T22:29:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 10 client.26600 fetch_fsmap learned FSMap version 66 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x564a6b66fc00 con 0x564a6b7636b0 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a6b7b2490 con 0x564a6b7636b0 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f1ccc00f950 con 0x564a6b7636b0 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 20 client.26600 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b920c10 0x564a6b923020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 4 client.26600 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a6b851e50 con 0x564a6b920c10 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.537+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b920c10 0x564a6b923020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.538+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b920c10 0x564a6b923020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac032b77584e969 server_cookie=770c24ec20693293 in_seq=0 out_seq=0 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.538+0000 7f1cb8ff9700 10 client.26600 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/2211082280 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a6b851e50 con 0x564a6b920c10 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1cb8ff9700 10 client.26600 handle_command_reply: tid=0 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 client.26600 shutdown 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 msgr2=0x564a6b9135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc003fc0 tx=0x7f1ccc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 shutdown_connections 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b920c10 0x564a6b923020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b90d660 0x564a6b910f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b7636b0 0x564a6b9135b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/2211082280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90d210 0x564a6b910830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1cb8ff9700 0 client.26600 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 >> 172.21.15.93:0/2211082280 conn(0x564a6b8fcae0 msgr2=0x564a6b9013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 shutdown_connections 2023-07-21T22:29:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.539+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/2211082280 wait complete. 2023-07-21T22:29:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.540+0000 7f1ce3b99b80 20 client.26600 trim_cache size 0 max 16384 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 Processor -- start 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 -- start start 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b913150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 0x564a6b90c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b91d550 con 0x564a6b91d180 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b9117b0 con 0x564a6b8fcae0 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b907f20 con 0x564a6b90b060 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd4faf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b913150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd5fb1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 0x564a6b90c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd4faf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48266/0 (socket says 172.21.15.93:48266) 2023-07-21T22:29:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd5fb1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 0x564a6b90c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59426/0 (socket says 172.21.15.93:59426) 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd4faf700 1 -- 172.21.15.93:0/145852861 learned_addr learned my addr 172.21.15.93:0/145852861 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.560+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b913150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35342/0 (socket says 172.21.15.93:35342) 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 -- 172.21.15.93:0/145852861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 msgr2=0x564a6b913150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b913150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 -- 172.21.15.93:0/145852861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 msgr2=0x564a6b90c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 0x564a6b90c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 -- 172.21.15.93:0/145852861 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a6b6ad1f0 con 0x564a6b91d180 2023-07-21T22:29:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005360 tx=0x7f1cc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141e595ad87fbb98 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cbbfff700 1 -- 172.21.15.93:0/145852861 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8005a70 con 0x564a6b91d180 2023-07-21T22:29:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cbbfff700 1 -- 172.21.15.93:0/145852861 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cc809d680 con 0x564a6b91d180 2023-07-21T22:29:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.561+0000 7f1cbbfff700 1 -- 172.21.15.93:0/145852861 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b77e0 con 0x564a6b91d180 2023-07-21T22:29:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/145852861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 msgr2=0x564a6b90bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005360 tx=0x7f1cc800b040 comp rx=0 tx=0).stop 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/145852861 shutdown_connections 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b8fcae0 0x564a6b90c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b913150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/145852861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b91d180 0x564a6b90bf90 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/145852861 >> 172.21.15.93:0/145852861 conn(0x564a6b90ac90 msgr2=0x564a6b90b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/145852861 shutdown_connections 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.562+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/145852861 wait complete. 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 -- start start 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 Processor -- start 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b917d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b91d180 0x564a6b914220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b914930 con 0x564a6b8fcae0 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b914aa0 con 0x564a6b91d180 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a6b914c60 con 0x564a6b90b060 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1ce3b99b80 10 client.0 _ll_register_callbacks cb 0x564a6b7f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd4faf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b917d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd4faf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b917d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35348/0 (socket says 172.21.15.93:35348) 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd57b0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48268/0 (socket says 172.21.15.93:48268) 2023-07-21T22:29:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd4faf700 1 -- 172.21.15.93:0/1063735558 learned_addr learned my addr 172.21.15.93:0/1063735558 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:29:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.563+0000 7f1cd5fb1700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b91d180 0x564a6b914220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 -- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 msgr2=0x564a6b917d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b917d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 -- 172.21.15.93:0/1063735558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b91d180 msgr2=0x564a6b914220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b91d180 0x564a6b914220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 -- 172.21.15.93:0/1063735558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a6b686d80 con 0x564a6b8fcae0 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cd57b0700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00fd30 tx=0x7f1ccc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab2c99934d82fe9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc02a040 con 0x564a6b8fcae0 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1ccc02f400 con 0x564a6b8fcae0 2023-07-21T22:29:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cc80a1aa0 con 0x564a6b8fcae0 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ccc00b5f0 con 0x564a6b8fcae0 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1ce3b99b80 10 client.17223 fetch_fsmap learned FSMap version 66 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.564+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x564a6b685250 con 0x564a6b8fcae0 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ccc050020 con 0x564a6b8fcae0 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f1ccc023700 con 0x564a6b8fcae0 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:29:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 20 client.17223 populate_metadata read hostname 'smithi093' 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b7636b0 0x564a6b988b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 4 client.17223 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a6b813ff0 con 0x564a6b7636b0 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.565+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b7636b0 0x564a6b988b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.567+0000 7f1cd4faf700 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b7636b0 0x564a6b988b20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0724d215e85e6e6 server_cookie=c28cfbf6e1b350cd in_seq=0 out_seq=0 2023-07-21T22:29:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.567+0000 7f1cb8ff9700 10 client.17223 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:53.698 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:53 smithi093 ceph-mon[96579]: pgmap v425: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 MiB/s wr, 711 op/s 2023-07-21T22:29:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1cb8ff9700 1 -- 172.21.15.93:0/1063735558 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a6b813ff0 con 0x564a6b7636b0 2023-07-21T22:29:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1cb8ff9700 10 client.17223 handle_command_reply: tid=0 2023-07-21T22:29:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 client.17223 shutdown 2023-07-21T22:29:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 msgr2=0x564a6b9177d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00fd30 tx=0x7f1ccc00f710 comp rx=0 tx=0).stop 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 shutdown_connections 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564a6b7636b0 0x564a6b988b20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a6b91d180 0x564a6b914220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1cb8ff9700 0 client.17223 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:29:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564a6b8fcae0 0x564a6b9177d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 --2- 172.21.15.93:0/1063735558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564a6b90b060 0x564a6b917d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.730+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 >> 172.21.15.93:0/1063735558 conn(0x564a6b90ac90 msgr2=0x564a6b90c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 shutdown_connections 2023-07-21T22:29:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1ce3b99b80 1 -- 172.21.15.93:0/1063735558 wait complete. 2023-07-21T22:29:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1ce3b99b80 20 client.17223 trim_cache size 0 max 16384 2023-07-21T22:29:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:29:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:29:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:29:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:29:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 msgr2=0x7f1cbc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 0x7f1cbc0a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00b5c0 tx=0x7f1ccc005fd0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 msgr2=0x7f1cd01a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd01a6b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8007ec0 tx=0x7f1cc800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 shutdown_connections 2023-07-21T22:29:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1cbc0a1340 0x7f1cbc0a37f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cd0111bd0 0x7f1cd01a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd0108620 0x7f1cd01a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 --2- 172.21.15.93:0/3381698019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd0108f30 0x7f1cd01a6b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.731+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 >> 172.21.15.93:0/3381698019 conn(0x7f1cd006c270 msgr2=0x7f1cd0107510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.732+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 shutdown_connections 2023-07-21T22:29:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:29:53.732+0000 7f1cd7a14700 1 -- 172.21.15.93:0/3381698019 wait complete. 2023-07-21T22:29:53.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:53 smithi194 ceph-mon[98288]: pgmap v425: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 MiB/s wr, 711 op/s 2023-07-21T22:29:53.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:53 smithi134 ceph-mon[97873]: pgmap v425: 97 pgs: 97 active+clean; 583 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 MiB/s wr, 711 op/s 2023-07-21T22:29:54.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 13319 29.79 MB/sec warmup 30 sec latency 100.119 ms 2023-07-21T22:29:55.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 14067 30.38 MB/sec warmup 31 sec latency 135.770 ms 2023-07-21T22:29:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:55 smithi093 ceph-mon[96579]: pgmap v426: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 650 op/s 2023-07-21T22:29:55.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:55 smithi194 ceph-mon[98288]: pgmap v426: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 650 op/s 2023-07-21T22:29:55.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:55 smithi134 ceph-mon[97873]: pgmap v426: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 17 MiB/s wr, 650 op/s 2023-07-21T22:29:56.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 14936 30.37 MB/sec warmup 32 sec latency 36.198 ms 2023-07-21T22:29:57.485 INFO:tasks.workunit.client.0.smithi093.stdout: 10 15878 31.24 MB/sec warmup 33 sec latency 66.473 ms 2023-07-21T22:29:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:57 smithi093 ceph-mon[96579]: pgmap v427: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 442 op/s 2023-07-21T22:29:57.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:57 smithi194 ceph-mon[98288]: pgmap v427: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 442 op/s 2023-07-21T22:29:57.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:57 smithi134 ceph-mon[97873]: pgmap v427: 97 pgs: 97 active+clean; 588 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 442 op/s 2023-07-21T22:29:58.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 16905 31.80 MB/sec warmup 34 sec latency 36.525 ms 2023-07-21T22:29:59.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 17833 32.63 MB/sec warmup 35 sec latency 39.814 ms 2023-07-21T22:29:59.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:29:59 smithi093 ceph-mon[96579]: pgmap v428: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 789 op/s 2023-07-21T22:29:59.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:29:59 smithi194 ceph-mon[98288]: pgmap v428: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 789 op/s 2023-07-21T22:29:59.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:29:59 smithi134 ceph-mon[97873]: pgmap v428: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 789 op/s 2023-07-21T22:30:00.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:29:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:30:00.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 18614 32.74 MB/sec warmup 36 sec latency 71.699 ms 2023-07-21T22:30:00.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:00 smithi093 ceph-mon[96579]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:30:00.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:00 smithi194 ceph-mon[98288]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:30:00.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:00 smithi134 ceph-mon[97873]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:30:01.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 18619 31.86 MB/sec warmup 37 sec latency 1013.986 ms 2023-07-21T22:30:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:01 smithi093 ceph-mon[96579]: pgmap v429: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 580 op/s 2023-07-21T22:30:01.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:01 smithi194 ceph-mon[98288]: pgmap v429: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 580 op/s 2023-07-21T22:30:01.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:01 smithi134 ceph-mon[97873]: pgmap v429: 97 pgs: 97 active+clean; 602 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 580 op/s 2023-07-21T22:30:02.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 18619 31.02 MB/sec warmup 38 sec latency 2014.442 ms 2023-07-21T22:30:03.487 INFO:tasks.workunit.client.0.smithi093.stdout: 10 18878 30.48 MB/sec warmup 39 sec latency 2497.300 ms 2023-07-21T22:30:03.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:03 smithi194 ceph-mon[98288]: pgmap v430: 97 pgs: 97 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 706 op/s 2023-07-21T22:30:03.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:03 smithi134 ceph-mon[97873]: pgmap v430: 97 pgs: 97 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 706 op/s 2023-07-21T22:30:03.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:03 smithi093 ceph-mon[96579]: pgmap v430: 97 pgs: 97 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 706 op/s 2023-07-21T22:30:04.491 INFO:tasks.workunit.client.0.smithi093.stdout: 10 19592 30.70 MB/sec warmup 40 sec latency 80.053 ms 2023-07-21T22:30:05.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 20312 30.64 MB/sec warmup 41 sec latency 85.020 ms 2023-07-21T22:30:05.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:05 smithi194 ceph-mon[98288]: pgmap v431: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2023-07-21T22:30:05.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:05 smithi194 ceph-mon[98288]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:30:05.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:05 smithi134 ceph-mon[97873]: pgmap v431: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2023-07-21T22:30:05.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:05 smithi134 ceph-mon[97873]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:30:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:05 smithi093 ceph-mon[96579]: pgmap v431: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.9 MiB/s wr, 661 op/s 2023-07-21T22:30:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:05 smithi093 ceph-mon[96579]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:30:06.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 21327 31.50 MB/sec warmup 42 sec latency 66.748 ms 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 -- 172.21.15.93:0/1362673943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 msgr2=0x7f2408104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 --2- 172.21.15.93:0/1362673943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408104e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098f0 tx=0x7f23f8009c00 comp rx=0 tx=0).stop 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 -- 172.21.15.93:0/1362673943 shutdown_connections 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 --2- 172.21.15.93:0/1362673943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080ff3e0 0x7f240810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 --2- 172.21.15.93:0/1362673943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408104e00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 --2- 172.21.15.93:0/1362673943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080fea30 0x7f24080feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 -- 172.21.15.93:0/1362673943 >> 172.21.15.93:0/1362673943 conn(0x7f2408074760 msgr2=0x7f2408074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 -- 172.21.15.93:0/1362673943 shutdown_connections 2023-07-21T22:30:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.885+0000 7f240d7b2700 1 -- 172.21.15.93:0/1362673943 wait complete. 2023-07-21T22:30:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 Processor -- start 2023-07-21T22:30:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 -- start start 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080fea30 0x7f240819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408196130 con 0x7f24080ff3e0 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.886+0000 7f240d7b2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24081962a0 con 0x7f2408104a30 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f240d7b2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408196460 con 0x7f24080fea30 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24067fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24067fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48282/0 (socket says 172.21.15.93:48282) 2023-07-21T22:30:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24067fc700 1 -- 172.21.15.93:0/2033330820 learned_addr learned my addr 172.21.15.93:0/2033330820 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080fea30 0x7f240819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 -- 172.21.15.93:0/2033330820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080fea30 msgr2=0x7f240819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080fea30 0x7f240819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 -- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 msgr2=0x7f240819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 -- 172.21.15.93:0/2033330820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc0095f0 con 0x7f2408104a30 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24067fc700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.887+0000 7f24077fe700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408195ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2400007f10 tx=0x7f240000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.888+0000 7f23effff700 1 -- 172.21.15.93:0/2033330820 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400009ea0 con 0x7f2408104a30 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.888+0000 7f23effff700 1 -- 172.21.15.93:0/2033330820 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240000f040 con 0x7f2408104a30 2023-07-21T22:30:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.888+0000 7f23effff700 1 -- 172.21.15.93:0/2033330820 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400004350 con 0x7f2408104a30 2023-07-21T22:30:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.889+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f8009520 con 0x7f2408104a30 2023-07-21T22:30:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.889+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2408100560 con 0x7f2408104a30 2023-07-21T22:30:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.892+0000 7f23effff700 1 -- 172.21.15.93:0/2033330820 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f240000db20 con 0x7f2408104a30 2023-07-21T22:30:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.892+0000 7f23effff700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 0x7f23f00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f23effff700 1 -- 172.21.15.93:0/2033330820 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f24000cc520 con 0x7f2408104a30 2023-07-21T22:30:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 0x7f23f00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f2419937b80 1 Processor -- start 2023-07-21T22:30:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f2419937b80 1 -- start start 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 0x7f23f00a38c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009fd0 tx=0x7f23f8019040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.893+0000 7f2419937b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2419937b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2419937b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2419937b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcefc5c50 con 0x561bcef1c6b0 2023-07-21T22:30:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2419937b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0189e0 con 0x561bcf0cee70 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2419937b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf011e70 con 0x561bcf0cea20 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48290/0 (socket says 172.21.15.93:48290) 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 -- 172.21.15.93:0/2908409079 learned_addr learned my addr 172.21.15.93:0/2908409079 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f24077fe700 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f24067fc700 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 -- 172.21.15.93:0/2908409079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 msgr2=0x561bcf0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 -- 172.21.15.93:0/2908409079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 msgr2=0x561bcf0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 -- 172.21.15.93:0/2908409079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcee3fd80 con 0x561bcef1c6b0 2023-07-21T22:30:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.894+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f23f8016000 tx=0x7f23f801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800a56f8cd809c36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.895+0000 7f23ed7fa700 1 -- 172.21.15.93:0/2908409079 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800b760 con 0x561bcef1c6b0 2023-07-21T22:30:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.895+0000 7f23ed7fa700 1 -- 172.21.15.93:0/2908409079 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f801d040 con 0x561bcef1c6b0 2023-07-21T22:30:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.895+0000 7f23ed7fa700 1 -- 172.21.15.93:0/2908409079 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80038a0 con 0x561bcef1c6b0 2023-07-21T22:30:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 -- 172.21.15.93:0/2908409079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 msgr2=0x561bcf0ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f23f8016000 tx=0x7f23f801e000 comp rx=0 tx=0).stop 2023-07-21T22:30:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 -- 172.21.15.93:0/2908409079 shutdown_connections 2023-07-21T22:30:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0ce4e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 --2- 172.21.15.93:0/2908409079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.896+0000 7f2419937b80 1 -- 172.21.15.93:0/2908409079 >> 172.21.15.93:0/2908409079 conn(0x561bcf0b5ae0 msgr2=0x561bcf0bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.897+0000 7f2419937b80 1 -- 172.21.15.93:0/2908409079 shutdown_connections 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.897+0000 7f2419937b80 1 -- 172.21.15.93:0/2908409079 wait complete. 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.897+0000 7f2419937b80 1 -- start start 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.897+0000 7f2419937b80 1 Processor -- start 2023-07-21T22:30:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 0x561bcf0d8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cee70 0x561bcf0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcefb1a60 con 0x561bcef1c6b0 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcefb1760 con 0x561bcf0cea20 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcee958c0 con 0x561bcf0cee70 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2419937b80 10 client.0 _ll_register_callbacks cb 0x561bcefaf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f24067fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 0x561bcf0d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48296/0 (socket says 172.21.15.93:48296) 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 -- 172.21.15.93:0/408738213 learned_addr learned my addr 172.21.15.93:0/408738213 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 -- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cee70 msgr2=0x561bcf0d2090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cee70 0x561bcf0d2090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 -- 172.21.15.93:0/408738213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 msgr2=0x561bcf0d8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 0x561bcf0d8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f2406ffd700 1 -- 172.21.15.93:0/408738213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240810c160 con 0x561bcef1c6b0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.898+0000 7f24067fc700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 0x561bcf0d8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f2406ffd700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098f0 tx=0x7f23f800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17ab3c9b77c30b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f23eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8005450 con 0x561bcef1c6b0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bcef6b490 con 0x561bcef1c6b0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f23f8003680 con 0x561bcef1c6b0 2023-07-21T22:30:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f8003840 con 0x561bcef1c6b0 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f2419937b80 10 client.17235 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x561bcee28c00 con 0x561bcef1c6b0 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.899+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bcef6b490 con 0x561bcef1c6b0 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f23f802faf0 con 0x561bcef1c6b0 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 20 client.17235 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf0e1f20 0x561bcf0e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 4 client.17235 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bcf00ae50 con 0x561bcf0e1f20 2023-07-21T22:30:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.900+0000 7f24067fc700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf0e1f20 0x561bcf0e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.901+0000 7f24067fc700 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf0e1f20 0x561bcf0e4330 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3633184fd0ab1f server_cookie=6697fb716d9b74ca in_seq=0 out_seq=0 2023-07-21T22:30:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.901+0000 7f23eaff5700 10 client.17235 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.901+0000 7f23eaff5700 1 -- 172.21.15.93:0/408738213 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561bcf00ae50 con 0x561bcf0e1f20 2023-07-21T22:30:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.901+0000 7f23eaff5700 10 client.17235 handle_command_reply: tid=0 2023-07-21T22:30:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.901+0000 7f2419937b80 1 client.17235 shutdown 2023-07-21T22:30:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 msgr2=0x561bcf0d7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098f0 tx=0x7f23f800bc40 comp rx=0 tx=0).stop 2023-07-21T22:30:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 shutdown_connections 2023-07-21T22:30:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf0e1f20 0x561bcf0e4330 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cee70 0x561bcf0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcef1c6b0 0x561bcf0d7e40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 --2- 172.21.15.93:0/408738213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cea20 0x561bcf0d8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 >> 172.21.15.93:0/408738213 conn(0x561bcf0b5ae0 msgr2=0x561bcf0ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f23eaff5700 0 client.17235 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 shutdown_connections 2023-07-21T22:30:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 1 -- 172.21.15.93:0/408738213 wait complete. 2023-07-21T22:30:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.902+0000 7f2419937b80 20 client.17235 trim_cache size 0 max 16384 2023-07-21T22:30:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 Processor -- start 2023-07-21T22:30:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 -- start start 2023-07-21T22:30:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 0x561bcf0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0b7c50 con 0x561bcf0cea20 2023-07-21T22:30:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0bbce0 con 0x561bcf0cee70 2023-07-21T22:30:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.923+0000 7f2419937b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0b7570 con 0x561bcef1c6b0 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f24067fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f24067fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48308/0 (socket says 172.21.15.93:48308) 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f24067fc700 1 -- 172.21.15.93:0/3374199114 learned_addr learned my addr 172.21.15.93:0/3374199114 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 0x561bcf0d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f24077fe700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 -- 172.21.15.93:0/3374199114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 msgr2=0x561bcf0bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 -- 172.21.15.93:0/3374199114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 msgr2=0x561bcf0bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f2406ffd700 1 -- 172.21.15.93:0/3374199114 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcee661f0 con 0x561bcef1c6b0 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.924+0000 7f24067fc700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.925+0000 7f2406ffd700 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 0x561bcf0d2090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098c0 tx=0x7f23f8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884795d88661716a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.925+0000 7f23edffb700 1 -- 172.21.15.93:0/3374199114 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8038490 con 0x561bcef1c6b0 2023-07-21T22:30:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.925+0000 7f23edffb700 1 -- 172.21.15.93:0/3374199114 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f8005710 con 0x561bcef1c6b0 2023-07-21T22:30:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.925+0000 7f23edffb700 1 -- 172.21.15.93:0/3374199114 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80038e0 con 0x561bcef1c6b0 2023-07-21T22:30:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 -- 172.21.15.93:0/3374199114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 msgr2=0x561bcf0d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 0x561bcf0d2090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098c0 tx=0x7f23f8005dc0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 -- 172.21.15.93:0/3374199114 shutdown_connections 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcf0cee70 0x561bcf0bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcef1c6b0 0x561bcf0d2090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 --2- 172.21.15.93:0/3374199114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cea20 0x561bcf0bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 -- 172.21.15.93:0/3374199114 >> 172.21.15.93:0/3374199114 conn(0x561bcf0b5ae0 msgr2=0x561bcf0d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 -- 172.21.15.93:0/3374199114 shutdown_connections 2023-07-21T22:30:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.926+0000 7f2419937b80 1 -- 172.21.15.93:0/3374199114 wait complete. 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 -- start start 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 Processor -- start 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcef1c6b0 0x561bcf12d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf12d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0d7910 con 0x561bcf0cee70 2023-07-21T22:30:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0d7a80 con 0x561bcef1c6b0 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcf0d7c40 con 0x561bcf0cea20 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2419937b80 10 client.0 _ll_register_callbacks cb 0x7f2400003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcef1c6b0 0x561bcf12d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48314/0 (socket says 172.21.15.93:48314) 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 -- 172.21.15.93:0/2899287222 learned_addr learned my addr 172.21.15.93:0/2899287222 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 -- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 msgr2=0x561bcf12d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf12d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 -- 172.21.15.93:0/2899287222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcef1c6b0 msgr2=0x561bcf12d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.927+0000 7f24077fe700 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcef1c6b0 0x561bcf12d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f24077fe700 1 -- 172.21.15.93:0/2899287222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc012da0 con 0x561bcf0cee70 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f24077fe700 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2400099550 tx=0x7f2400099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e8d3d037b1fb82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f23eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:30:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000b9070 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f24000995c0 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23f801d040 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f240009d5f0 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f2419937b80 10 client.17241 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x561bcee28c00 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.928+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24000c7080 con 0x561bcf0cee70 2023-07-21T22:30:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f2400098ab0 con 0x561bcf0cee70 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 10 client.17241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 20 client.17241 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf1361f0 0x561bcf138600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 4 client.17241 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bcefccff0 con 0x561bcf1361f0 2023-07-21T22:30:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.929+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf1361f0 0x561bcf138600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.931+0000 7f23eaff5700 10 client.17241 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:06.931+0000 7f2406ffd700 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf1361f0 0x561bcf138600 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8960f12051fb49 server_cookie=213749be64b8304a in_seq=0 out_seq=0 2023-07-21T22:30:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.094+0000 7f23eaff5700 1 -- 172.21.15.93:0/2899287222 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bcefccff0 con 0x561bcf1361f0 2023-07-21T22:30:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.094+0000 7f23eaff5700 10 client.17241 handle_command_reply: tid=0 2023-07-21T22:30:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.094+0000 7f2419937b80 1 client.17241 shutdown 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 msgr2=0x561bcf0d7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2400099550 tx=0x7f2400099100 comp rx=0 tx=0).stop 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 shutdown_connections 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x561bcf1361f0 0x561bcf138600 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bcf0cee70 0x561bcf0d7200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bcef1c6b0 0x561bcf12d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f23eaff5700 0 client.17241 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 --2- 172.21.15.93:0/2899287222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561bcf0cea20 0x561bcf12d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 >> 172.21.15.93:0/2899287222 conn(0x561bcf0b5ae0 msgr2=0x561bcf0cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 shutdown_connections 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 1 -- 172.21.15.93:0/2899287222 wait complete. 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.095+0000 7f2419937b80 20 client.17241 trim_cache size 0 max 16384 2023-07-21T22:30:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:30:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:30:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:30:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 msgr2=0x7f23f00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 0x7f23f00a38c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009fd0 tx=0x7f23f8019040 comp rx=0 tx=0).stop 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 msgr2=0x7f2408195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408195ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2400007f10 tx=0x7f240000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 shutdown_connections 2023-07-21T22:30:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2408104a30 0x7f2408195ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24080fea30 0x7f240819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f00a1410 0x7f23f00a38c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 --2- 172.21.15.93:0/2033330820 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24080ff3e0 0x7f240819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 >> 172.21.15.93:0/2033330820 conn(0x7f2408074760 msgr2=0x7f24081028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 shutdown_connections 2023-07-21T22:30:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:07.096+0000 7f240d7b2700 1 -- 172.21.15.93:0/2033330820 wait complete. 2023-07-21T22:30:07.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 22109 31.92 MB/sec warmup 43 sec latency 80.213 ms 2023-07-21T22:30:07.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:07 smithi194 ceph-mon[98288]: pgmap v433: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 651 op/s 2023-07-21T22:30:07.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:07 smithi134 ceph-mon[97873]: pgmap v433: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 651 op/s 2023-07-21T22:30:07.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:07 smithi093 ceph-mon[96579]: pgmap v433: 97 pgs: 97 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 651 op/s 2023-07-21T22:30:08.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 22797 31.85 MB/sec warmup 44 sec latency 42.919 ms 2023-07-21T22:30:09.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 23734 32.11 MB/sec warmup 45 sec latency 78.276 ms 2023-07-21T22:30:09.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:09 smithi194 ceph-mon[98288]: pgmap v434: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:09.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:09 smithi134 ceph-mon[97873]: pgmap v434: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:09.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:09 smithi093 ceph-mon[96579]: pgmap v434: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:10.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 24745 32.73 MB/sec warmup 46 sec latency 54.423 ms 2023-07-21T22:30:11.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 25567 33.16 MB/sec warmup 47 sec latency 48.760 ms 2023-07-21T22:30:11.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:11 smithi194 ceph-mon[98288]: pgmap v435: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:11.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:11 smithi134 ceph-mon[97873]: pgmap v435: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:11.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:11 smithi093 ceph-mon[96579]: pgmap v435: 97 pgs: 97 active+clean; 640 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 906 op/s 2023-07-21T22:30:12.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 26077 32.87 MB/sec warmup 48 sec latency 107.076 ms 2023-07-21T22:30:13.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 26918 33.09 MB/sec warmup 49 sec latency 72.643 ms 2023-07-21T22:30:13.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:13 smithi194 ceph-mon[98288]: pgmap v436: 97 pgs: 97 active+clean; 660 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 964 op/s 2023-07-21T22:30:13.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:13 smithi194 ceph-mon[98288]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:30:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:13 smithi134 ceph-mon[97873]: pgmap v436: 97 pgs: 97 active+clean; 660 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 964 op/s 2023-07-21T22:30:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:13 smithi134 ceph-mon[97873]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:30:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:13 smithi093 ceph-mon[96579]: pgmap v436: 97 pgs: 97 active+clean; 660 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 17 MiB/s wr, 964 op/s 2023-07-21T22:30:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:13 smithi093 ceph-mon[96579]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:30:14.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 27714 33.15 MB/sec warmup 50 sec latency 159.314 ms 2023-07-21T22:30:15.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 28507 33.51 MB/sec warmup 51 sec latency 77.903 ms 2023-07-21T22:30:15.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:15 smithi194 ceph-mon[98288]: pgmap v438: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-21T22:30:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:15 smithi134 ceph-mon[97873]: pgmap v438: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-21T22:30:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:15 smithi093 ceph-mon[96579]: pgmap v438: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-21T22:30:16.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 29249 33.78 MB/sec warmup 52 sec latency 53.342 ms 2023-07-21T22:30:17.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 30020 33.76 MB/sec warmup 53 sec latency 44.370 ms 2023-07-21T22:30:17.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:17 smithi194 ceph-mon[98288]: pgmap v439: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 991 op/s 2023-07-21T22:30:17.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:17 smithi134 ceph-mon[97873]: pgmap v439: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 991 op/s 2023-07-21T22:30:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:17 smithi093 ceph-mon[96579]: pgmap v439: 97 pgs: 97 active+clean; 672 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 18 MiB/s wr, 991 op/s 2023-07-21T22:30:18.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 30841 33.96 MB/sec warmup 54 sec latency 85.253 ms 2023-07-21T22:30:19.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 31714 34.15 MB/sec warmup 55 sec latency 59.280 ms 2023-07-21T22:30:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:19 smithi194 ceph-mon[98288]: pgmap v440: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:19.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:19 smithi134 ceph-mon[97873]: pgmap v440: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:19.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:19 smithi093 ceph-mon[96579]: pgmap v440: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 -- 172.21.15.93:0/2227813522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 msgr2=0x7f50480679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 --2- 172.21.15.93:0/2227813522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50480679a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f504000b0a0 tx=0x7f504000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 -- 172.21.15.93:0/2227813522 shutdown_connections 2023-07-21T22:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 --2- 172.21.15.93:0/2227813522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50481024f0 0x7f5048102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 --2- 172.21.15.93:0/2227813522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50480679a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 --2- 172.21.15.93:0/2227813522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5048067ee0 0x7f5048068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 -- 172.21.15.93:0/2227813522 >> 172.21.15.93:0/2227813522 conn(0x7f5048074760 msgr2=0x7f5048074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 -- 172.21.15.93:0/2227813522 shutdown_connections 2023-07-21T22:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.235+0000 7f504fcb3700 1 -- 172.21.15.93:0/2227813522 wait complete. 2023-07-21T22:30:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.236+0000 7f504fcb3700 1 Processor -- start 2023-07-21T22:30:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.236+0000 7f504fcb3700 1 -- start start 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5048067ee0 0x7f504819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 0x7f504819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481a15a0 con 0x7f5048104a60 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481a1710 con 0x7f50481024f0 2023-07-21T22:30:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504fcb3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481a18d0 con 0x7f5048067ee0 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504e250700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504e250700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48330/0 (socket says 172.21.15.93:48330) 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504e250700 1 -- 172.21.15.93:0/3181084749 learned_addr learned my addr 172.21.15.93:0/3181084749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 0x7f504819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504da4f700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5048067ee0 0x7f504819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 -- 172.21.15.93:0/3181084749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5048067ee0 msgr2=0x7f504819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5048067ee0 0x7f504819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 -- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 msgr2=0x7f50481a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 -- 172.21.15.93:0/3181084749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009d00 con 0x7f50481024f0 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504e250700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.237+0000 7f504d24e700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 0x7f504819c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f504400e9f0 tx=0x7f504400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.239+0000 7f503effd700 1 -- 172.21.15.93:0/3181084749 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504400c9c0 con 0x7f50481024f0 2023-07-21T22:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.239+0000 7f503effd700 1 -- 172.21.15.93:0/3181084749 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5044010430 con 0x7f50481024f0 2023-07-21T22:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.239+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50481a1bb0 con 0x7f50481024f0 2023-07-21T22:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.239+0000 7f503effd700 1 -- 172.21.15.93:0/3181084749 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044018680 con 0x7f50481024f0 2023-07-21T22:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.239+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50481a93f0 con 0x7f50481024f0 2023-07-21T22:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.241+0000 7f503effd700 1 -- 172.21.15.93:0/3181084749 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5044010860 con 0x7f50481024f0 2023-07-21T22:30:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.242+0000 7f503effd700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 0x7f50340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.242+0000 7f504da4f700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 0x7f50340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.242+0000 7f503effd700 1 -- 172.21.15.93:0/3181084749 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5044014070 con 0x7f50481024f0 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.243+0000 7f504da4f700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 0x7f50340a37f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5040009fd0 tx=0x7f504000b700 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.243+0000 7f505be38b80 1 Processor -- start 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.243+0000 7f505be38b80 1 -- start start 2023-07-21T22:30:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563690cfc50 con 0x5563690266b0 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563691229e0 con 0x5563691d0660 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f505be38b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636911be70 con 0x5563691d0210 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48334/0 (socket says 172.21.15.93:48334) 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 -- 172.21.15.93:0/3914365271 learned_addr learned my addr 172.21.15.93:0/3914365271 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504d24e700 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504e250700 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 -- 172.21.15.93:0/3914365271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 msgr2=0x5563691c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 -- 172.21.15.93:0/3914365271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 msgr2=0x5563691ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.244+0000 7f504da4f700 1 -- 172.21.15.93:0/3914365271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556368f49d80 con 0x5563690266b0 2023-07-21T22:30:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.245+0000 7f504da4f700 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f504000bcb0 tx=0x7f504001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6688cd4da49bb26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.245+0000 7f5033fff700 1 -- 172.21.15.93:0/3914365271 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504000e040 con 0x5563690266b0 2023-07-21T22:30:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.245+0000 7f5033fff700 1 -- 172.21.15.93:0/3914365271 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50400076e0 con 0x5563690266b0 2023-07-21T22:30:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.245+0000 7f5033fff700 1 -- 172.21.15.93:0/3914365271 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400052a0 con 0x5563690266b0 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- 172.21.15.93:0/3914365271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 msgr2=0x5563691cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f504000bcb0 tx=0x7f504001c000 comp rx=0 tx=0).stop 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- 172.21.15.93:0/3914365271 shutdown_connections 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563690266b0 0x5563691cfcd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 --2- 172.21.15.93:0/3914365271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- 172.21.15.93:0/3914365271 >> 172.21.15.93:0/3914365271 conn(0x5563691bfae0 msgr2=0x5563691c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- 172.21.15.93:0/3914365271 shutdown_connections 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- 172.21.15.93:0/3914365271 wait complete. 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.246+0000 7f505be38b80 1 -- start start 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 Processor -- start 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563690bba60 con 0x5563691d0210 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563690bb760 con 0x5563691d0660 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556368f9f8c0 con 0x5563690266b0 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f505be38b80 10 client.0 _ll_register_callbacks cb 0x5563690b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f504d24e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f504d24e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48342/0 (socket says 172.21.15.93:48342) 2023-07-21T22:30:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f504d24e700 1 -- 172.21.15.93:0/1120424176 learned_addr learned my addr 172.21.15.93:0/1120424176 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.247+0000 7f504e250700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 -- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 msgr2=0x5563691d6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 -- 172.21.15.93:0/1120424176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 msgr2=0x5563691d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 -- 172.21.15.93:0/1120424176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5048113930 con 0x5563691d0210 2023-07-21T22:30:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f504d24e700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f50440ca5d0 tx=0x7f5044099f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc780ecbaa4a1d9b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f50317fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:30:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044099590 con 0x5563691d0210 2023-07-21T22:30:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f50440a1e60 con 0x5563691d0210 2023-07-21T22:30:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556369075490 con 0x5563691d0210 2023-07-21T22:30:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50440a2020 con 0x5563691d0210 2023-07-21T22:30:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f505be38b80 10 client.17253 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x556368f32c00 con 0x5563691d0210 2023-07-21T22:30:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.248+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556369075490 con 0x5563691d0210 2023-07-21T22:30:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f50440c9fe0 con 0x5563691d0210 2023-07-21T22:30:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 20 client.17253 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691e3b60 0x5563691e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 4 client.17253 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556369114e50 con 0x5563691e3b60 2023-07-21T22:30:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f504da4f700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691e3b60 0x5563691e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f504da4f700 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691e3b60 0x5563691e5f70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e59184b9d30ad39 server_cookie=be0bf8168f6c4bc8 in_seq=0 out_seq=0 2023-07-21T22:30:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.249+0000 7f50317fa700 10 client.17253 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f50317fa700 1 -- 172.21.15.93:0/1120424176 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556369114e50 con 0x5563691e3b60 2023-07-21T22:30:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f50317fa700 10 client.17253 handle_command_reply: tid=0 2023-07-21T22:30:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 client.17253 shutdown 2023-07-21T22:30:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 msgr2=0x5563691d37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f50440ca5d0 tx=0x7f5044099f50 comp rx=0 tx=0).stop 2023-07-21T22:30:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 shutdown_connections 2023-07-21T22:30:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691e3b60 0x5563691e5f70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0660 0x5563691d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f50317fa700 0 client.17253 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 --2- 172.21.15.93:0/1120424176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691d0210 0x5563691d37c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 >> 172.21.15.93:0/1120424176 conn(0x5563691bfae0 msgr2=0x5563691c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 shutdown_connections 2023-07-21T22:30:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.252+0000 7f505be38b80 1 -- 172.21.15.93:0/1120424176 wait complete. 2023-07-21T22:30:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.253+0000 7f505be38b80 20 client.17253 trim_cache size 0 max 16384 2023-07-21T22:30:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 Processor -- start 2023-07-21T22:30:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 -- start start 2023-07-21T22:30:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.270+0000 7f505be38b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563691e1140 con 0x5563691bfae0 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f505be38b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563691caf20 con 0x5563691d0210 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f505be38b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563691cacd0 con 0x5563690266b0 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504da4f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504da4f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48348/0 (socket says 172.21.15.93:48348) 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504da4f700 1 -- 172.21.15.93:0/1505679636 learned_addr learned my addr 172.21.15.93:0/1505679636 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504d24e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 -- 172.21.15.93:0/1505679636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 msgr2=0x5563691d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 -- 172.21.15.93:0/1505679636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 msgr2=0x5563691cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 -- 172.21.15.93:0/1505679636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563691eaba0 con 0x5563691d0210 2023-07-21T22:30:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504d24e700 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.271+0000 7f504e250700 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 0x5563691c9f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50380096d0 tx=0x7f5038010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc61171ea30bff9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.272+0000 7f503cff9700 1 -- 172.21.15.93:0/1505679636 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503800f680 con 0x5563691d0210 2023-07-21T22:30:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.272+0000 7f503cff9700 1 -- 172.21.15.93:0/1505679636 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5038017040 con 0x5563691d0210 2023-07-21T22:30:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.272+0000 7f503cff9700 1 -- 172.21.15.93:0/1505679636 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503801ba20 con 0x5563691d0210 2023-07-21T22:30:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.272+0000 7f505be38b80 1 -- 172.21.15.93:0/1505679636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 msgr2=0x5563691c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.272+0000 7f505be38b80 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 0x5563691c9f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50380096d0 tx=0x7f5038010a30 comp rx=0 tx=0).stop 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 -- 172.21.15.93:0/1505679636 shutdown_connections 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563691d0210 0x5563691c9f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 --2- 172.21.15.93:0/1505679636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563690266b0 0x5563691d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 -- 172.21.15.93:0/1505679636 >> 172.21.15.93:0/1505679636 conn(0x5563691df670 msgr2=0x5563691d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 -- 172.21.15.93:0/1505679636 shutdown_connections 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 -- 172.21.15.93:0/1505679636 wait complete. 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 -- start start 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.273+0000 7f505be38b80 1 Processor -- start 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563691ccb50 con 0x5563691bfae0 2023-07-21T22:30:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556369237a20 con 0x5563690266b0 2023-07-21T22:30:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556369237b60 con 0x5563691d0210 2023-07-21T22:30:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f505be38b80 10 client.0 _ll_register_callbacks cb 0x7f50440999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59516/0 (socket says 172.21.15.93:59516) 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504e250700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 -- 172.21.15.93:0/2037206965 learned_addr learned my addr 172.21.15.93:0/2037206965 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504d24e700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 -- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 msgr2=0x5563691cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 -- 172.21.15.93:0/2037206965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 msgr2=0x5563691cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 -- 172.21.15.93:0/2037206965 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556368f32c00 con 0x5563690266b0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f504da4f700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5040000f80 tx=0x7f5040009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc162342328a97f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.274+0000 7f50317fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040035040 con 0x5563690266b0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50400076e0 con 0x5563690266b0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f504003b360 con 0x5563690266b0 2023-07-21T22:30:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504000e040 con 0x5563690266b0 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 10 client.26642 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7f5038010430 con 0x5563690266b0 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50400076e0 con 0x5563690266b0 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f5040039030 con 0x5563690266b0 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 10 client.26642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 20 client.26642 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691d0660 0x55636923ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 4 client.26642 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5563690d6ff0 con 0x5563691d0660 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.275+0000 7f504d24e700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691d0660 0x55636923ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.276+0000 7f50317fa700 10 client.26642 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.276+0000 7f504d24e700 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691d0660 0x55636923ae30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4ac6fb5d7093d2 server_cookie=6dbe22b404b9dcd8 in_seq=0 out_seq=0 2023-07-21T22:30:20.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 32558 34.53 MB/sec warmup 56 sec latency 103.191 ms 2023-07-21T22:30:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f50317fa700 1 -- 172.21.15.93:0/2037206965 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5563690d6ff0 con 0x5563691d0660 2023-07-21T22:30:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f50317fa700 10 client.26642 handle_command_reply: tid=0 2023-07-21T22:30:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 client.26642 shutdown 2023-07-21T22:30:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 msgr2=0x5563691dad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5040000f80 tx=0x7f5040009c30 comp rx=0 tx=0).stop 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 shutdown_connections 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5563691d0660 0x55636923ae30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5563691d0210 0x5563691cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f50317fa700 0 client.26642 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563690266b0 0x5563691dad40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 --2- 172.21.15.93:0/2037206965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5563691bfae0 0x5563691cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 >> 172.21.15.93:0/2037206965 conn(0x5563691df670 msgr2=0x5563691cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.461+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 shutdown_connections 2023-07-21T22:30:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f505be38b80 1 -- 172.21.15.93:0/2037206965 wait complete. 2023-07-21T22:30:20.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f505be38b80 20 client.26642 trim_cache size 0 max 16384 2023-07-21T22:30:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:30:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:30:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:30:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:30:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 msgr2=0x7f50340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 0x7f50340a37f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5040009fd0 tx=0x7f504000b700 comp rx=0 tx=0).stop 2023-07-21T22:30:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 msgr2=0x7f504819c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 0x7f504819c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f504400e9f0 tx=0x7f504400ed00 comp rx=0 tx=0).stop 2023-07-21T22:30:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 shutdown_connections 2023-07-21T22:30:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f50340a1340 0x7f50340a37f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5048104a60 0x7f50481a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5048067ee0 0x7f504819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 --2- 172.21.15.93:0/3181084749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50481024f0 0x7f504819c8f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 >> 172.21.15.93:0/3181084749 conn(0x7f5048074760 msgr2=0x7f50481014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.462+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 shutdown_connections 2023-07-21T22:30:20.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:20.463+0000 7f504fcb3700 1 -- 172.21.15.93:0/3181084749 wait complete. 2023-07-21T22:30:21.489 INFO:tasks.workunit.client.0.smithi093.stdout: 10 33334 34.63 MB/sec warmup 57 sec latency 86.778 ms 2023-07-21T22:30:21.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:21 smithi194 ceph-mon[98288]: pgmap v441: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:21.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:21 smithi134 ceph-mon[97873]: pgmap v441: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:21 smithi093 ceph-mon[96579]: pgmap v441: 97 pgs: 97 active+clean; 699 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-07-21T22:30:22.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 34124 34.73 MB/sec warmup 58 sec latency 35.802 ms 2023-07-21T22:30:23.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 34961 34.72 MB/sec warmup 59 sec latency 82.669 ms 2023-07-21T22:30:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:23 smithi194 ceph-mon[98288]: pgmap v442: 97 pgs: 97 active+clean; 712 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2023-07-21T22:30:23.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:23 smithi134 ceph-mon[97873]: pgmap v442: 97 pgs: 97 active+clean; 712 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2023-07-21T22:30:23.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:23 smithi093 ceph-mon[96579]: pgmap v442: 97 pgs: 97 active+clean; 712 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 678 op/s 2023-07-21T22:30:24.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 35974 35.14 MB/sec warmup 60 sec latency 57.632 ms 2023-07-21T22:30:25.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 36722 35.34 MB/sec warmup 61 sec latency 156.399 ms 2023-07-21T22:30:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:25 smithi194 ceph-mon[98288]: pgmap v443: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 689 op/s 2023-07-21T22:30:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:25 smithi134 ceph-mon[97873]: pgmap v443: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 689 op/s 2023-07-21T22:30:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:25 smithi093 ceph-mon[96579]: pgmap v443: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 689 op/s 2023-07-21T22:30:26.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 37442 35.36 MB/sec warmup 62 sec latency 46.334 ms 2023-07-21T22:30:27.490 INFO:tasks.workunit.client.0.smithi093.stdout: 10 38273 35.43 MB/sec warmup 63 sec latency 67.850 ms 2023-07-21T22:30:27.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:27 smithi194 ceph-mon[98288]: pgmap v444: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 597 op/s 2023-07-21T22:30:27.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:27 smithi134 ceph-mon[97873]: pgmap v444: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 597 op/s 2023-07-21T22:30:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:27 smithi093 ceph-mon[96579]: pgmap v444: 97 pgs: 97 active+clean; 722 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 597 op/s 2023-07-21T22:30:28.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 39258 35.71 MB/sec warmup 64 sec latency 60.108 ms 2023-07-21T22:30:29.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 39286 35.16 MB/sec warmup 65 sec latency 917.469 ms 2023-07-21T22:30:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:29 smithi194 ceph-mon[98288]: pgmap v445: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.2 MiB/s wr, 868 op/s 2023-07-21T22:30:29.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:29 smithi134 ceph-mon[97873]: pgmap v445: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.2 MiB/s wr, 868 op/s 2023-07-21T22:30:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:29 smithi093 ceph-mon[96579]: pgmap v445: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.2 MiB/s wr, 868 op/s 2023-07-21T22:30:30.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 39776 35.15 MB/sec warmup 66 sec latency 936.925 ms 2023-07-21T22:30:31.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 40297 35.04 MB/sec warmup 67 sec latency 110.799 ms 2023-07-21T22:30:31.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:31 smithi194 ceph-mon[98288]: pgmap v446: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 616 op/s 2023-07-21T22:30:31.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:31 smithi134 ceph-mon[97873]: pgmap v446: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 616 op/s 2023-07-21T22:30:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:31 smithi093 ceph-mon[96579]: pgmap v446: 97 pgs: 97 active+clean; 751 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 616 op/s 2023-07-21T22:30:32.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 41061 35.07 MB/sec warmup 68 sec latency 50.258 ms 2023-07-21T22:30:33.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 41945 35.22 MB/sec warmup 69 sec latency 45.069 ms 2023-07-21T22:30:33.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:33 smithi194 ceph-mon[98288]: pgmap v447: 97 pgs: 97 active+clean; 763 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-21T22:30:33.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:33 smithi134 ceph-mon[97873]: pgmap v447: 97 pgs: 97 active+clean; 763 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-21T22:30:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.599+0000 7fd52d3a6700 1 -- 172.21.15.93:0/1272246126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd528104a30 msgr2=0x7fd528104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.599+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/1272246126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd528104a30 0x7fd528104e00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd518009990 tx=0x7fd518009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 -- 172.21.15.93:0/1272246126 shutdown_connections 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/1272246126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5280ff3e0 0x7fd52810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/1272246126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd528104a30 0x7fd528104e00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/1272246126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280fea30 0x7fd5280feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 -- 172.21.15.93:0/1272246126 >> 172.21.15.93:0/1272246126 conn(0x7fd528074760 msgr2=0x7fd528074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 -- 172.21.15.93:0/1272246126 shutdown_connections 2023-07-21T22:30:33.648 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 -- 172.21.15.93:0/1272246126 wait complete. 2023-07-21T22:30:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.600+0000 7fd52d3a6700 1 Processor -- start 2023-07-21T22:30:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 -- start start 2023-07-21T22:30:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280ff3e0 0x7fd52819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd528104a30 0x7fd5281959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528196040 con 0x7fd5280fea30 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5281961b0 con 0x7fd5280ff3e0 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd52d3a6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528196370 con 0x7fd528104a30 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48368/0 (socket says 172.21.15.93:48368) 2023-07-21T22:30:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 -- 172.21.15.93:0/2940202175 learned_addr learned my addr 172.21.15.93:0/2940202175 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280ff3e0 0x7fd52819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd5277fe700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd528104a30 0x7fd5281959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 -- 172.21.15.93:0/2940202175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd528104a30 msgr2=0x7fd5281959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd528104a30 0x7fd5281959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 -- 172.21.15.93:0/2940202175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280ff3e0 msgr2=0x7fd52819c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280ff3e0 0x7fd52819c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 -- 172.21.15.93:0/2940202175 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180095f0 con 0x7fd5280fea30 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.601+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd518009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.602+0000 7fd50ffff700 1 -- 172.21.15.93:0/2940202175 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801d070 con 0x7fd5280fea30 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.602+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5281965f0 con 0x7fd5280fea30 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.602+0000 7fd50ffff700 1 -- 172.21.15.93:0/2940202175 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd518022470 con 0x7fd5280fea30 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.602+0000 7fd50ffff700 1 -- 172.21.15.93:0/2940202175 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518017630 con 0x7fd5280fea30 2023-07-21T22:30:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.602+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5281a2b60 con 0x7fd5280fea30 2023-07-21T22:30:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.604+0000 7fd50ffff700 1 -- 172.21.15.93:0/2940202175 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd5180228e0 con 0x7fd5280fea30 2023-07-21T22:30:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.605+0000 7fd50ffff700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 0x7fd5100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.605+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 0x7fd5100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.605+0000 7fd50ffff700 1 -- 172.21.15.93:0/2940202175 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd5180cde90 con 0x7fd5280fea30 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.605+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 0x7fd5100a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5280fc860 tx=0x7fd51c009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 Processor -- start 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 -- start start 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed660 0x5598ea2e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea1ecc50 con 0x5598ea2ed210 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea23f9e0 con 0x5598ea1436b0 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd53952bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea238e70 con 0x5598ea2ed660 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd5267fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd5267fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48378/0 (socket says 172.21.15.93:48378) 2023-07-21T22:30:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd5267fc700 1 -- 172.21.15.93:0/3948531225 learned_addr learned my addr 172.21.15.93:0/3948531225 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.606+0000 7fd526ffd700 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5277fe700 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed660 0x5598ea2e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 -- 172.21.15.93:0/3948531225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed660 msgr2=0x5598ea2e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed660 0x5598ea2e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 -- 172.21.15.93:0/3948531225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 msgr2=0x5598ea2eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 -- 172.21.15.93:0/3948531225 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ea066d80 con 0x5598ea2ed210 2023-07-21T22:30:33.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd5267fc700 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011d50 tx=0x7fd51c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17314e34ef475c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd50d7fa700 1 -- 172.21.15.93:0/3948531225 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00eb20 con 0x5598ea2ed210 2023-07-21T22:30:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd50d7fa700 1 -- 172.21.15.93:0/3948531225 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c021070 con 0x5598ea2ed210 2023-07-21T22:30:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.607+0000 7fd50d7fa700 1 -- 172.21.15.93:0/3948531225 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c01d670 con 0x5598ea2ed210 2023-07-21T22:30:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- 172.21.15.93:0/3948531225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 msgr2=0x5598ea2e6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011d50 tx=0x7fd51c014000 comp rx=0 tx=0).stop 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- 172.21.15.93:0/3948531225 shutdown_connections 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed660 0x5598ea2e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 --2- 172.21.15.93:0/3948531225 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e6f40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- 172.21.15.93:0/3948531225 >> 172.21.15.93:0/3948531225 conn(0x5598ea2dcae0 msgr2=0x5598ea2e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- 172.21.15.93:0/3948531225 shutdown_connections 2023-07-21T22:30:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- 172.21.15.93:0/3948531225 wait complete. 2023-07-21T22:30:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 -- start start 2023-07-21T22:30:33.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.608+0000 7fd53952bb80 1 Processor -- start 2023-07-21T22:30:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea1d8a60 con 0x5598ea1436b0 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea1d8760 con 0x5598ea2ed660 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea0bc8c0 con 0x5598ea2ed210 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 10 client.0 _ll_register_callbacks cb 0x5598ea1d6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48382/0 (socket says 172.21.15.93:48382) 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 -- 172.21.15.93:0/1534769718 learned_addr learned my addr 172.21.15.93:0/1534769718 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:33.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 -- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 msgr2=0x5598ea2f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 -- 172.21.15.93:0/1534769718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 msgr2=0x5598ea2f0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 -- 172.21.15.93:0/1534769718 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c017070 con 0x5598ea1436b0 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd526ffd700 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd518022c90 tx=0x7fd51809aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c55dfe4229df24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd50aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180a33b0 con 0x5598ea1436b0 2023-07-21T22:30:33.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd51809a4e0 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.609+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598ea192490 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd51809a6a0 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 10 client.17271 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5598ea04fc00 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5180b9030 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fd518096de0 con 0x5598ea1436b0 2023-07-21T22:30:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 10 client.17271 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 20 client.17271 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3008d0 0x5598ea302ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 4 client.17271 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598ea231e50 con 0x5598ea3008d0 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.610+0000 7fd5267fc700 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3008d0 0x5598ea302ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.611+0000 7fd5267fc700 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3008d0 0x5598ea302ce0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4e72fab385c75f server_cookie=98df9ebe1ee1c6ec in_seq=0 out_seq=0 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.611+0000 7fd50aff5700 10 client.17271 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd50aff5700 1 -- 172.21.15.93:0/1534769718 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598ea231e50 con 0x5598ea3008d0 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd50aff5700 10 client.17271 handle_command_reply: tid=0 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 client.17271 shutdown 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 msgr2=0x5598ea2f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd518022c90 tx=0x7fd51809aa80 comp rx=0 tx=0).stop 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 shutdown_connections 2023-07-21T22:30:33.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3008d0 0x5598ea302ce0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd50aff5700 0 client.17271 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea1436b0 0x5598ea2f3630 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 --2- 172.21.15.93:0/1534769718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 >> 172.21.15.93:0/1534769718 conn(0x5598ea2dcae0 msgr2=0x5598ea2e13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 shutdown_connections 2023-07-21T22:30:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 1 -- 172.21.15.93:0/1534769718 wait complete. 2023-07-21T22:30:33.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.613+0000 7fd53952bb80 20 client.17271 trim_cache size 0 max 16384 2023-07-21T22:30:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 Processor -- start 2023-07-21T22:30:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 -- start start 2023-07-21T22:30:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea1436b0 0x5598ea2e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea231e50 con 0x5598ea2ed210 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea2eccd0 con 0x5598ea2ed660 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.632+0000 7fd53952bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea2eb2a0 con 0x5598ea1436b0 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59554/0 (socket says 172.21.15.93:59554) 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 -- 172.21.15.93:0/4145696691 learned_addr learned my addr 172.21.15.93:0/4145696691 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd526ffd700 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea1436b0 0x5598ea2e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5267fc700 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 -- 172.21.15.93:0/4145696691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea1436b0 msgr2=0x5598ea2e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea1436b0 0x5598ea2e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 -- 172.21.15.93:0/4145696691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 msgr2=0x5598ea2e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 -- 172.21.15.93:0/4145696691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ea2e37e0 con 0x5598ea2ed660 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd5277fe700 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5200093d0 tx=0x7fd520009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03133f0bbd06aa6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.633+0000 7fd50dffb700 1 -- 172.21.15.93:0/4145696691 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000e040 con 0x5598ea2ed660 2023-07-21T22:30:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd50dffb700 1 -- 172.21.15.93:0/4145696691 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd52000f040 con 0x5598ea2ed660 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd50dffb700 1 -- 172.21.15.93:0/4145696691 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520016e50 con 0x5598ea2ed660 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 -- 172.21.15.93:0/4145696691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 msgr2=0x5598ea2e9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5200093d0 tx=0x7fd520009400 comp rx=0 tx=0).stop 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 -- 172.21.15.93:0/4145696691 shutdown_connections 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea2ed660 0x5598ea2e9740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea1436b0 0x5598ea2e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 --2- 172.21.15.93:0/4145696691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed210 0x5598ea2e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 -- 172.21.15.93:0/4145696691 >> 172.21.15.93:0/4145696691 conn(0x5598ea2dcae0 msgr2=0x5598ea2ee330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 -- 172.21.15.93:0/4145696691 shutdown_connections 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.634+0000 7fd53952bb80 1 -- 172.21.15.93:0/4145696691 wait complete. 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 -- start start 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 Processor -- start 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed660 0x5598ea34d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea34dd10 con 0x5598ea2ed660 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea34de80 con 0x5598ea1436b0 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ea34dff0 con 0x5598ea2ed210 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd53952bb80 10 client.0 _ll_register_callbacks cb 0x5598ea1d6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd526ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59558/0 (socket says 172.21.15.93:59558) 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd526ffd700 1 -- 172.21.15.93:0/2941805402 learned_addr learned my addr 172.21.15.93:0/2941805402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.635+0000 7fd5277fe700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed660 0x5598ea34d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 -- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 msgr2=0x5598ea2f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 -- 172.21.15.93:0/2941805402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed660 msgr2=0x5598ea34d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed660 0x5598ea34d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 -- 172.21.15.93:0/2941805402 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd520009050 con 0x5598ea1436b0 2023-07-21T22:30:33.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd526ffd700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5180973e0 tx=0x7fd5180092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc4b16ceda8de4d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd50aff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:30:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180c9070 con 0x5598ea1436b0 2023-07-21T22:30:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd5180cb2b0 con 0x5598ea1436b0 2023-07-21T22:30:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd51c021070 con 0x5598ea1436b0 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5180cb470 con 0x5598ea1436b0 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd53952bb80 10 client.26657 fetch_fsmap learned FSMap version 66 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5598ea08d1f0 con 0x5598ea1436b0 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.636+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c021070 con 0x5598ea1436b0 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 66) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fd518099f70 con 0x5598ea1436b0 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 10 client.26657 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 20 client.26657 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3552d0 0x5598ea3576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 4 client.26657 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5598ea1f3ff0 con 0x5598ea3552d0 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3552d0 0x5598ea3576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd50aff5700 10 client.26657 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:33.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.637+0000 7fd5267fc700 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3552d0 0x5598ea3576b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e2cf13af88225e server_cookie=389f799ff956c5a1 in_seq=0 out_seq=0 2023-07-21T22:30:33.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:33 smithi093 ceph-mon[96579]: pgmap v447: 97 pgs: 97 active+clean; 763 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 969 op/s 2023-07-21T22:30:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.788+0000 7fd50aff5700 1 -- 172.21.15.93:0/2941805402 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5598ea1f3ff0 con 0x5598ea3552d0 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.788+0000 7fd50aff5700 10 client.26657 handle_command_reply: tid=0 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.788+0000 7fd53952bb80 1 client.26657 shutdown 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 msgr2=0x5598ea2f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5180973e0 tx=0x7fd5180092f0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 shutdown_connections 2023-07-21T22:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5598ea3552d0 0x5598ea3576b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5598ea2ed660 0x5598ea34d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598ea1436b0 0x5598ea2f4350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 --2- 172.21.15.93:0/2941805402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598ea2ed210 0x5598ea2f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 >> 172.21.15.93:0/2941805402 conn(0x5598ea2dcae0 msgr2=0x5598ea2eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd50aff5700 0 client.26657 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 shutdown_connections 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 1 -- 172.21.15.93:0/2941805402 wait complete. 2023-07-21T22:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.789+0000 7fd53952bb80 20 client.26657 trim_cache size 0 max 16384 2023-07-21T22:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 msgr2=0x7fd5100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 0x7fd5100a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5280fc860 tx=0x7fd51c009260 comp rx=0 tx=0).stop 2023-07-21T22:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 msgr2=0x7fd52819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd518000c00 tx=0x7fd518009580 comp rx=0 tx=0).stop 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 shutdown_connections 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd528104a30 0x7fd5281959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5100a12c0 0x7fd5100a3770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5280fea30 0x7fd52819bb70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 --2- 172.21.15.93:0/2940202175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5280ff3e0 0x7fd52819c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.790+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 >> 172.21.15.93:0/2940202175 conn(0x7fd528074760 msgr2=0x7fd5280fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.791+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 shutdown_connections 2023-07-21T22:30:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:33.791+0000 7fd52d3a6700 1 -- 172.21.15.93:0/2940202175 wait complete. 2023-07-21T22:30:34.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 42669 35.34 MB/sec warmup 70 sec latency 108.578 ms 2023-07-21T22:30:34.882 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:30:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:30:34.883 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:30:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:30:34.883 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:30:34 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/1224496833 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5606ffbc71e0 con 0x5606ffb9c800 2023-07-21T22:30:34.883 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:30:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:30:35.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 43355 35.42 MB/sec warmup 71 sec latency 115.399 ms 2023-07-21T22:30:35.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:35 smithi194 ceph-mon[98288]: pgmap v448: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2023-07-21T22:30:35.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:35 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:30:35.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:35 smithi134 ceph-mon[97873]: pgmap v448: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2023-07-21T22:30:35.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:35 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:30:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:35 smithi093 ceph-mon[96579]: pgmap v448: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 21 MiB/s wr, 1.09k op/s 2023-07-21T22:30:35.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:35 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:30:36.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 44122 35.47 MB/sec warmup 72 sec latency 46.588 ms 2023-07-21T22:30:36.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:36 smithi194 ceph-mon[98288]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:30:36.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:36 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/1309294921,v1:172.21.15.134:6837/1309294921] up:boot 2023-07-21T22:30:36.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:30:36.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:36 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:30:36.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:36.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:36 smithi134 ceph-mon[97873]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:30:36.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:36 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/1309294921,v1:172.21.15.134:6837/1309294921] up:boot 2023-07-21T22:30:36.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:30:36.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:36 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:30:36.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:36.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:36 smithi093 ceph-mon[96579]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:30:36.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:36 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/1309294921,v1:172.21.15.134:6837/1309294921] up:boot 2023-07-21T22:30:36.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-21T22:30:36.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:36 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:30:36.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:37.499 INFO:tasks.workunit.client.0.smithi093.stdout: 10 44980 35.59 MB/sec warmup 73 sec latency 50.154 ms 2023-07-21T22:30:37.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:37 smithi194 ceph-mon[98288]: pgmap v450: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2023-07-21T22:30:37.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:37 smithi134 ceph-mon[97873]: pgmap v450: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2023-07-21T22:30:37.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:37 smithi093 ceph-mon[96579]: pgmap v450: 97 pgs: 97 active+clean; 774 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 24 MiB/s wr, 1.16k op/s 2023-07-21T22:30:38.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 45886 35.69 MB/sec warmup 74 sec latency 45.582 ms 2023-07-21T22:30:39.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 46863 36.06 MB/sec warmup 75 sec latency 44.639 ms 2023-07-21T22:30:39.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:39 smithi194 ceph-mon[98288]: pgmap v451: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:39.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:39 smithi134 ceph-mon[97873]: pgmap v451: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:39 smithi093 ceph-mon[96579]: pgmap v451: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:40.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 47698 36.14 MB/sec warmup 76 sec latency 38.802 ms 2023-07-21T22:30:41.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 48631 36.29 MB/sec warmup 77 sec latency 36.894 ms 2023-07-21T22:30:41.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:41 smithi194 ceph-mon[98288]: pgmap v452: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:41.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:41 smithi134 ceph-mon[97873]: pgmap v452: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:41.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:41 smithi093 ceph-mon[96579]: pgmap v452: 97 pgs: 97 active+clean; 783 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2023-07-21T22:30:42.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 49486 36.39 MB/sec warmup 78 sec latency 37.286 ms 2023-07-21T22:30:43.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 50336 36.55 MB/sec warmup 79 sec latency 57.025 ms 2023-07-21T22:30:43.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:43 smithi194 ceph-mon[98288]: pgmap v453: 97 pgs: 97 active+clean; 802 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 993 op/s 2023-07-21T22:30:43.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:43 smithi134 ceph-mon[97873]: pgmap v453: 97 pgs: 97 active+clean; 802 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 993 op/s 2023-07-21T22:30:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:43 smithi093 ceph-mon[96579]: pgmap v453: 97 pgs: 97 active+clean; 802 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 17 MiB/s wr, 993 op/s 2023-07-21T22:30:44.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 51180 36.64 MB/sec warmup 80 sec latency 67.958 ms 2023-07-21T22:30:45.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 51851 36.65 MB/sec warmup 81 sec latency 126.030 ms 2023-07-21T22:30:45.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:45 smithi194 ceph-mon[98288]: pgmap v454: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.6 MiB/s wr, 733 op/s 2023-07-21T22:30:45.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:45 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:45.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:45 smithi134 ceph-mon[97873]: pgmap v454: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.6 MiB/s wr, 733 op/s 2023-07-21T22:30:45.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:45 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:45 smithi093 ceph-mon[96579]: pgmap v454: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 7.6 MiB/s wr, 733 op/s 2023-07-21T22:30:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:45 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:46.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 52652 36.65 MB/sec warmup 82 sec latency 86.913 ms 2023-07-21T22:30:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 -- 172.21.15.93:0/1996707722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 msgr2=0x7f0fc4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/1996707722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc4104ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40098f0 tx=0x7f0fb4009c00 comp rx=0 tx=0).stop 2023-07-21T22:30:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 -- 172.21.15.93:0/1996707722 shutdown_connections 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/1996707722 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40ff470 0x7f0fc410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/1996707722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc4104ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/1996707722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40feac0 0x7f0fc40fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.937+0000 7f0fca0cb700 1 -- 172.21.15.93:0/1996707722 >> 172.21.15.93:0/1996707722 conn(0x7f0fc4075bd0 msgr2=0x7f0fc4075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.938+0000 7f0fca0cb700 1 -- 172.21.15.93:0/1996707722 shutdown_connections 2023-07-21T22:30:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.938+0000 7f0fca0cb700 1 -- 172.21.15.93:0/1996707722 wait complete. 2023-07-21T22:30:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.938+0000 7f0fca0cb700 1 Processor -- start 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 -- start start 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 0x7f0fc4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc41a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc41a2190 con 0x7f0fc40feac0 2023-07-21T22:30:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc41a2300 con 0x7f0fc40ff470 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fca0cb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4194930 con 0x7f0fc4104ad0 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48414/0 (socket says 172.21.15.93:48414) 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fc37fe700 1 -- 172.21.15.93:0/2217591097 learned_addr learned my addr 172.21.15.93:0/2217591097 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc41a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.939+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 0x7f0fc4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 -- 172.21.15.93:0/2217591097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 msgr2=0x7f0fc41a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc41a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 -- 172.21.15.93:0/2217591097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 msgr2=0x7f0fc4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 0x7f0fc4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 -- 172.21.15.93:0/2217591097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009520 con 0x7f0fc40feac0 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 0x7f0fc4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.940+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009fd0 tx=0x7f0fb400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.941+0000 7f0fc0ff9700 1 -- 172.21.15.93:0/2217591097 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb401c070 con 0x7f0fc40feac0 2023-07-21T22:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.941+0000 7f0fc0ff9700 1 -- 172.21.15.93:0/2217591097 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb4005040 con 0x7f0fc40feac0 2023-07-21T22:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.941+0000 7f0fc0ff9700 1 -- 172.21.15.93:0/2217591097 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4004a70 con 0x7f0fc40feac0 2023-07-21T22:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.941+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc4194bb0 con 0x7f0fc40feac0 2023-07-21T22:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.941+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc4195060 con 0x7f0fc40feac0 2023-07-21T22:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.944+0000 7f0fc0ff9700 1 -- 172.21.15.93:0/2217591097 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0fb4020070 con 0x7f0fc40feac0 2023-07-21T22:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.945+0000 7f0fc0ff9700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 0x7f0fac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.945+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 0x7f0fac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.945+0000 7f0fc0ff9700 1 -- 172.21.15.93:0/2217591097 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0fb40cda10 con 0x7f0fc40feac0 2023-07-21T22:30:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 Processor -- start 2023-07-21T22:30:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 -- start start 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x556517837cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517734c50 con 0x55651768b6b0 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565177879e0 con 0x55651783da20 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517780e70 con 0x55651783de70 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fc3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x556517837cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59582/0 (socket says 172.21.15.93:59582) 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.946+0000 7f0fc2ffd700 1 -- 172.21.15.93:0/1893336489 learned_addr learned my addr 172.21.15.93:0/1893336489 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 -- 172.21.15.93:0/1893336489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 msgr2=0x556517837cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x556517837cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 -- 172.21.15.93:0/1893336489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 msgr2=0x55651783d4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 -- 172.21.15.93:0/1893336489 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565175aed80 con 0x55651783da20 2023-07-21T22:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 0x7f0fac0a37f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40698b0 tx=0x7f0fb8008040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.947+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009790 tx=0x7f0fb8010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94a951199daccfe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.948+0000 7f0fa9ffb700 1 -- 172.21.15.93:0/1893336489 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8014640 con 0x55651783da20 2023-07-21T22:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.948+0000 7f0fa9ffb700 1 -- 172.21.15.93:0/1893336489 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb8014c60 con 0x55651783da20 2023-07-21T22:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.948+0000 7f0fa9ffb700 1 -- 172.21.15.93:0/1893336489 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb801fc80 con 0x55651783da20 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.948+0000 7f0fd6250b80 1 -- 172.21.15.93:0/1893336489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 msgr2=0x556517837750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.948+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009790 tx=0x7f0fb8010000 comp rx=0 tx=0).stop 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 -- 172.21.15.93:0/1893336489 shutdown_connections 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x556517837cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/1893336489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517837750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 -- 172.21.15.93:0/1893336489 >> 172.21.15.93:0/1893336489 conn(0x556517824ae0 msgr2=0x55651782a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 -- 172.21.15.93:0/1893336489 shutdown_connections 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 -- 172.21.15.93:0/1893336489 wait complete. 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 -- start start 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.949+0000 7f0fd6250b80 1 Processor -- start 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 0x556517843e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 0x556517841120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517841860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517720a60 con 0x55651783da20 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517720760 con 0x55651783de70 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565176048c0 con 0x55651768b6b0 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fd6250b80 10 client.0 _ll_register_callbacks cb 0x55651771e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 0x556517843e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 0x556517841120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 0x556517843e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35504/0 (socket says 172.21.15.93:35504) 2023-07-21T22:30:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc37fe700 1 -- 172.21.15.93:0/511970330 learned_addr learned my addr 172.21.15.93:0/511970330 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517841860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 0x556517841120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48424/0 (socket says 172.21.15.93:48424) 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 -- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 msgr2=0x556517843e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 0x556517843e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 -- 172.21.15.93:0/511970330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 msgr2=0x556517841120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 0x556517841120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.950+0000 7f0fc3fff700 1 -- 172.21.15.93:0/511970330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb800e040 con 0x55651783de70 2023-07-21T22:30:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517841860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc007bb0 tx=0x7f0fbc007be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b3c38d0d0a13e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00e040 con 0x55651783de70 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0fbc015920 con 0x55651783de70 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565176da490 con 0x55651783de70 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fbc015ae0 con 0x55651783de70 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fd6250b80 10 client.26672 fetch_fsmap learned FSMap version 68 2023-07-21T22:30:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x556517597c00 con 0x55651783de70 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.951+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565176da490 con 0x55651783de70 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f0fbc01e3b0 con 0x55651783de70 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 10 client.26672 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 20 client.26672 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556517851530 0x556517853940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 4 client.26672 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556517779e50 con 0x556517851530 2023-07-21T22:30:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556517851530 0x556517853940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fa2ffd700 10 client.26672 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.952+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556517851530 0x556517853940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35d55c6bbe2b7c3 server_cookie=6ae991966d05ceea in_seq=0 out_seq=0 2023-07-21T22:30:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.957+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/511970330 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556517779e50 con 0x556517851530 2023-07-21T22:30:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.957+0000 7f0fa2ffd700 10 client.26672 handle_command_reply: tid=0 2023-07-21T22:30:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.957+0000 7f0fd6250b80 1 client.26672 shutdown 2023-07-21T22:30:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 msgr2=0x556517841860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517841860 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc007bb0 tx=0x7f0fbc007be0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 shutdown_connections 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556517851530 0x556517853940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517841860 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fa2ffd700 0 client.26672 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651768b6b0 0x556517843e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/511970330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651783da20 0x556517841120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 >> 172.21.15.93:0/511970330 conn(0x556517824ae0 msgr2=0x5565178293e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 shutdown_connections 2023-07-21T22:30:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 1 -- 172.21.15.93:0/511970330 wait complete. 2023-07-21T22:30:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.958+0000 7f0fd6250b80 20 client.26672 trim_cache size 0 max 16384 2023-07-21T22:30:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 Processor -- start 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 -- start start 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783da20 0x5565178469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517829b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651783bdb0 con 0x55651768b6b0 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651782b9d0 con 0x55651783de70 2023-07-21T22:30:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517821ab0 con 0x55651783da20 2023-07-21T22:30:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48432/0 (socket says 172.21.15.93:48432) 2023-07-21T22:30:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fc37fe700 1 -- 172.21.15.93:0/3764098416 learned_addr learned my addr 172.21.15.93:0/3764098416 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.979+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783da20 0x5565178469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 -- 172.21.15.93:0/3764098416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783da20 msgr2=0x5565178469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783da20 0x5565178469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 -- 172.21.15.93:0/3764098416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 msgr2=0x556517829b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517829b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 -- 172.21.15.93:0/3764098416 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565175d51f0 con 0x55651768b6b0 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0fb409ab20 tx=0x7f0fb40cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b86330a80ea749a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0faa7fc700 1 -- 172.21.15.93:0/3764098416 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40ba040 con 0x55651768b6b0 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0faa7fc700 1 -- 172.21.15.93:0/3764098416 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb40b3210 con 0x55651768b6b0 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.980+0000 7f0faa7fc700 1 -- 172.21.15.93:0/3764098416 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40054b0 con 0x55651768b6b0 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 -- 172.21.15.93:0/3764098416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 msgr2=0x55651783ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0fb409ab20 tx=0x7f0fb40cd040 comp rx=0 tx=0).stop 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 -- 172.21.15.93:0/3764098416 shutdown_connections 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783de70 0x556517829b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x55651783ca50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/3764098416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783da20 0x5565178469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 -- 172.21.15.93:0/3764098416 >> 172.21.15.93:0/3764098416 conn(0x556517824ae0 msgr2=0x55651783e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 -- 172.21.15.93:0/3764098416 shutdown_connections 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.981+0000 7f0fd6250b80 1 -- 172.21.15.93:0/3764098416 wait complete. 2023-07-21T22:30:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.982+0000 7f0fd6250b80 1 -- start start 2023-07-21T22:30:47.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.982+0000 7f0fd6250b80 1 Processor -- start 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517844b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x5565178450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517845610 con 0x55651768b6b0 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517845780 con 0x55651783da20 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556517845940 con 0x55651783de70 2023-07-21T22:30:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fd6250b80 10 client.0 _ll_register_callbacks cb 0x7f0fbc009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48438/0 (socket says 172.21.15.93:48438) 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 -- 172.21.15.93:0/180035739 learned_addr learned my addr 172.21.15.93:0/180035739 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc3fff700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x5565178450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517844b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 -- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 msgr2=0x5565178450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x5565178450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 -- 172.21.15.93:0/180035739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 msgr2=0x556517844b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517844b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 -- 172.21.15.93:0/180035739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc4195060 con 0x55651768b6b0 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fc37fe700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4097090 tx=0x7f0fb40a30d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4316b2ec028555 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.983+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40ba030 con 0x55651768b6b0 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7f0fb40054b0 con 0x55651768b6b0 2023-07-21T22:30:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fb8014c60 con 0x55651768b6b0 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb4005670 con 0x55651768b6b0 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 10 client.17298 fetch_fsmap learned FSMap version 68 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5565175ad250 con 0x55651768b6b0 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb8014c60 con 0x55651768b6b0 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f0fb409a080 con 0x55651768b6b0 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:30:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 20 client.17298 populate_metadata read hostname 'smithi093' 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565178a6490 0x5565178a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 4 client.17298 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55651773bff0 con 0x5565178a6490 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.984+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565178a6490 0x5565178a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.985+0000 7f0fa2ffd700 10 client.17298 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:46.985+0000 7f0fc2ffd700 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565178a6490 0x5565178a88a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb55a9763cb2a33 server_cookie=92f6ce2cc6db024d in_seq=0 out_seq=0 2023-07-21T22:30:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fa2ffd700 1 -- 172.21.15.93:0/180035739 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55651773bff0 con 0x5565178a6490 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fa2ffd700 10 client.17298 handle_command_reply: tid=0 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 client.17298 shutdown 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 msgr2=0x556517846860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4097090 tx=0x7f0fb40a30d0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 shutdown_connections 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5565178a6490 0x5565178a88a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55651783de70 0x5565178450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55651768b6b0 0x556517846860 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fa2ffd700 0 client.17298 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 --2- 172.21.15.93:0/180035739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55651783da20 0x556517844b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 >> 172.21.15.93:0/180035739 conn(0x556517824ae0 msgr2=0x55651783ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 shutdown_connections 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 1 -- 172.21.15.93:0/180035739 wait complete. 2023-07-21T22:30:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.206+0000 7f0fd6250b80 20 client.17298 trim_cache size 0 max 16384 2023-07-21T22:30:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:30:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:30:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:30:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:30:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 msgr2=0x7f0fac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 0x7f0fac0a37f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40698b0 tx=0x7f0fb8008040 comp rx=0 tx=0).stop 2023-07-21T22:30:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 msgr2=0x7f0fc4196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009fd0 tx=0x7f0fb400fa70 comp rx=0 tx=0).stop 2023-07-21T22:30:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 shutdown_connections 2023-07-21T22:30:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fc4104ad0 0x7f0fc41a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0fac0a1340 0x7f0fac0a37f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0fc40feac0 0x7f0fc4196d20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 --2- 172.21.15.93:0/2217591097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0fc40ff470 0x7f0fc4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 >> 172.21.15.93:0/2217591097 conn(0x7f0fc4075bd0 msgr2=0x7f0fc4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 shutdown_connections 2023-07-21T22:30:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:30:47.207+0000 7f0fca0cb700 1 -- 172.21.15.93:0/2217591097 wait complete. 2023-07-21T22:30:47.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 53448 36.85 MB/sec warmup 83 sec latency 114.801 ms 2023-07-21T22:30:47.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:47 smithi194 ceph-mon[98288]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2023-07-21T22:30:47.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:47 smithi134 ceph-mon[97873]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2023-07-21T22:30:47.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:47 smithi093 ceph-mon[96579]: pgmap v455: 97 pgs: 97 active+clean; 816 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.0 MiB/s wr, 669 op/s 2023-07-21T22:30:48.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 54238 36.91 MB/sec warmup 84 sec latency 36.639 ms 2023-07-21T22:30:48.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:48 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.592 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:48 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.592 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:48 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:30:48.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:48 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:48 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:48 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:30:48.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:48 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:48 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:48 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:30:49.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 55001 36.96 MB/sec warmup 85 sec latency 58.625 ms 2023-07-21T22:30:49.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:49 smithi194 ceph-mon[98288]: pgmap v456: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 943 op/s 2023-07-21T22:30:49.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:49 smithi134 ceph-mon[97873]: pgmap v456: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 943 op/s 2023-07-21T22:30:49.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:49 smithi093 ceph-mon[96579]: pgmap v456: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.6 MiB/s wr, 943 op/s 2023-07-21T22:30:50.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 55918 36.97 MB/sec warmup 86 sec latency 58.273 ms 2023-07-21T22:30:51.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 56741 37.09 MB/sec warmup 87 sec latency 148.567 ms 2023-07-21T22:30:51.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:51 smithi134 ceph-mon[97873]: pgmap v457: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 625 op/s 2023-07-21T22:30:51.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:51 smithi093 ceph-mon[96579]: pgmap v457: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 625 op/s 2023-07-21T22:30:51.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:51 smithi194 ceph-mon[98288]: pgmap v457: 97 pgs: 97 active+clean; 837 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 625 op/s 2023-07-21T22:30:52.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 57776 37.45 MB/sec warmup 88 sec latency 69.613 ms 2023-07-21T22:30:53.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 58539 37.40 MB/sec warmup 89 sec latency 36.044 ms 2023-07-21T22:30:53.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:53 smithi134 ceph-mon[97873]: pgmap v458: 97 pgs: 97 active+clean; 863 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 854 op/s 2023-07-21T22:30:53.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:53 smithi093 ceph-mon[96579]: pgmap v458: 97 pgs: 97 active+clean; 863 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 854 op/s 2023-07-21T22:30:53.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:53 smithi194 ceph-mon[98288]: pgmap v458: 97 pgs: 97 active+clean; 863 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 854 op/s 2023-07-21T22:30:54.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 59459 37.48 MB/sec warmup 90 sec latency 41.079 ms 2023-07-21T22:30:55.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 60370 37.58 MB/sec warmup 91 sec latency 51.085 ms 2023-07-21T22:30:55.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:55 smithi134 ceph-mon[97873]: pgmap v459: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-07-21T22:30:55.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:55 smithi093 ceph-mon[96579]: pgmap v459: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-07-21T22:30:55.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:55 smithi194 ceph-mon[98288]: pgmap v459: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 748 op/s 2023-07-21T22:30:56.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 61194 37.79 MB/sec warmup 92 sec latency 127.269 ms 2023-07-21T22:30:57.501 INFO:tasks.workunit.client.0.smithi093.stdout: 10 61949 37.77 MB/sec warmup 93 sec latency 54.307 ms 2023-07-21T22:30:57.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:57 smithi134 ceph-mon[97873]: pgmap v460: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 619 op/s 2023-07-21T22:30:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:57 smithi093 ceph-mon[96579]: pgmap v460: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 619 op/s 2023-07-21T22:30:57.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:57 smithi194 ceph-mon[98288]: pgmap v460: 97 pgs: 97 active+clean; 868 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 619 op/s 2023-07-21T22:30:58.500 INFO:tasks.workunit.client.0.smithi093.stdout: 10 62851 37.89 MB/sec warmup 94 sec latency 66.780 ms 2023-07-21T22:30:59.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63447 37.73 MB/sec warmup 95 sec latency 312.894 ms 2023-07-21T22:30:59.634 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:30:59 smithi134 ceph-mon[97873]: pgmap v461: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 891 op/s 2023-07-21T22:30:59.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:30:59 smithi093 ceph-mon[96579]: pgmap v461: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 891 op/s 2023-07-21T22:30:59.844 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:30:59 smithi194 ceph-mon[98288]: pgmap v461: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.4 MiB/s wr, 891 op/s 2023-07-21T22:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.374+0000 7fd838947700 1 -- 172.21.15.93:0/1417437025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 msgr2=0x7fd834104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.374+0000 7fd838947700 1 --2- 172.21.15.93:0/1417437025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834104e20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd824009990 tx=0x7fd824009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 -- 172.21.15.93:0/1417437025 shutdown_connections 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 --2- 172.21.15.93:0/1417437025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 0x7fd83410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 --2- 172.21.15.93:0/1417437025 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834104e20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 --2- 172.21.15.93:0/1417437025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd8340feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 -- 172.21.15.93:0/1417437025 >> 172.21.15.93:0/1417437025 conn(0x7fd8340fa540 msgr2=0x7fd8340fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 -- 172.21.15.93:0/1417437025 shutdown_connections 2023-07-21T22:31:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 -- 172.21.15.93:0/1417437025 wait complete. 2023-07-21T22:31:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 Processor -- start 2023-07-21T22:31:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.375+0000 7fd838947700 1 -- start start 2023-07-21T22:31:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 0x7fd834107240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834107780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834107d50 con 0x7fd834104a50 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834107ec0 con 0x7fd8340ff3f0 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd838947700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834108080 con 0x7fd8340fea40 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35530/0 (socket says 172.21.15.93:35530) 2023-07-21T22:31:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 -- 172.21.15.93:0/1453969085 learned_addr learned my addr 172.21.15.93:0/1453969085 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834107780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 0x7fd834107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 -- 172.21.15.93:0/1453969085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 msgr2=0x7fd834107240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 0x7fd834107240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 -- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 msgr2=0x7fd834107780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834107780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd83259c700 1 -- 172.21.15.93:0/1453969085 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240095f0 con 0x7fd8340fea40 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.376+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd83259c700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd824004740 tx=0x7fd824009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd8237fe700 1 -- 172.21.15.93:0/1453969085 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82401d070 con 0x7fd8340fea40 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd8237fe700 1 -- 172.21.15.93:0/1453969085 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd824004830 con 0x7fd8340fea40 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8341a8be0 con 0x7fd8340fea40 2023-07-21T22:31:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd8237fe700 1 -- 172.21.15.93:0/1453969085 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8240052a0 con 0x7fd8340fea40 2023-07-21T22:31:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.377+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8341a9060 con 0x7fd8340fea40 2023-07-21T22:31:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.380+0000 7fd8237fe700 1 -- 172.21.15.93:0/1453969085 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd824005460 con 0x7fd8340fea40 2023-07-21T22:31:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.380+0000 7fd844accb80 1 Processor -- start 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 -- start start 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 0x5626f8c1fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 0x5626f8c19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8b1fc50 con 0x5626f8c20660 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8b729e0 con 0x5626f8a766b0 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd844accb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8b6be70 con 0x5626f8c20210 2023-07-21T22:31:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd8237fe700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 0x7fd81c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 0x5626f8c1fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48458/0 (socket says 172.21.15.93:48458) 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd831d9b700 1 -- 172.21.15.93:0/2906968113 learned_addr learned my addr 172.21.15.93:0/2906968113 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 0x7fd81c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd83259c700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 0x5626f8c19f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd8237fe700 1 -- 172.21.15.93:0/1453969085 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd8240cdd10 con 0x7fd8340fea40 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 -- 172.21.15.93:0/2906968113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 msgr2=0x5626f8c19f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 0x5626f8c19f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 -- 172.21.15.93:0/2906968113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 msgr2=0x5626f8c1a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd832d9d700 1 -- 172.21.15.93:0/2906968113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f8999d80 con 0x5626f8a766b0 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd83259c700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 0x5626f8c19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.381+0000 7fd831d9b700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.382+0000 7fd832d9d700 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 0x5626f8c1fcd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd82c008980 tx=0x7fd82c008c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9103cc2f5c97b486 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.382+0000 7fd820ff9700 1 -- 172.21.15.93:0/2906968113 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c008420 con 0x5626f8a766b0 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.382+0000 7fd820ff9700 1 -- 172.21.15.93:0/2906968113 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c00f040 con 0x5626f8a766b0 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.382+0000 7fd820ff9700 1 -- 172.21.15.93:0/2906968113 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0237f0 con 0x5626f8a766b0 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 0x7fd81c0a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd82c020000 tx=0x7fd82c01e000 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- 172.21.15.93:0/2906968113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 msgr2=0x5626f8c1fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 0x5626f8c1fcd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd82c008980 tx=0x7fd82c008c90 comp rx=0 tx=0).stop 2023-07-21T22:31:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- 172.21.15.93:0/2906968113 shutdown_connections 2023-07-21T22:31:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20660 0x5626f8c1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8a766b0 0x5626f8c1fcd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- 172.21.15.93:0/2906968113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20210 0x5626f8c19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- 172.21.15.93:0/2906968113 >> 172.21.15.93:0/2906968113 conn(0x5626f8c0fae0 msgr2=0x5626f8c150e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- 172.21.15.93:0/2906968113 shutdown_connections 2023-07-21T22:31:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- 172.21.15.93:0/2906968113 wait complete. 2023-07-21T22:31:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- start start 2023-07-21T22:31:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 Processor -- start 2023-07-21T22:31:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8b0ba60 con 0x5626f8c20210 2023-07-21T22:31:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8b0b760 con 0x5626f8c20660 2023-07-21T22:31:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.383+0000 7fd844accb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f89ef8c0 con 0x5626f8a766b0 2023-07-21T22:31:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd844accb80 10 client.0 _ll_register_callbacks cb 0x5626f8b09700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48466/0 (socket says 172.21.15.93:48466) 2023-07-21T22:31:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 -- 172.21.15.93:0/1313468581 learned_addr learned my addr 172.21.15.93:0/1313468581 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd83259c700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 -- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 msgr2=0x5626f8c264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 -- 172.21.15.93:0/1313468581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 msgr2=0x5626f8c23d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c23d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd831d9b700 1 -- 172.21.15.93:0/1313468581 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c0085b0 con 0x5626f8c20210 2023-07-21T22:31:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.384+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c23d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.385+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd828012740 tx=0x7fd828012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0b79475a63005d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.385+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828017070 con 0x5626f8c20210 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd828004050 con 0x5626f8c20210 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626f8ac5490 con 0x5626f8c20210 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd828004210 con 0x5626f8c20210 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd844accb80 10 client.17310 fetch_fsmap learned FSMap version 68 2023-07-21T22:31:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x5626f8982c00 con 0x5626f8c20210 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.386+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626f8ac5490 con 0x5626f8c20210 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fd82800f570 con 0x5626f8c20210 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 10 client.17310 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 10 client.17310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 20 client.17310 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c33b20 0x5626f8c35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 4 client.17310 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.387+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626f8b64e50 con 0x5626f8c33b20 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.388+0000 7fd83259c700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c33b20 0x5626f8c35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.389+0000 7fd819ffb700 10 client.17310 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.389+0000 7fd83259c700 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c33b20 0x5626f8c35f30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3533109683da12 server_cookie=ad516784c18378c5 in_seq=0 out_seq=0 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.391+0000 7fd819ffb700 1 -- 172.21.15.93:0/1313468581 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626f8b64e50 con 0x5626f8c33b20 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.391+0000 7fd819ffb700 10 client.17310 handle_command_reply: tid=0 2023-07-21T22:31:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.391+0000 7fd844accb80 1 client.17310 shutdown 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.391+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 msgr2=0x5626f8c237b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd828012740 tx=0x7fd828012a50 comp rx=0 tx=0).stop 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 shutdown_connections 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c33b20 0x5626f8c35f30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c23d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd819ffb700 0 client.17310 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 --2- 172.21.15.93:0/1313468581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c237b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 >> 172.21.15.93:0/1313468581 conn(0x5626f8c0fae0 msgr2=0x5626f8c143e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 shutdown_connections 2023-07-21T22:31:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 1 -- 172.21.15.93:0/1313468581 wait complete. 2023-07-21T22:31:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.392+0000 7fd844accb80 20 client.17310 trim_cache size 0 max 16384 2023-07-21T22:31:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.412+0000 7fd844accb80 1 Processor -- start 2023-07-21T22:31:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.412+0000 7fd844accb80 1 -- start start 2023-07-21T22:31:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 0x5626f8c1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20660 0x5626f8c26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c31110 con 0x5626f8a766b0 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c30430 con 0x5626f8c20210 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd844accb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c15f10 con 0x5626f8c20660 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48472/0 (socket says 172.21.15.93:48472) 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 -- 172.21.15.93:0/1778862622 learned_addr learned my addr 172.21.15.93:0/1778862622 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 0x5626f8c1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd832d9d700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20660 0x5626f8c26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 -- 172.21.15.93:0/1778862622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20660 msgr2=0x5626f8c26060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20660 0x5626f8c26060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 -- 172.21.15.93:0/1778862622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 msgr2=0x5626f8c1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 0x5626f8c1f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 -- 172.21.15.93:0/1778862622 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f8c3ab20 con 0x5626f8a766b0 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd831d9b700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 0x5626f8c1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.413+0000 7fd83259c700 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd82409ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9789a62b3dfc2ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.414+0000 7fd8217fa700 1 -- 172.21.15.93:0/1778862622 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82409eec0 con 0x5626f8a766b0 2023-07-21T22:31:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.414+0000 7fd8217fa700 1 -- 172.21.15.93:0/1778862622 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8240a3250 con 0x5626f8a766b0 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.414+0000 7fd8217fa700 1 -- 172.21.15.93:0/1778862622 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8240c99d0 con 0x5626f8a766b0 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 -- 172.21.15.93:0/1778862622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 msgr2=0x5626f8c2ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd82409ad80 comp rx=0 tx=0).stop 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 -- 172.21.15.93:0/1778862622 shutdown_connections 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8c20660 0x5626f8c26060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8a766b0 0x5626f8c2ac30 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 --2- 172.21.15.93:0/1778862622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20210 0x5626f8c1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 -- 172.21.15.93:0/1778862622 >> 172.21.15.93:0/1778862622 conn(0x5626f8c0fae0 msgr2=0x5626f8c1e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.415+0000 7fd844accb80 1 -- 172.21.15.93:0/1778862622 shutdown_connections 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.416+0000 7fd844accb80 1 -- 172.21.15.93:0/1778862622 wait complete. 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 -- start start 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 Processor -- start 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c1c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c1cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c1b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c2af30 con 0x5626f8c20210 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c2b0a0 con 0x5626f8c20660 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f8c2b210 con 0x5626f8a766b0 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd844accb80 10 client.0 _ll_register_callbacks cb 0x7fd828009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c1cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd83259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c1c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd83259c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c1c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35558/0 (socket says 172.21.15.93:35558) 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd83259c700 1 -- 172.21.15.93:0/698690373 learned_addr learned my addr 172.21.15.93:0/698690373 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.417+0000 7fd832d9d700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c1b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 -- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 msgr2=0x5626f8c1c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c1c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 -- 172.21.15.93:0/698690373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 msgr2=0x5626f8c1b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c1b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 -- 172.21.15.93:0/698690373 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626f8982c00 con 0x5626f8c20210 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd831d9b700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c1cd60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd828012da0 tx=0x7fd82800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fba9de4eaebba0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82802a020 con 0x5626f8c20210 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8483+0+0 (secure 0 0 0) 0x7fd828000e40 con 0x5626f8c20210 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd82c00f040 con 0x5626f8c20210 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd828017040 con 0x5626f8c20210 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd844accb80 10 client.17316 fetch_fsmap learned FSMap version 68 2023-07-21T22:31:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.418+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7fd8341a9060 con 0x5626f8c20210 2023-07-21T22:31:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c00f040 con 0x5626f8c20210 2023-07-21T22:31:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fd8280052d0 con 0x5626f8c20210 2023-07-21T22:31:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 10 client.17316 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:31:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 20 client.17316 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c884e0 0x5626f8c8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 4 client.17316 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626f8b26ff0 con 0x5626f8c884e0 2023-07-21T22:31:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.419+0000 7fd83259c700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c884e0 0x5626f8c8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.420+0000 7fd819ffb700 10 client.17316 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.420+0000 7fd83259c700 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c884e0 0x5626f8c8a8f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3365e255fe7d07b server_cookie=1119bac89c04eaa6 in_seq=0 out_seq=0 2023-07-21T22:31:00.502 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63485 37.38 MB/sec warmup 96 sec latency 1111.506 ms 2023-07-21T22:31:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.659+0000 7fd819ffb700 1 -- 172.21.15.93:0/698690373 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626f8b26ff0 con 0x5626f8c884e0 2023-07-21T22:31:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.659+0000 7fd819ffb700 10 client.17316 handle_command_reply: tid=0 2023-07-21T22:31:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.659+0000 7fd844accb80 1 client.17316 shutdown 2023-07-21T22:31:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.660+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 msgr2=0x5626f8c1cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.660+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c1cd60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd828012da0 tx=0x7fd82800fa20 comp rx=0 tx=0).stop 2023-07-21T22:31:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 shutdown_connections 2023-07-21T22:31:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5626f8c884e0 0x5626f8c8a8f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626f8c20660 0x5626f8c1b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5626f8a766b0 0x5626f8c1c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 --2- 172.21.15.93:0/698690373 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626f8c20210 0x5626f8c1cd60 secure :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd828012da0 tx=0x7fd82800fa20 comp rx=0 tx=0).stop 2023-07-21T22:31:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 >> 172.21.15.93:0/698690373 conn(0x5626f8c0fae0 msgr2=0x5626f8c197c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.661+0000 7fd819ffb700 0 client.17316 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.662+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 shutdown_connections 2023-07-21T22:31:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.662+0000 7fd844accb80 1 -- 172.21.15.93:0/698690373 wait complete. 2023-07-21T22:31:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:31:00.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:31:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:31:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:31:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.662+0000 7fd844accb80 20 client.17316 trim_cache size 0 max 16384 2023-07-21T22:31:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.664+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 msgr2=0x7fd81c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.664+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 0x7fd81c0a37f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd82c020000 tx=0x7fd82c01e000 comp rx=0 tx=0).stop 2023-07-21T22:31:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.664+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 msgr2=0x7fd83410b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.664+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd824004740 tx=0x7fd824009580 comp rx=0 tx=0).stop 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 shutdown_connections 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd81c0a1340 0x7fd81c0a37f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd834104a50 0x7fd834107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8340fea40 0x7fd83410b450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 --2- 172.21.15.93:0/1453969085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8340ff3f0 0x7fd834107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 >> 172.21.15.93:0/1453969085 conn(0x7fd8340fa540 msgr2=0x7fd834102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 shutdown_connections 2023-07-21T22:31:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:00.665+0000 7fd838947700 1 -- 172.21.15.93:0/1453969085 wait complete. 2023-07-21T22:31:01.502 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63785 37.22 MB/sec warmup 97 sec latency 500.768 ms 2023-07-21T22:31:01.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:01 smithi134 ceph-mon[97873]: pgmap v462: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 559 op/s 2023-07-21T22:31:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:01 smithi093 ceph-mon[96579]: pgmap v462: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 559 op/s 2023-07-21T22:31:01.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:01 smithi194 ceph-mon[98288]: pgmap v462: 97 pgs: 97 active+clean; 901 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 559 op/s 2023-07-21T22:31:02.502 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63785 36.84 MB/sec warmup 98 sec latency 1500.853 ms 2023-07-21T22:31:03.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63785 36.47 MB/sec warmup 99 sec latency 2500.953 ms 2023-07-21T22:31:03.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:03 smithi134 ceph-mon[97873]: pgmap v463: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 965 op/s 2023-07-21T22:31:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:03 smithi093 ceph-mon[96579]: pgmap v463: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 965 op/s 2023-07-21T22:31:03.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:03 smithi194 ceph-mon[98288]: pgmap v463: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 16 MiB/s wr, 965 op/s 2023-07-21T22:31:04.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63890 36.15 MB/sec warmup 100 sec latency 2731.679 ms 2023-07-21T22:31:05.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63890 35.80 MB/sec warmup 101 sec latency 1647.150 ms 2023-07-21T22:31:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:05 smithi093 ceph-mon[96579]: pgmap v464: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2023-07-21T22:31:05.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:05 smithi194 ceph-mon[98288]: pgmap v464: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2023-07-21T22:31:05.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:05 smithi134 ceph-mon[97873]: pgmap v464: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 23 MiB/s wr, 1.13k op/s 2023-07-21T22:31:06.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 63890 35.45 MB/sec warmup 102 sec latency 2647.217 ms 2023-07-21T22:31:07.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 64520 35.46 MB/sec warmup 103 sec latency 3012.699 ms 2023-07-21T22:31:07.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:07 smithi093 ceph-mon[96579]: pgmap v465: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-07-21T22:31:07.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:07 smithi194 ceph-mon[98288]: pgmap v465: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-07-21T22:31:07.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:07 smithi134 ceph-mon[97873]: pgmap v465: 97 pgs: 97 active+clean; 897 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 1.07k op/s 2023-07-21T22:31:08.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 64959 35.31 MB/sec warmup 104 sec latency 97.646 ms 2023-07-21T22:31:09.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 65815 35.49 MB/sec warmup 105 sec latency 59.806 ms 2023-07-21T22:31:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:09 smithi093 ceph-mon[96579]: pgmap v466: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 23 MiB/s wr, 1.12k op/s 2023-07-21T22:31:09.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:09 smithi194 ceph-mon[98288]: pgmap v466: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 23 MiB/s wr, 1.12k op/s 2023-07-21T22:31:09.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:09 smithi134 ceph-mon[97873]: pgmap v466: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 23 MiB/s wr, 1.12k op/s 2023-07-21T22:31:10.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 66268 35.29 MB/sec warmup 106 sec latency 106.466 ms 2023-07-21T22:31:10.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:10 smithi093 ceph-mon[96579]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:31:10.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:10 smithi194 ceph-mon[98288]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:31:10.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:10 smithi134 ceph-mon[97873]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:31:11.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 67114 35.39 MB/sec warmup 107 sec latency 41.798 ms 2023-07-21T22:31:11.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:11 smithi093 ceph-mon[96579]: pgmap v468: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-07-21T22:31:11.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:11 smithi194 ceph-mon[98288]: pgmap v468: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-07-21T22:31:11.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:11 smithi134 ceph-mon[97873]: pgmap v468: 97 pgs: 97 active+clean; 902 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 24 MiB/s wr, 1.02k op/s 2023-07-21T22:31:12.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 67964 35.55 MB/sec warmup 108 sec latency 46.010 ms 2023-07-21T22:31:13.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 68581 35.48 MB/sec warmup 109 sec latency 114.778 ms 2023-07-21T22:31:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:13 smithi093 ceph-mon[96579]: pgmap v469: 97 pgs: 97 active+clean; 914 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 788 op/s 2023-07-21T22:31:13.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:13 smithi194 ceph-mon[98288]: pgmap v469: 97 pgs: 97 active+clean; 914 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 788 op/s 2023-07-21T22:31:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 -- 172.21.15.93:0/3525598983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0fe980 msgr2=0x7f5e0c0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 --2- 172.21.15.93:0/3525598983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c0fedf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009990 tx=0x7f5e00009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 -- 172.21.15.93:0/3525598983 shutdown_connections 2023-07-21T22:31:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 --2- 172.21.15.93:0/3525598983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 --2- 172.21.15.93:0/3525598983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 0x7f5e0c104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 --2- 172.21.15.93:0/3525598983 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c0fedf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.816+0000 7f5e10adb700 1 -- 172.21.15.93:0/3525598983 >> 172.21.15.93:0/3525598983 conn(0x7f5e0c074680 msgr2=0x7f5e0c074a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.817+0000 7f5e10adb700 1 -- 172.21.15.93:0/3525598983 shutdown_connections 2023-07-21T22:31:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.817+0000 7f5e10adb700 1 -- 172.21.15.93:0/3525598983 wait complete. 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 Processor -- start 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 -- start start 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c106c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 0x7f5e0c1a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c107a40 con 0x7f5e0c0ff330 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1a8560 con 0x7f5e0c0fe980 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e10adb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1a86a0 con 0x7f5e0c104990 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e09d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e09d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48494/0 (socket says 172.21.15.93:48494) 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e09d9b700 1 -- 172.21.15.93:0/2461266745 learned_addr learned my addr 172.21.15.93:0/2461266745 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.818+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 0x7f5e0c1a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 -- 172.21.15.93:0/2461266745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 msgr2=0x7f5e0c1a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 0x7f5e0c1a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 -- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 msgr2=0x7f5e0c107170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 -- 172.21.15.93:0/2461266745 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000095f0 con 0x7f5e0c0fe980 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c106c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5dfb7fe700 1 -- 172.21.15.93:0/2461266745 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc004220 con 0x7f5e0c0fe980 2023-07-21T22:31:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5dfb7fe700 1 -- 172.21.15.93:0/2461266745 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc004380 con 0x7f5e0c0fe980 2023-07-21T22:31:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5dfb7fe700 1 -- 172.21.15.93:0/2461266745 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc003c50 con 0x7f5e0c0fe980 2023-07-21T22:31:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c1a8980 con 0x7f5e0c0fe980 2023-07-21T22:31:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.819+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c1a8e40 con 0x7f5e0c0fe980 2023-07-21T22:31:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.822+0000 7f5dfb7fe700 1 -- 172.21.15.93:0/2461266745 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5dfc00c7c0 con 0x7f5e0c0fe980 2023-07-21T22:31:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.823+0000 7f5dfb7fe700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 0x7f5df40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.823+0000 7f5dfb7fe700 1 -- 172.21.15.93:0/2461266745 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5dfc014070 con 0x7f5e0c0fe980 2023-07-21T22:31:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.823+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 0x7f5df40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.824+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 0x7f5df40a37f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c108420 tx=0x7f5e00005960 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.824+0000 7f5e1cc60b80 1 Processor -- start 2023-07-21T22:31:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.824+0000 7f5e1cc60b80 1 -- start start 2023-07-21T22:31:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b8f1c50 con 0x55765b8486b0 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b9449e0 con 0x55765b9f2210 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b93de70 con 0x55765b9f2660 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48500/0 (socket says 172.21.15.93:48500) 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 -- 172.21.15.93:0/111748344 learned_addr learned my addr 172.21.15.93:0/111748344 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 -- 172.21.15.93:0/111748344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 msgr2=0x55765b9ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 -- 172.21.15.93:0/111748344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 msgr2=0x55765b9ebf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9ebf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 -- 172.21.15.93:0/111748344 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765b76bd80 con 0x55765b8486b0 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9ebf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.825+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc096450 tx=0x7f5dfc099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504c19c1a72ecf65 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.826+0000 7f5df8ff9700 1 -- 172.21.15.93:0/111748344 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc099f50 con 0x55765b8486b0 2023-07-21T22:31:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.826+0000 7f5df8ff9700 1 -- 172.21.15.93:0/111748344 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc09df80 con 0x55765b8486b0 2023-07-21T22:31:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.826+0000 7f5df8ff9700 1 -- 172.21.15.93:0/111748344 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09a410 con 0x55765b8486b0 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/111748344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 msgr2=0x55765b9f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc096450 tx=0x7f5dfc099930 comp rx=0 tx=0).stop 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/111748344 shutdown_connections 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f1cd0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/111748344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9ebf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/111748344 >> 172.21.15.93:0/111748344 conn(0x55765b9e1ae0 msgr2=0x55765b9e70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/111748344 shutdown_connections 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/111748344 wait complete. 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 -- start start 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.827+0000 7f5e1cc60b80 1 Processor -- start 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b8dda60 con 0x55765b8486b0 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b8dd760 con 0x55765b9f2210 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b7c18c0 con 0x55765b9f2660 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e1cc60b80 10 client.0 _ll_register_callbacks cb 0x55765b8db700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e0ad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e0ad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35588/0 (socket says 172.21.15.93:35588) 2023-07-21T22:31:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/1540338666 learned_addr learned my addr 172.21.15.93:0/1540338666 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.828+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/1540338666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 msgr2=0x55765b9f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/1540338666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 msgr2=0x55765b9f5870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f5870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/1540338666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c1a8e40 con 0x55765b9f2660 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e04011ce0 tx=0x7f5e0400f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a14fb7d26c40250 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5df1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:31:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400e040 con 0x55765b9f2660 2023-07-21T22:31:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5e04011e80 con 0x55765b9f2660 2023-07-21T22:31:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.829+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55765b897490 con 0x55765b9f2660 2023-07-21T22:31:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.832+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e0400fe30 con 0x55765b9f2660 2023-07-21T22:31:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.832+0000 7f5e1cc60b80 10 client.26626 fetch_fsmap learned FSMap version 68 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.832+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55765b754c00 con 0x55765b9f2660 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.832+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55765b897490 con 0x55765b9f2660 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f5e0401b030 con 0x55765b9f2660 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 10 client.26626 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 10 client.26626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 20 client.26626 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba27c60 0x55765ba2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 4 client.26626 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55765b936e50 con 0x55765ba27c60 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba27c60 0x55765ba2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5df1ffb700 10 client.26626 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.833+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba27c60 0x55765ba2a070 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93dfd40a485f0ef server_cookie=bae93652de38f9e8 in_seq=0 out_seq=0 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1540338666 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55765b936e50 con 0x55765ba27c60 2023-07-21T22:31:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5df1ffb700 10 client.26626 handle_command_reply: tid=0 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 client.26626 shutdown 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 msgr2=0x55765b9f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e04011ce0 tx=0x7f5e0400f490 comp rx=0 tx=0).stop 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 shutdown_connections 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba27c60 0x55765ba2a070 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2660 0x55765b9f62f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5df1ffb700 0 client.26626 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765b9f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1540338666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 >> 172.21.15.93:0/1540338666 conn(0x55765b9e1ae0 msgr2=0x55765b9e63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 shutdown_connections 2023-07-21T22:31:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.834+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1540338666 wait complete. 2023-07-21T22:31:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.835+0000 7f5e1cc60b80 20 client.26626 trim_cache size 0 max 16384 2023-07-21T22:31:13.886 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:13 smithi134 ceph-mon[97873]: pgmap v469: 97 pgs: 97 active+clean; 914 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 788 op/s 2023-07-21T22:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.856+0000 7f5e1cc60b80 1 Processor -- start 2023-07-21T22:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.856+0000 7f5e1cc60b80 1 -- start start 2023-07-21T22:31:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b8486b0 0x55765ba0dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b9f19a0 con 0x55765b9f2660 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b9e3520 con 0x55765b9f2210 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765ba25200 con 0x55765b8486b0 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48514/0 (socket says 172.21.15.93:48514) 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/496495638 learned_addr learned my addr 172.21.15.93:0/496495638 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b8486b0 0x55765ba0dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/496495638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b8486b0 msgr2=0x55765ba0dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b8486b0 0x55765ba0dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/496495638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 msgr2=0x55765b9f6120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:31:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.857+0000 7f5e0ad9d700 1 -- 172.21.15.93:0/496495638 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765b7921f0 con 0x55765b9f2660 2023-07-21T22:31:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.860+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e04025790 tx=0x7f5e04006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcee560f13256b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.860+0000 7f5df97fa700 1 -- 172.21.15.93:0/496495638 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400e040 con 0x55765b9f2660 2023-07-21T22:31:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.860+0000 7f5df97fa700 1 -- 172.21.15.93:0/496495638 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0400f040 con 0x55765b9f2660 2023-07-21T22:31:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.860+0000 7f5df97fa700 1 -- 172.21.15.93:0/496495638 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e040050e0 con 0x55765b9f2660 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/496495638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 msgr2=0x55765b9f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e04025790 tx=0x7f5e04006000 comp rx=0 tx=0).stop 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/496495638 shutdown_connections 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b9f2660 0x55765b9f4680 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b8486b0 0x55765ba0dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/496495638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2210 0x55765b9f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/496495638 >> 172.21.15.93:0/496495638 conn(0x55765b9e1ae0 msgr2=0x55765b9f0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/496495638 shutdown_connections 2023-07-21T22:31:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.861+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/496495638 wait complete. 2023-07-21T22:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 -- start start 2023-07-21T22:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 Processor -- start 2023-07-21T22:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2210 0x55765b9edba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 0x55765b9ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b9ee7d0 con 0x55765b8486b0 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b9ee940 con 0x55765b9f2660 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765ba7bb20 con 0x55765b9f2210 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.862+0000 7f5e1cc60b80 10 client.0 _ll_register_callbacks cb 0x55765b8db700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48516/0 (socket says 172.21.15.93:48516) 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 -- 172.21.15.93:0/1113423762 learned_addr learned my addr 172.21.15.93:0/1113423762 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2210 0x55765b9edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 0x55765b9ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 -- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2210 msgr2=0x55765b9edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2210 0x55765b9edba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 -- 172.21.15.93:0/1113423762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 msgr2=0x55765b9ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 0x55765b9ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 -- 172.21.15.93:0/1113423762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765b754c00 con 0x55765b8486b0 2023-07-21T22:31:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0ad9d700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 0x55765b9ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e0a59c700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc099ee0 tx=0x7f5dfc0a24a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c79a8079c28202e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5df1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0982f0 con 0x55765b8486b0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5dfc0b81e0 con 0x55765b8486b0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dfc09df80 con 0x55765b8486b0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc0b83a0 con 0x55765b8486b0 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e1cc60b80 10 client.17331 fetch_fsmap learned FSMap version 68 2023-07-21T22:31:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.863+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55765b76a250 con 0x55765b8486b0 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc09df80 con 0x55765b8486b0 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 68) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f5dfc0cad20 con 0x55765b8486b0 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 68 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 20 client.17331 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba7cdd0 0x55765ba7f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 4 client.17331 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55765b8f8ff0 con 0x55765ba7cdd0 2023-07-21T22:31:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.864+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba7cdd0 0x55765ba7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.865+0000 7f5df1ffb700 10 client.17331 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:13.865+0000 7f5e09d9b700 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba7cdd0 0x55765ba7f1e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a375466445b40d server_cookie=259da25209a18b96 in_seq=0 out_seq=0 2023-07-21T22:31:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5df1ffb700 1 -- 172.21.15.93:0/1113423762 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55765b8f8ff0 con 0x55765ba7cdd0 2023-07-21T22:31:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5df1ffb700 10 client.17331 handle_command_reply: tid=0 2023-07-21T22:31:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5e1cc60b80 1 client.17331 shutdown 2023-07-21T22:31:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 msgr2=0x55765ba0ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc099ee0 tx=0x7f5dfc0a24a0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.019+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 shutdown_connections 2023-07-21T22:31:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55765ba7cdd0 0x55765ba7f1e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55765b9f2660 0x55765b9ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55765b8486b0 0x55765ba0ddb0 secure :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc099ee0 tx=0x7f5dfc0a24a0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 --2- 172.21.15.93:0/1113423762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55765b9f2210 0x55765b9edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 >> 172.21.15.93:0/1113423762 conn(0x55765b9e1ae0 msgr2=0x55765b9f4680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5df1ffb700 0 client.17331 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 shutdown_connections 2023-07-21T22:31:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 1 -- 172.21.15.93:0/1113423762 wait complete. 2023-07-21T22:31:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.020+0000 7f5e1cc60b80 20 client.17331 trim_cache size 0 max 16384 2023-07-21T22:31:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:31:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:31:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:31:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.021+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 msgr2=0x7f5df40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.021+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 0x7f5df40a37f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c108420 tx=0x7f5e00005960 comp rx=0 tx=0).stop 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.021+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 msgr2=0x7f5e0c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.021+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c106c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ecd0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 shutdown_connections 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e0c104990 0x7f5e0c1a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0c0fe980 0x7f5e0c106c30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df40a1340 0x7f5df40a37f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 --2- 172.21.15.93:0/2461266745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e0c0ff330 0x7f5e0c107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.023+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 >> 172.21.15.93:0/2461266745 conn(0x7f5e0c074680 msgr2=0x7f5e0c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.025+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 shutdown_connections 2023-07-21T22:31:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:14.025+0000 7f5e10adb700 1 -- 172.21.15.93:0/2461266745 wait complete. 2023-07-21T22:31:14.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 69297 35.56 MB/sec warmup 110 sec latency 77.302 ms 2023-07-21T22:31:15.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 70123 35.58 MB/sec warmup 111 sec latency 48.988 ms 2023-07-21T22:31:15.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:15 smithi093 ceph-mon[96579]: pgmap v470: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:15.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:15 smithi194 ceph-mon[98288]: pgmap v470: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:15.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:15 smithi134 ceph-mon[97873]: pgmap v470: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:16.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 71124 35.81 MB/sec warmup 112 sec latency 49.551 ms 2023-07-21T22:31:17.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 71958 35.86 MB/sec warmup 113 sec latency 61.991 ms 2023-07-21T22:31:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:17 smithi093 ceph-mon[96579]: pgmap v471: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:17.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:17 smithi093 ceph-mon[96579]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:31:17.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:17 smithi194 ceph-mon[98288]: pgmap v471: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:17.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:17 smithi194 ceph-mon[98288]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:31:17.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:17 smithi134 ceph-mon[97873]: pgmap v471: 97 pgs: 97 active+clean; 909 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 438 op/s 2023-07-21T22:31:17.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:17 smithi134 ceph-mon[97873]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:31:18.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 72767 35.96 MB/sec warmup 114 sec latency 123.613 ms 2023-07-21T22:31:19.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 73631 36.00 MB/sec warmup 115 sec latency 44.253 ms 2023-07-21T22:31:19.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:19 smithi093 ceph-mon[96579]: pgmap v473: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 853 op/s 2023-07-21T22:31:19.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:19 smithi194 ceph-mon[98288]: pgmap v473: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 853 op/s 2023-07-21T22:31:19.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:19 smithi134 ceph-mon[97873]: pgmap v473: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.5 MiB/s wr, 853 op/s 2023-07-21T22:31:20.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 74612 36.21 MB/sec warmup 116 sec latency 43.522 ms 2023-07-21T22:31:20.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:20 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:20.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:20 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:20.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:20 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:21.503 INFO:tasks.workunit.client.0.smithi093.stdout: 10 75454 36.21 MB/sec warmup 117 sec latency 49.175 ms 2023-07-21T22:31:21.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:21 smithi093 ceph-mon[96579]: pgmap v474: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 756 op/s 2023-07-21T22:31:21.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:21 smithi194 ceph-mon[98288]: pgmap v474: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 756 op/s 2023-07-21T22:31:21.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:21 smithi134 ceph-mon[97873]: pgmap v474: 97 pgs: 97 active+clean; 930 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.5 MiB/s wr, 756 op/s 2023-07-21T22:31:22.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 76398 36.32 MB/sec warmup 118 sec latency 41.186 ms 2023-07-21T22:31:23.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 77284 36.46 MB/sec warmup 119 sec latency 53.730 ms 2023-07-21T22:31:23.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:23 smithi093 ceph-mon[96579]: pgmap v475: 97 pgs: 97 active+clean; 960 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.3 MiB/s wr, 760 op/s 2023-07-21T22:31:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:23 smithi194 ceph-mon[98288]: pgmap v475: 97 pgs: 97 active+clean; 960 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.3 MiB/s wr, 760 op/s 2023-07-21T22:31:23.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:23 smithi134 ceph-mon[97873]: pgmap v475: 97 pgs: 97 active+clean; 960 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.3 MiB/s wr, 760 op/s 2023-07-21T22:31:25.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 78871 35.79 MB/sec execute 1 sec latency 45.319 ms 2023-07-21T22:31:25.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:25 smithi093 ceph-mon[96579]: pgmap v476: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:25.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:25 smithi194 ceph-mon[98288]: pgmap v476: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:25.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:25 smithi134 ceph-mon[97873]: pgmap v476: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:26.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 79773 41.83 MB/sec execute 2 sec latency 46.716 ms 2023-07-21T22:31:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 -- 172.21.15.93:0/1293946317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f80ff470 msgr2=0x7fb2f810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 --2- 172.21.15.93:0/1293946317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f80ff470 0x7fb2f810b900 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b0a0 tx=0x7fb2f000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 -- 172.21.15.93:0/1293946317 shutdown_connections 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 --2- 172.21.15.93:0/1293946317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f80ff470 0x7fb2f810b900 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 --2- 172.21.15.93:0/1293946317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f8104ad0 0x7fb2f8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 --2- 172.21.15.93:0/1293946317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f80feac0 0x7fb2f80fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.162+0000 7fb3000f1700 1 -- 172.21.15.93:0/1293946317 >> 172.21.15.93:0/1293946317 conn(0x7fb2f8075bd0 msgr2=0x7fb2f8075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.163+0000 7fb3000f1700 1 -- 172.21.15.93:0/1293946317 shutdown_connections 2023-07-21T22:31:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.163+0000 7fb3000f1700 1 -- 172.21.15.93:0/1293946317 wait complete. 2023-07-21T22:31:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.163+0000 7fb3000f1700 1 Processor -- start 2023-07-21T22:31:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.163+0000 7fb3000f1700 1 -- start start 2023-07-21T22:31:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f80feac0 0x7fb2f806a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 0x7fb2f806a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81accb0 con 0x7fb2f8104ad0 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81acdf0 con 0x7fb2f80ff470 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb3000f1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81acf60 con 0x7fb2f80feac0 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 0x7fb2f806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 0x7fb2f806a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59716/0 (socket says 172.21.15.93:59716) 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48556/0 (socket says 172.21.15.93:48556) 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fd68c700 1 -- 172.21.15.93:0/437675168 learned_addr learned my addr 172.21.15.93:0/437675168 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 -- 172.21.15.93:0/437675168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f80feac0 msgr2=0x7fb2f806a000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f80feac0 0x7fb2f806a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 -- 172.21.15.93:0/437675168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 msgr2=0x7fb2f806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 0x7fb2f806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.164+0000 7fb2fe68e700 1 -- 172.21.15.93:0/437675168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009d00 con 0x7fb2f8104ad0 2023-07-21T22:31:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.165+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b0a0 tx=0x7fb2f0007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.166+0000 7fb2eaffd700 1 -- 172.21.15.93:0/437675168 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000e040 con 0x7fb2f8104ad0 2023-07-21T22:31:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.166+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f81ad1e0 con 0x7fb2f8104ad0 2023-07-21T22:31:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.166+0000 7fb2eaffd700 1 -- 172.21.15.93:0/437675168 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f0004c50 con 0x7fb2f8104ad0 2023-07-21T22:31:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.166+0000 7fb2eaffd700 1 -- 172.21.15.93:0/437675168 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00052a0 con 0x7fb2f8104ad0 2023-07-21T22:31:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.166+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f81ad660 con 0x7fb2f8104ad0 2023-07-21T22:31:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.169+0000 7fb2eaffd700 1 -- 172.21.15.93:0/437675168 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb2f001d470 con 0x7fb2f8104ad0 2023-07-21T22:31:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.169+0000 7fb2eaffd700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 0x7fb2e40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.170+0000 7fb2eaffd700 1 -- 172.21.15.93:0/437675168 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb2f00ce030 con 0x7fb2f8104ad0 2023-07-21T22:31:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.171+0000 7fb30c276b80 1 Processor -- start 2023-07-21T22:31:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.171+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 0x7fb2e40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.171+0000 7fb30c276b80 1 -- start start 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 0x7fb2e40a38f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80ffc00 tx=0x7fb2ec006cb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbc44c50 con 0x557cfbd4da20 2023-07-21T22:31:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbc979e0 con 0x557cfbd4de70 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb30c276b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbc90e70 con 0x557cfbb9b6b0 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48562/0 (socket says 172.21.15.93:48562) 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 -- 172.21.15.93:0/3482618421 learned_addr learned my addr 172.21.15.93:0/3482618421 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 -- 172.21.15.93:0/3482618421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 msgr2=0x557cfbd4d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd4d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 -- 172.21.15.93:0/3482618421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 msgr2=0x557cfbd47cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd47cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 -- 172.21.15.93:0/3482618421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cfbabed80 con 0x557cfbd4da20 2023-07-21T22:31:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd47cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.172+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009d90 tx=0x7fb2f4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd40b218cfd9158b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.173+0000 7fb2e3fff700 1 -- 172.21.15.93:0/3482618421 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4004220 con 0x557cfbd4da20 2023-07-21T22:31:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.173+0000 7fb2e3fff700 1 -- 172.21.15.93:0/3482618421 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f4004cf0 con 0x557cfbd4da20 2023-07-21T22:31:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.173+0000 7fb2e3fff700 1 -- 172.21.15.93:0/3482618421 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f40055c0 con 0x557cfbd4da20 2023-07-21T22:31:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.174+0000 7fb30c276b80 1 -- 172.21.15.93:0/3482618421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 msgr2=0x557cfbd47750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.174+0000 7fb30c276b80 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009d90 tx=0x7fb2f4011720 comp rx=0 tx=0).stop 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 -- 172.21.15.93:0/3482618421 shutdown_connections 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd47cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd4d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 --2- 172.21.15.93:0/3482618421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd47750 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 -- 172.21.15.93:0/3482618421 >> 172.21.15.93:0/3482618421 conn(0x557cfbd34ae0 msgr2=0x557cfbd3a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.175+0000 7fb30c276b80 1 -- 172.21.15.93:0/3482618421 shutdown_connections 2023-07-21T22:31:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.177+0000 7fb30c276b80 1 -- 172.21.15.93:0/3482618421 wait complete. 2023-07-21T22:31:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.177+0000 7fb30c276b80 1 -- start start 2023-07-21T22:31:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 Processor -- start 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbc30a60 con 0x557cfbd4da20 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbc30760 con 0x557cfbd4de70 2023-07-21T22:31:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbb148c0 con 0x557cfbb9b6b0 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb30c276b80 10 client.0 _ll_register_callbacks cb 0x557cfbc2e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fde8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fde8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35646/0 (socket says 172.21.15.93:35646) 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fde8d700 1 -- 172.21.15.93:0/550746431 learned_addr learned my addr 172.21.15.93:0/550746431 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48570/0 (socket says 172.21.15.93:48570) 2023-07-21T22:31:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.178+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 -- 172.21.15.93:0/550746431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 msgr2=0x557cfbd515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 -- 172.21.15.93:0/550746431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 msgr2=0x557cfbd50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 -- 172.21.15.93:0/550746431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f81ad660 con 0x557cfbb9b6b0 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec012660 tx=0x7fb2ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e218b1e5fe54010 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.179+0000 7fb2e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec01f5f0 con 0x557cfbb9b6b0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb2ec01f750 con 0x557cfbb9b6b0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cfbbea490 con 0x557cfbb9b6b0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2ec01f910 con 0x557cfbb9b6b0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 10 client.26641 fetch_fsmap learned FSMap version 69 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7fb2f40099f0 con 0x557cfbb9b6b0 2023-07-21T22:31:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2ec02c020 con 0x557cfbb9b6b0 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 69) v1 ==== 7530+0+0 (secure 0 0 0) 0x7fb2ec027b50 con 0x557cfbb9b6b0 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 10 client.26641 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 10 client.26641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 20 client.26641 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbd613c0 0x557cfbd637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 4 client.26641 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.180+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cfbc89e50 con 0x557cfbd613c0 2023-07-21T22:31:27.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.184+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbd613c0 0x557cfbd637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.185+0000 7fb2e17fa700 10 client.26641 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.185+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbd613c0 0x557cfbd637d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3456221f345cb497 server_cookie=116f9042a1120e42 in_seq=0 out_seq=0 2023-07-21T22:31:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.188+0000 7fb2e17fa700 1 -- 172.21.15.93:0/550746431 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557cfbc89e50 con 0x557cfbd613c0 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb2e17fa700 10 client.26641 handle_command_reply: tid=0 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 client.26641 shutdown 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 msgr2=0x557cfbd53d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec012660 tx=0x7fb2ec004750 comp rx=0 tx=0).stop 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 shutdown_connections 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbd613c0 0x557cfbd637d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd53d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 --2- 172.21.15.93:0/550746431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 >> 172.21.15.93:0/550746431 conn(0x557cfbd34ae0 msgr2=0x557cfbd393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.189+0000 7fb2e17fa700 0 client.26641 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.192+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 shutdown_connections 2023-07-21T22:31:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.192+0000 7fb30c276b80 1 -- 172.21.15.93:0/550746431 wait complete. 2023-07-21T22:31:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.192+0000 7fb30c276b80 20 client.26641 trim_cache size 0 max 16384 2023-07-21T22:31:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 Processor -- start 2023-07-21T22:31:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 -- start start 2023-07-21T22:31:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd39b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd5e8e0 con 0x557cfbd4da20 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd31b20 con 0x557cfbd4de70 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb30c276b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd36c00 con 0x557cfbb9b6b0 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48576/0 (socket says 172.21.15.93:48576) 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb2fd68c700 1 -- 172.21.15.93:0/4005153918 learned_addr learned my addr 172.21.15.93:0/4005153918 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.216+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 -- 172.21.15.93:0/4005153918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 msgr2=0x557cfbd58360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 -- 172.21.15.93:0/4005153918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 msgr2=0x557cfbd39b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd39b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 -- 172.21.15.93:0/4005153918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cfbd68400 con 0x557cfbd4da20 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb2f401b040 tx=0x7fb2f4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e292fb52351b563f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2e8ff9700 1 -- 172.21.15.93:0/4005153918 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f402b040 con 0x557cfbd4da20 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2e8ff9700 1 -- 172.21.15.93:0/4005153918 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f4004a60 con 0x557cfbd4da20 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.217+0000 7fb2e8ff9700 1 -- 172.21.15.93:0/4005153918 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f402ca50 con 0x557cfbd4da20 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 -- 172.21.15.93:0/4005153918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 msgr2=0x557cfbd53890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb2f401b040 tx=0x7fb2f4011720 comp rx=0 tx=0).stop 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 -- 172.21.15.93:0/4005153918 shutdown_connections 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4de70 0x557cfbd39b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 --2- 172.21.15.93:0/4005153918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4da20 0x557cfbd53890 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 -- 172.21.15.93:0/4005153918 >> 172.21.15.93:0/4005153918 conn(0x557cfbd34ae0 msgr2=0x557cfbd4b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 -- 172.21.15.93:0/4005153918 shutdown_connections 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.218+0000 7fb30c276b80 1 -- 172.21.15.93:0/4005153918 wait complete. 2023-07-21T22:31:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.219+0000 7fb30c276b80 1 -- start start 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.219+0000 7fb30c276b80 1 Processor -- start 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 0x557cfbd48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd49b70 con 0x557cfbd4de70 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd49ce0 con 0x557cfbd4da20 2023-07-21T22:31:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cfbd49ea0 con 0x557cfbb9b6b0 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb30c276b80 10 client.0 _ll_register_callbacks cb 0x7fb2ec013b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fde8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fd68c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 0x557cfbd48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48584/0 (socket says 172.21.15.93:48584) 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fde8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35660/0 (socket says 172.21.15.93:35660) 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 -- 172.21.15.93:0/728922849 learned_addr learned my addr 172.21.15.93:0/728922849 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 -- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 msgr2=0x557cfbd58560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 -- 172.21.15.93:0/728922849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 msgr2=0x557cfbd48f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 0x557cfbd48f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 -- 172.21.15.93:0/728922849 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cfbabed80 con 0x557cfbd4de70 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fd68c700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 0x557cfbd48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2fe68e700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb2f00cbd80 tx=0x7fb2f009abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c272b060e58dd0a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.220+0000 7fb2e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f009f1c0 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb2f00a3550 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2f4004cf0 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f0004dc0 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 10 client.17349 fetch_fsmap learned FSMap version 69 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x557cfbaa7c00 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f4004cf0 con 0x557cfbd4de70 2023-07-21T22:31:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 69) v1 ==== 7530+0+0 (secure 0 0 0) 0x7fb2f00a30c0 con 0x557cfbd4de70 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 10 client.17349 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 20 client.17349 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbdb57e0 0x557cfbdb7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 4 client.17349 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.221+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557cfbc4bff0 con 0x557cfbdb57e0 2023-07-21T22:31:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.224+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbdb57e0 0x557cfbdb7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.225+0000 7fb2e17fa700 10 client.17349 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.225+0000 7fb2fde8d700 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbdb57e0 0x557cfbdb7bf0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fefbb8fe84850a1 server_cookie=ce4315d7f396c67 in_seq=0 out_seq=0 2023-07-21T22:31:27.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 80515 41.64 MB/sec execute 3 sec latency 163.375 ms 2023-07-21T22:31:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.459+0000 7fb2e17fa700 1 -- 172.21.15.93:0/728922849 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fb2ec02c040 con 0x557cfbdb57e0 2023-07-21T22:31:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.459+0000 7fb2e17fa700 10 client.17349 handle_command_reply: tid=0 2023-07-21T22:31:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.460+0000 7fb30c276b80 1 client.17349 shutdown 2023-07-21T22:31:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.460+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 msgr2=0x557cfbd49480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.460+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb2f00cbd80 tx=0x7fb2f009abc0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 shutdown_connections 2023-07-21T22:31:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557cfbdb57e0 0x557cfbdb7bf0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557cfbd4de70 0x557cfbd49480 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb2e17fa700 0 client.17349 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557cfbb9b6b0 0x557cfbd58560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 --2- 172.21.15.93:0/728922849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557cfbd4da20 0x557cfbd48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 >> 172.21.15.93:0/728922849 conn(0x557cfbd34ae0 msgr2=0x557cfbd4e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 shutdown_connections 2023-07-21T22:31:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 1 -- 172.21.15.93:0/728922849 wait complete. 2023-07-21T22:31:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb30c276b80 20 client.17349 trim_cache size 0 max 16384 2023-07-21T22:31:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:31:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:31:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:31:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 msgr2=0x7fb2e40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 0x7fb2e40a38f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80ffc00 tx=0x7fb2ec006cb0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 msgr2=0x7fb2f806aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.461+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b0a0 tx=0x7fb2f0007990 comp rx=0 tx=0).stop 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 shutdown_connections 2023-07-21T22:31:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb2e40a1440 0x7fb2e40a38f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2f8104ad0 0x7fb2f806aa80 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2f80feac0 0x7fb2f806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 --2- 172.21.15.93:0/437675168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2f80ff470 0x7fb2f806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 >> 172.21.15.93:0/437675168 conn(0x7fb2f8075bd0 msgr2=0x7fb2f8105940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 shutdown_connections 2023-07-21T22:31:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:27.462+0000 7fb3000f1700 1 -- 172.21.15.93:0/437675168 wait complete. 2023-07-21T22:31:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:27 smithi093 ceph-mon[96579]: pgmap v477: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:27.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:27 smithi194 ceph-mon[98288]: pgmap v477: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:27.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:27 smithi134 ceph-mon[97873]: pgmap v477: 97 pgs: 97 active+clean; 969 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.5 MiB/s wr, 732 op/s 2023-07-21T22:31:28.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 81343 43.38 MB/sec execute 4 sec latency 50.117 ms 2023-07-21T22:31:29.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 82127 42.61 MB/sec execute 5 sec latency 56.687 ms 2023-07-21T22:31:29.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:29 smithi194 ceph-mon[98288]: pgmap v478: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.2 MiB/s wr, 769 op/s 2023-07-21T22:31:29.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:29 smithi134 ceph-mon[97873]: pgmap v478: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.2 MiB/s wr, 769 op/s 2023-07-21T22:31:29.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:29 smithi093 ceph-mon[96579]: pgmap v478: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.2 MiB/s wr, 769 op/s 2023-07-21T22:31:30.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 82919 42.46 MB/sec execute 6 sec latency 43.693 ms 2023-07-21T22:31:31.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 83846 43.72 MB/sec execute 7 sec latency 41.823 ms 2023-07-21T22:31:31.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:31 smithi194 ceph-mon[98288]: pgmap v479: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2023-07-21T22:31:31.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:31 smithi134 ceph-mon[97873]: pgmap v479: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2023-07-21T22:31:31.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:31 smithi093 ceph-mon[96579]: pgmap v479: 97 pgs: 97 active+clean; 1004 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.0 MiB/s wr, 564 op/s 2023-07-21T22:31:32.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 84638 44.21 MB/sec execute 8 sec latency 42.690 ms 2023-07-21T22:31:33.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 85459 43.96 MB/sec execute 9 sec latency 76.876 ms 2023-07-21T22:31:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:33 smithi194 ceph-mon[98288]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 834 op/s 2023-07-21T22:31:33.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:33 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:33.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:33 smithi134 ceph-mon[97873]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 834 op/s 2023-07-21T22:31:33.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:33 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:33.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:33 smithi093 ceph-mon[96579]: pgmap v480: 97 pgs: 97 active+clean; 1.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 834 op/s 2023-07-21T22:31:33.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:33 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:34.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 86271 43.13 MB/sec execute 10 sec latency 66.972 ms 2023-07-21T22:31:35.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 86702 41.80 MB/sec execute 11 sec latency 475.606 ms 2023-07-21T22:31:35.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:35 smithi194 ceph-mon[98288]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2023-07-21T22:31:35.892 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:35 smithi134 ceph-mon[97873]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2023-07-21T22:31:35.945 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:35 smithi093 ceph-mon[96579]: pgmap v481: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.3 MiB/s wr, 679 op/s 2023-07-21T22:31:36.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 86794 38.80 MB/sec execute 12 sec latency 793.537 ms 2023-07-21T22:31:37.504 INFO:tasks.workunit.client.0.smithi093.stdout: 10 87240 38.11 MB/sec execute 13 sec latency 572.875 ms 2023-07-21T22:31:37.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:37 smithi194 ceph-mon[98288]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 605 op/s 2023-07-21T22:31:37.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:37 smithi134 ceph-mon[97873]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 605 op/s 2023-07-21T22:31:37.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:37 smithi093 ceph-mon[96579]: pgmap v482: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 605 op/s 2023-07-21T22:31:38.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 87733 36.86 MB/sec execute 14 sec latency 110.375 ms 2023-07-21T22:31:39.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 88536 37.19 MB/sec execute 15 sec latency 46.861 ms 2023-07-21T22:31:39.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:39 smithi194 ceph-mon[98288]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.20k op/s 2023-07-21T22:31:39.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:39 smithi134 ceph-mon[97873]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.20k op/s 2023-07-21T22:31:39.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:39 smithi093 ceph-mon[96579]: pgmap v483: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.20k op/s 2023-07-21T22:31:40.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 89469 37.97 MB/sec execute 16 sec latency 49.234 ms 2023-07-21T22:31:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 -- 172.21.15.93:0/2073548856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06d410 msgr2=0x7f8a8c078770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2073548856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06d410 0x7f8a8c078770 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b0a0 tx=0x7f8a8400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 -- 172.21.15.93:0/2073548856 shutdown_connections 2023-07-21T22:31:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2073548856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06d410 0x7f8a8c078770 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2073548856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2073548856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c06ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 -- 172.21.15.93:0/2073548856 >> 172.21.15.93:0/2073548856 conn(0x7f8a8c06bd70 msgr2=0x7f8a8c070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 -- 172.21.15.93:0/2073548856 shutdown_connections 2023-07-21T22:31:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.614+0000 7f8a911ba700 1 -- 172.21.15.93:0/2073548856 wait complete. 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.615+0000 7f8a911ba700 1 Processor -- start 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.615+0000 7f8a911ba700 1 -- start start 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c0831d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c0817a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1bdce0 con 0x7f8a8c06ca80 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1bde20 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a911ba700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c1bdf60 con 0x7f8a8c06d410 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c0831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35676/0 (socket says 172.21.15.93:35676) 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8a59c700 1 -- 172.21.15.93:0/2342781670 learned_addr learned my addr 172.21.15.93:0/2342781670 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c0817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 -- 172.21.15.93:0/2342781670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 msgr2=0x7f8a8c081030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 -- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 msgr2=0x7f8a8c0831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c0831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8b59e700 1 -- 172.21.15.93:0/2342781670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009520 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c0831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.616+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c0817a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b070 tx=0x7f8a840077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a73fff700 1 -- 172.21.15.93:0/2342781670 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400e040 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a84009d00 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a73fff700 1 -- 172.21.15.93:0/2342781670 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a84004e60 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a73fff700 1 -- 172.21.15.93:0/2342781670 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005040 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.617+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c1be4a0 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.620+0000 7f8a73fff700 1 -- 172.21.15.93:0/2342781670 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8a8401d470 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.620+0000 7f8a73fff700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 0x7f8a740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.620+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 0x7f8a740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.620+0000 7f8a73fff700 1 -- 172.21.15.93:0/2342781670 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8a840ce090 con 0x7f8a8c10c1c0 2023-07-21T22:31:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.621+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 0x7f8a740a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a80011630 tx=0x7f8a80009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.621+0000 7f8a9d33fb80 1 Processor -- start 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.621+0000 7f8a9d33fb80 1 -- start start 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555c67c50 con 0x55e555d68660 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555cba9e0 con 0x55e555bbe6b0 2023-07-21T22:31:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555cb3e70 con 0x55e555d68210 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59766/0 (socket says 172.21.15.93:59766) 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35682/0 (socket says 172.21.15.93:35682) 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/738877566 learned_addr learned my addr 172.21.15.93:0/738877566 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/738877566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 msgr2=0x55e555d61f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/738877566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 msgr2=0x55e555d624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/738877566 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e555ae1d80 con 0x55e555bbe6b0 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.622+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a80023e40 tx=0x7f8a80014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c0a93c0e4d516d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a717fa700 1 -- 172.21.15.93:0/738877566 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80028070 con 0x55e555bbe6b0 2023-07-21T22:31:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a717fa700 1 -- 172.21.15.93:0/738877566 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8000eaa0 con 0x55e555bbe6b0 2023-07-21T22:31:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a717fa700 1 -- 172.21.15.93:0/738877566 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800237b0 con 0x55e555bbe6b0 2023-07-21T22:31:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/738877566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 msgr2=0x55e555d67cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a80023e40 tx=0x7f8a80014000 comp rx=0 tx=0).stop 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/738877566 shutdown_connections 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555d67cd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.623+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/738877566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68210 0x55e555d61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/738877566 >> 172.21.15.93:0/738877566 conn(0x55e555d57ae0 msgr2=0x55e555d5d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/738877566 shutdown_connections 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/738877566 wait complete. 2023-07-21T22:31:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 -- start start 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 Processor -- start 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.624+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 0x55e555d6e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d6be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555c53a60 con 0x55e555d68660 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555c53760 con 0x55e555d68210 2023-07-21T22:31:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555b378c0 con 0x55e555bbe6b0 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a9d33fb80 10 client.0 _ll_register_callbacks cb 0x55e555c51700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 0x55e555d6e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 0x55e555d6e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35688/0 (socket says 172.21.15.93:35688) 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/1688952930 learned_addr learned my addr 172.21.15.93:0/1688952930 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d6be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 -- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 msgr2=0x55e555d6e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 0x55e555d6e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 -- 172.21.15.93:0/1688952930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 msgr2=0x55e555d6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.625+0000 7f8a8b59e700 1 -- 172.21.15.93:0/1688952930 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c00be00 con 0x55e555d68660 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d6be10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8a840cb7e0 tx=0x7f8a8409abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c3b95f7272dfdd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8401d930 con 0x55e555d68660 2023-07-21T22:31:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8a8409b1f0 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e555c0d490 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a8409f220 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a9d33fb80 10 client.17361 fetch_fsmap learned FSMap version 71 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x7f8a8c1be4a0 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.626+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e555c0d490 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 71) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f8a8409a770 con 0x55e555d68660 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 10 client.17361 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T22:31:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 20 client.17361 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555d9d940 0x55e555d9fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 4 client.17361 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.627+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e555cace50 con 0x55e555d9d940 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.628+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555d9d940 0x55e555d9fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.628+0000 7f8a6eff5700 10 client.17361 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.628+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555d9d940 0x55e555d9fd50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41577acf2722b675 server_cookie=1d077fcb3a9072c1 in_seq=0 out_seq=0 2023-07-21T22:31:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.629+0000 7f8a6eff5700 1 -- 172.21.15.93:0/1688952930 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e555cace50 con 0x55e555d9d940 2023-07-21T22:31:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.629+0000 7f8a6eff5700 10 client.17361 handle_command_reply: tid=0 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.629+0000 7f8a9d33fb80 1 client.17361 shutdown 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 msgr2=0x55e555d6be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d6be10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8a840cb7e0 tx=0x7f8a8409abd0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 shutdown_connections 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555d9d940 0x55e555d9fd50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68660 0x55e555d6be10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555bbe6b0 0x55e555d6e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/1688952930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 >> 172.21.15.93:0/1688952930 conn(0x55e555d57ae0 msgr2=0x55e555d5c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a6eff5700 0 client.17361 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 shutdown_connections 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/1688952930 wait complete. 2023-07-21T22:31:40.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.630+0000 7f8a9d33fb80 20 client.17361 trim_cache size 0 max 16384 2023-07-21T22:31:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.650+0000 7f8a9d33fb80 1 Processor -- start 2023-07-21T22:31:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 -- start start 2023-07-21T22:31:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d5cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d67900 con 0x55e555bbe6b0 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d594f0 con 0x55e555d68210 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d69da0 con 0x55e555d68660 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48616/0 (socket says 172.21.15.93:48616) 2023-07-21T22:31:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/716582559 learned_addr learned my addr 172.21.15.93:0/716582559 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d5cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/716582559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 msgr2=0x55e555d5cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d5cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/716582559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 msgr2=0x55e555d6b000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8ad9d700 1 -- 172.21.15.93:0/716582559 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80017070 con 0x55e555bbe6b0 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.651+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d5cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.652+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8a80023e40 tx=0x7f8a80014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96228fea15a4bd2f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.652+0000 7f8a71ffb700 1 -- 172.21.15.93:0/716582559 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80028040 con 0x55e555bbe6b0 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.652+0000 7f8a71ffb700 1 -- 172.21.15.93:0/716582559 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8000ec10 con 0x55e555bbe6b0 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.652+0000 7f8a71ffb700 1 -- 172.21.15.93:0/716582559 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800227f0 con 0x55e555bbe6b0 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/716582559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 msgr2=0x55e555d72b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8a80023e40 tx=0x7f8a80014000 comp rx=0 tx=0).stop 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/716582559 shutdown_connections 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d5cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555bbe6b0 0x55e555d72b10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/716582559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555d68210 0x55e555d6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/716582559 >> 172.21.15.93:0/716582559 conn(0x55e555d57ae0 msgr2=0x55e555d6b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/716582559 shutdown_connections 2023-07-21T22:31:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/716582559 wait complete. 2023-07-21T22:31:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.653+0000 7f8a9d33fb80 1 -- start start 2023-07-21T22:31:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 Processor -- start 2023-07-21T22:31:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555deaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d6eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d6f060 con 0x55e555d68210 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d6f1d0 con 0x55e555bbe6b0 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e555d6f390 con 0x55e555d68660 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a9d33fb80 10 client.0 _ll_register_callbacks cb 0x7f8a8409afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48622/0 (socket says 172.21.15.93:48622) 2023-07-21T22:31:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 -- 172.21.15.93:0/3997166649 learned_addr learned my addr 172.21.15.93:0/3997166649 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555deaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8b59e700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d6eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 -- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 msgr2=0x55e555d6eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d6eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 -- 172.21.15.93:0/3997166649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 msgr2=0x55e555deaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555deaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.654+0000 7f8a8a59c700 1 -- 172.21.15.93:0/3997166649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c1be4a0 con 0x55e555d68210 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a8a59c700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x55e555d6cd40 tx=0x7f8a7c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77503b2a8b799c6a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c016030 con 0x55e555d68210 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8a7c004520 con 0x55e555d68210 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8000eaa0 con 0x55e555d68210 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a7c011880 con 0x55e555d68210 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a9d33fb80 10 client.17367 fetch_fsmap learned FSMap version 71 2023-07-21T22:31:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x55e555ae1d80 con 0x55e555d68210 2023-07-21T22:31:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.655+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7c029020 con 0x55e555d68210 2023-07-21T22:31:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 71) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f8a7c01c460 con 0x55e555d68210 2023-07-21T22:31:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 10 client.17367 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T22:31:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 10 client.17367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 20 client.17367 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555df29c0 0x55e555df4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 4 client.17367 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e555c6eff0 con 0x55e555df29c0 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.656+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555df29c0 0x55e555df4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.657+0000 7f8a6eff5700 10 client.17367 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.657+0000 7f8a8ad9d700 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555df29c0 0x55e555df4dd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4014a3eb160790b server_cookie=95fbabaca39b4ac0 in_seq=0 out_seq=0 2023-07-21T22:31:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a6eff5700 1 -- 172.21.15.93:0/3997166649 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e555c6eff0 con 0x55e555df29c0 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a6eff5700 10 client.17367 handle_command_reply: tid=0 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 client.17367 shutdown 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 msgr2=0x55e555deaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x55e555d6cd40 tx=0x7f8a7c010da0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 shutdown_connections 2023-07-21T22:31:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e555df29c0 0x55e555df4dd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e555d68660 0x55e555d6eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e555bbe6b0 0x55e555deaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 --2- 172.21.15.93:0/3997166649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e555d68210 0x55e555deaf60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 >> 172.21.15.93:0/3997166649 conn(0x55e555d57ae0 msgr2=0x55e555d671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a6eff5700 0 client.17367 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 shutdown_connections 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 1 -- 172.21.15.93:0/3997166649 wait complete. 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.776+0000 7f8a9d33fb80 20 client.17367 trim_cache size 0 max 16384 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:31:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:31:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 msgr2=0x7f8a740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 0x7f8a740a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a80011630 tx=0x7f8a80009260 comp rx=0 tx=0).stop 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 msgr2=0x7f8a8c0817a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c0817a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b070 tx=0x7f8a840077f0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 shutdown_connections 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a8c10c1c0 0x7f8a8c0817a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8a740a12a0 0x7f8a740a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a8c06ca80 0x7f8a8c0831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 --2- 172.21.15.93:0/2342781670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a8c06d410 0x7f8a8c081030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 >> 172.21.15.93:0/2342781670 conn(0x7f8a8c06bd70 msgr2=0x7f8a8c107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 shutdown_connections 2023-07-21T22:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:40.777+0000 7f8a911ba700 1 -- 172.21.15.93:0/2342781670 wait complete. 2023-07-21T22:31:40.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:40 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:40.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:40 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:40.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:40 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:41.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 90314 38.64 MB/sec execute 17 sec latency 88.786 ms 2023-07-21T22:31:41.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:41 smithi194 ceph-mon[98288]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 922 op/s 2023-07-21T22:31:41.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:41 smithi134 ceph-mon[97873]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 922 op/s 2023-07-21T22:31:41.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:41 smithi093 ceph-mon[96579]: pgmap v484: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 23 MiB/s wr, 922 op/s 2023-07-21T22:31:42.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 91072 38.50 MB/sec execute 18 sec latency 37.879 ms 2023-07-21T22:31:43.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 91886 38.82 MB/sec execute 19 sec latency 55.114 ms 2023-07-21T22:31:43.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:43 smithi194 ceph-mon[98288]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-21T22:31:43.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:43 smithi134 ceph-mon[97873]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-21T22:31:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:43 smithi093 ceph-mon[96579]: pgmap v485: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.27k op/s 2023-07-21T22:31:44.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 92908 39.55 MB/sec execute 20 sec latency 42.456 ms 2023-07-21T22:31:45.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 93748 40.34 MB/sec execute 21 sec latency 48.151 ms 2023-07-21T22:31:45.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:45 smithi194 ceph-mon[98288]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.09k op/s 2023-07-21T22:31:45.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:45 smithi134 ceph-mon[97873]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.09k op/s 2023-07-21T22:31:45.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:45 smithi093 ceph-mon[96579]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.09k op/s 2023-07-21T22:31:46.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 94475 39.92 MB/sec execute 22 sec latency 78.643 ms 2023-07-21T22:31:47.505 INFO:tasks.workunit.client.0.smithi093.stdout: 10 95249 39.89 MB/sec execute 23 sec latency 58.543 ms 2023-07-21T22:31:47.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:47 smithi194 ceph-mon[98288]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-07-21T22:31:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:47 smithi134 ceph-mon[97873]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-07-21T22:31:47.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:47 smithi093 ceph-mon[96579]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-07-21T22:31:48.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 96227 40.07 MB/sec execute 24 sec latency 66.325 ms 2023-07-21T22:31:48.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:48 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:48.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:48 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:48.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:48 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:49.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 97113 41.38 MB/sec execute 25 sec latency 80.997 ms 2023-07-21T22:31:49.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:49 smithi194 ceph-mon[98288]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.31k op/s 2023-07-21T22:31:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:49 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:49 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:49 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:31:49.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:49 smithi134 ceph-mon[97873]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.31k op/s 2023-07-21T22:31:49.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:49 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:49.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:49 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:49.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:49 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:31:49.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:49 smithi093 ceph-mon[96579]: pgmap v488: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 26 MiB/s wr, 1.31k op/s 2023-07-21T22:31:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:49 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:49 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:49 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:31:50.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 97713 40.81 MB/sec execute 26 sec latency 56.891 ms 2023-07-21T22:31:51.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 98392 40.06 MB/sec execute 27 sec latency 112.179 ms 2023-07-21T22:31:51.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:51 smithi194 ceph-mon[98288]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 718 op/s 2023-07-21T22:31:51.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:51 smithi134 ceph-mon[97873]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 718 op/s 2023-07-21T22:31:51.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:51 smithi093 ceph-mon[96579]: pgmap v489: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.2 MiB/s wr, 718 op/s 2023-07-21T22:31:52.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 99045 40.04 MB/sec execute 28 sec latency 383.720 ms 2023-07-21T22:31:53.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 100016 40.71 MB/sec execute 29 sec latency 45.582 ms 2023-07-21T22:31:53.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:53 smithi194 ceph-mon[98288]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T22:31:53.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:53 smithi134 ceph-mon[97873]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T22:31:53.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:53 smithi093 ceph-mon[96579]: pgmap v490: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T22:31:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 -- 172.21.15.93:0/2169160460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740fed20 msgr2=0x7fa1740ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 --2- 172.21.15.93:0/2169160460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740fed20 0x7fa1740ff190 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa164009990 tx=0x7fa164009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 -- 172.21.15.93:0/2169160460 shutdown_connections 2023-07-21T22:31:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 --2- 172.21.15.93:0/2169160460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740ff6d0 0x7fa17410b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 --2- 172.21.15.93:0/2169160460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 0x7fa174104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 --2- 172.21.15.93:0/2169160460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740fed20 0x7fa1740ff190 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 -- 172.21.15.93:0/2169160460 >> 172.21.15.93:0/2169160460 conn(0x7fa174075270 msgr2=0x7fa174075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 -- 172.21.15.93:0/2169160460 shutdown_connections 2023-07-21T22:31:53.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.922+0000 7fa17b993700 1 -- 172.21.15.93:0/2169160460 wait complete. 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 Processor -- start 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 -- start start 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740fed20 0x7fa17419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 0x7fa174195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174196110 con 0x7fa1740ff6d0 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174196280 con 0x7fa174104370 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17b993700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174196440 con 0x7fa1740fed20 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa178f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa178f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48636/0 (socket says 172.21.15.93:48636) 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa178f2e700 1 -- 172.21.15.93:0/4255859978 learned_addr learned my addr 172.21.15.93:0/4255859978 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa17972f700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740fed20 0x7fa17419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.923+0000 7fa179f30700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 0x7fa174195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 -- 172.21.15.93:0/4255859978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740fed20 msgr2=0x7fa17419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740fed20 0x7fa17419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 -- 172.21.15.93:0/4255859978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 msgr2=0x7fa174195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 0x7fa174195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 -- 172.21.15.93:0/4255859978 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640095f0 con 0x7fa1740ff6d0 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa178f2e700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa164000c00 tx=0x7fa1640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa16a7fc700 1 -- 172.21.15.93:0/4255859978 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16401d070 con 0x7fa1740ff6d0 2023-07-21T22:31:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1741966c0 con 0x7fa1740ff6d0 2023-07-21T22:31:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa16a7fc700 1 -- 172.21.15.93:0/4255859978 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa164021410 con 0x7fa1740ff6d0 2023-07-21T22:31:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa16a7fc700 1 -- 172.21.15.93:0/4255859978 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164017650 con 0x7fa1740ff6d0 2023-07-21T22:31:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.924+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741a2c30 con 0x7fa1740ff6d0 2023-07-21T22:31:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.927+0000 7fa16a7fc700 1 -- 172.21.15.93:0/4255859978 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa1640217f0 con 0x7fa1740ff6d0 2023-07-21T22:31:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa16a7fc700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 0x7fa1600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa17972f700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 0x7fa1600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa16a7fc700 1 -- 172.21.15.93:0/4255859978 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa1640ce0b0 con 0x7fa1740ff6d0 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa17972f700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 0x7fa1600a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa170009be0 tx=0x7fa170009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa187b18b80 1 Processor -- start 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.928+0000 7fa187b18b80 1 -- start start 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45bb76b0 0x555a45d60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d61210 0x555a45d5af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45c60c50 con 0x555a45d61660 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45cb39e0 con 0x555a45bb76b0 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa187b18b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45cace70 con 0x555a45d61210 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48642/0 (socket says 172.21.15.93:48642) 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 -- 172.21.15.93:0/4274226479 learned_addr learned my addr 172.21.15.93:0/4274226479 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa17972f700 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45bb76b0 0x555a45d60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 -- 172.21.15.93:0/4274226479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d61210 msgr2=0x555a45d5af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d61210 0x555a45d5af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 -- 172.21.15.93:0/4274226479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45bb76b0 msgr2=0x555a45d60cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45bb76b0 0x555a45d60cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 -- 172.21.15.93:0/4274226479 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a45adad80 con 0x555a45d61660 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.929+0000 7fa179f30700 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa16c003fc0 tx=0x7fa16c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ac1e843f616b72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.930+0000 7fa15f7fe700 1 -- 172.21.15.93:0/4274226479 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c019070 con 0x555a45d61660 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.930+0000 7fa15f7fe700 1 -- 172.21.15.93:0/4274226479 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16c00f040 con 0x555a45d61660 2023-07-21T22:31:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.930+0000 7fa15f7fe700 1 -- 172.21.15.93:0/4274226479 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c007a80 con 0x555a45d61660 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- 172.21.15.93:0/4274226479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 msgr2=0x555a45d5b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa16c003fc0 tx=0x7fa16c009ba0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- 172.21.15.93:0/4274226479 shutdown_connections 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61660 0x555a45d5b4a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45bb76b0 0x555a45d60cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 --2- 172.21.15.93:0/4274226479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d61210 0x555a45d5af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- 172.21.15.93:0/4274226479 >> 172.21.15.93:0/4274226479 conn(0x555a45d50ae0 msgr2=0x555a45d560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- 172.21.15.93:0/4274226479 shutdown_connections 2023-07-21T22:31:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- 172.21.15.93:0/4274226479 wait complete. 2023-07-21T22:31:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 -- start start 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.931+0000 7fa187b18b80 1 Processor -- start 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d64970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d64eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45c4ca60 con 0x555a45d61210 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45c4c760 con 0x555a45d61660 2023-07-21T22:31:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45b308c0 con 0x555a45bb76b0 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa187b18b80 10 client.0 _ll_register_callbacks cb 0x555a45c4a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa178f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d64eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa17972f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d64970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa17972f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d64970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35728/0 (socket says 172.21.15.93:35728) 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa178f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d64eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48650/0 (socket says 172.21.15.93:48650) 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59814/0 (socket says 172.21.15.93:59814) 2023-07-21T22:31:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa17972f700 1 -- 172.21.15.93:0/2790149398 learned_addr learned my addr 172.21.15.93:0/2790149398 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 -- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 msgr2=0x555a45d64970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d64970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 -- 172.21.15.93:0/2790149398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 msgr2=0x555a45d64eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d64eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.932+0000 7fa179f30700 1 -- 172.21.15.93:0/2790149398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c003bf0 con 0x555a45d61660 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa179f30700 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x555a45d61a70 tx=0x7fa16c007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e52afbbb3d5781b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c019070 con 0x555a45d61660 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa16c007d60 con 0x555a45d61660 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a45c06490 con 0x555a45d61660 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16c004ad0 con 0x555a45d61660 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 10 client.26750 fetch_fsmap learned FSMap version 71 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x555a45ac3c00 con 0x555a45d61660 2023-07-21T22:31:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a45c06490 con 0x555a45d61660 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 71) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fa16c015c40 con 0x555a45d61660 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 10 client.26750 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 20 client.26750 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d74bb0 0x555a45d76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 4 client.26750 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.933+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a45ca5e50 con 0x555a45d74bb0 2023-07-21T22:31:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.934+0000 7fa17972f700 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d74bb0 0x555a45d76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.934+0000 7fa15cff9700 10 client.26750 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.934+0000 7fa17972f700 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d74bb0 0x555a45d76fc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65672bff9b58269d server_cookie=65d2c149c7d62c6f in_seq=0 out_seq=0 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.935+0000 7fa15cff9700 1 -- 172.21.15.93:0/2790149398 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a45ca5e50 con 0x555a45d74bb0 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.935+0000 7fa15cff9700 10 client.26750 handle_command_reply: tid=0 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 client.26750 shutdown 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 msgr2=0x555a45d653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x555a45d61a70 tx=0x7fa16c007870 comp rx=0 tx=0).stop 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 shutdown_connections 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d74bb0 0x555a45d76fc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61660 0x555a45d653f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa15cff9700 0 client.26750 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d64970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 --2- 172.21.15.93:0/2790149398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d64eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 >> 172.21.15.93:0/2790149398 conn(0x555a45d50ae0 msgr2=0x555a45d553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 shutdown_connections 2023-07-21T22:31:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 1 -- 172.21.15.93:0/2790149398 wait complete. 2023-07-21T22:31:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.936+0000 7fa187b18b80 20 client.26750 trim_cache size 0 max 16384 2023-07-21T22:31:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 Processor -- start 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 -- start start 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 0x555a45d6b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d5af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45d577f0 con 0x555a45d61210 2023-07-21T22:31:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45d62f40 con 0x555a45d50ae0 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.955+0000 7fa187b18b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45d5c010 con 0x555a45bb76b0 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa17972f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 0x555a45d6b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48660/0 (socket says 172.21.15.93:48660) 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 -- 172.21.15.93:0/675246309 learned_addr learned my addr 172.21.15.93:0/675246309 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa178f2e700 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d5af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 -- 172.21.15.93:0/675246309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 msgr2=0x555a45d5af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d5af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 -- 172.21.15.93:0/675246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 msgr2=0x555a45d6b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 0x555a45d6b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 -- 172.21.15.93:0/675246309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a45b011f0 con 0x555a45d61210 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa17972f700 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 0x555a45d6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa179f30700 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007610 tx=0x7fa16c01e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b213031e341a0485 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa15ffff700 1 -- 172.21.15.93:0/675246309 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c019070 con 0x555a45d61210 2023-07-21T22:31:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa15ffff700 1 -- 172.21.15.93:0/675246309 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16c01a040 con 0x555a45d61210 2023-07-21T22:31:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.956+0000 7fa15ffff700 1 -- 172.21.15.93:0/675246309 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c01cbf0 con 0x555a45d61210 2023-07-21T22:31:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- 172.21.15.93:0/675246309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 msgr2=0x555a45d60120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007610 tx=0x7fa16c01e5e0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- 172.21.15.93:0/675246309 shutdown_connections 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45d61210 0x555a45d60120 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d50ae0 0x555a45d6b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 --2- 172.21.15.93:0/675246309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45bb76b0 0x555a45d5af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- 172.21.15.93:0/675246309 >> 172.21.15.93:0/675246309 conn(0x555a45d71240 msgr2=0x555a45d5fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- 172.21.15.93:0/675246309 shutdown_connections 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- 172.21.15.93:0/675246309 wait complete. 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 -- start start 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.957+0000 7fa187b18b80 1 Processor -- start 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d50ae0 0x555a45dbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61210 0x555a45dc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45dc7fc0 con 0x555a45bb76b0 2023-07-21T22:31:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45dc8100 con 0x555a45d61210 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a45dc8240 con 0x555a45d50ae0 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa187b18b80 10 client.0 _ll_register_callbacks cb 0x7fa16c003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48662/0 (socket says 172.21.15.93:48662) 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 -- 172.21.15.93:0/1301667150 learned_addr learned my addr 172.21.15.93:0/1301667150 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa178f2e700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d50ae0 0x555a45dbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa179f30700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61210 0x555a45dc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 -- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d50ae0 msgr2=0x555a45dbd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d50ae0 0x555a45dbd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 -- 172.21.15.93:0/1301667150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61210 msgr2=0x555a45dc7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61210 0x555a45dc7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 -- 172.21.15.93:0/1301667150 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa170017070 con 0x555a45bb76b0 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa17972f700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x555a45d778a0 tx=0x7fa1700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dcdb64ee1cd0a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.958+0000 7fa15cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170024040 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa170029b60 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa16c00f040 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa170029d20 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 10 client.17385 fetch_fsmap learned FSMap version 71 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x555a45ac3c00 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16c00f040 con 0x555a45bb76b0 2023-07-21T22:31:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 71) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fa17001f5f0 con 0x555a45bb76b0 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 10 client.17385 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 10 client.17385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 20 client.17385 populate_metadata read hostname 'smithi093' 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d61660 0x555a45dcbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 4 client.17385 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a45c67ff0 con 0x555a45d61660 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.959+0000 7fa178f2e700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d61660 0x555a45dcbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.960+0000 7fa15cff9700 10 client.17385 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:53.960+0000 7fa178f2e700 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d61660 0x555a45dcbab0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e175c1ca7113bb server_cookie=c2feffeb5aae936f in_seq=0 out_seq=0 2023-07-21T22:31:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa15cff9700 1 -- 172.21.15.93:0/1301667150 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a45c67ff0 con 0x555a45d61660 2023-07-21T22:31:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa15cff9700 10 client.17385 handle_command_reply: tid=0 2023-07-21T22:31:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 client.17385 shutdown 2023-07-21T22:31:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 msgr2=0x555a45dbcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x555a45d778a0 tx=0x7fa1700047c0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 shutdown_connections 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a45d61660 0x555a45dcbab0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a45d61210 0x555a45dc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa15cff9700 0 client.17385 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a45bb76b0 0x555a45dbcbf0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 --2- 172.21.15.93:0/1301667150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a45d50ae0 0x555a45dbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 >> 172.21.15.93:0/1301667150 conn(0x555a45d71240 msgr2=0x555a45d66aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 shutdown_connections 2023-07-21T22:31:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.202+0000 7fa187b18b80 1 -- 172.21.15.93:0/1301667150 wait complete. 2023-07-21T22:31:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa187b18b80 20 client.17385 trim_cache size 0 max 16384 2023-07-21T22:31:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:31:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:31:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:31:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:31:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 msgr2=0x7fa1600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 0x7fa1600a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa170009be0 tx=0x7fa170009260 comp rx=0 tx=0).stop 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 msgr2=0x7fa17419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa164000c00 tx=0x7fa1640047b0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 shutdown_connections 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa174104370 0x7fa174195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1740fed20 0x7fa17419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa1600a12c0 0x7fa1600a3770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 --2- 172.21.15.93:0/4255859978 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1740ff6d0 0x7fa17419c180 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 >> 172.21.15.93:0/4255859978 conn(0x7fa174075270 msgr2=0x7fa1740fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 shutdown_connections 2023-07-21T22:31:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:31:54.203+0000 7fa17b993700 1 -- 172.21.15.93:0/4255859978 wait complete. 2023-07-21T22:31:54.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 100678 40.60 MB/sec execute 30 sec latency 163.801 ms 2023-07-21T22:31:55.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 101356 40.24 MB/sec execute 31 sec latency 55.381 ms 2023-07-21T22:31:55.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:55 smithi194 ceph-mon[98288]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 636 op/s 2023-07-21T22:31:55.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:55 smithi134 ceph-mon[97873]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 636 op/s 2023-07-21T22:31:55.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:55 smithi093 ceph-mon[96579]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.1 MiB/s wr, 636 op/s 2023-07-21T22:31:56.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 102240 40.42 MB/sec execute 32 sec latency 54.879 ms 2023-07-21T22:31:57.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 103154 40.92 MB/sec execute 33 sec latency 98.979 ms 2023-07-21T22:31:57.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:57 smithi194 ceph-mon[98288]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2023-07-21T22:31:57.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:57 smithi134 ceph-mon[97873]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2023-07-21T22:31:57.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:57 smithi093 ceph-mon[96579]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2023-07-21T22:31:58.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 103818 40.74 MB/sec execute 34 sec latency 65.452 ms 2023-07-21T22:31:58.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:58 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:58.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:31:58 smithi194 ceph-mon[98288]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2023-07-21T22:31:58.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:58 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:58.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:31:58 smithi134 ceph-mon[97873]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2023-07-21T22:31:58.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:58 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:58.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:31:58 smithi093 ceph-mon[96579]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2023-07-21T22:31:59.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 104546 40.75 MB/sec execute 35 sec latency 36.513 ms 2023-07-21T22:32:00.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 105314 40.39 MB/sec execute 36 sec latency 67.675 ms 2023-07-21T22:32:01.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 105665 39.84 MB/sec execute 37 sec latency 662.220 ms 2023-07-21T22:32:01.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:01 smithi194 ceph-mon[98288]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-07-21T22:32:01.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:01 smithi134 ceph-mon[97873]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-07-21T22:32:01.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:01 smithi093 ceph-mon[96579]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 592 op/s 2023-07-21T22:32:02.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 105788 38.96 MB/sec execute 38 sec latency 887.621 ms 2023-07-21T22:32:03.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 106267 38.64 MB/sec execute 39 sec latency 955.446 ms 2023-07-21T22:32:03.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:03 smithi194 ceph-mon[98288]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-07-21T22:32:03.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:03 smithi134 ceph-mon[97873]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-07-21T22:32:03.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:03 smithi093 ceph-mon[96579]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 760 op/s 2023-07-21T22:32:04.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 106521 38.19 MB/sec execute 40 sec latency 183.889 ms 2023-07-21T22:32:04.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:04 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:04.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:04 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:04.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:04 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:05.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 106711 37.57 MB/sec execute 41 sec latency 538.329 ms 2023-07-21T22:32:05.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:05 smithi134 ceph-mon[97873]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-07-21T22:32:05.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:05 smithi134 ceph-mon[97873]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:32:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:05 smithi093 ceph-mon[96579]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-07-21T22:32:05.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:05 smithi093 ceph-mon[96579]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:32:05.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:05 smithi194 ceph-mon[98288]: pgmap v496: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-07-21T22:32:05.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:05 smithi194 ceph-mon[98288]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:32:06.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 107494 37.74 MB/sec execute 42 sec latency 38.430 ms 2023-07-21T22:32:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 -- 172.21.15.93:0/4289096855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0feaa0 msgr2=0x7f633c0fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 --2- 172.21.15.93:0/4289096855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0feaa0 0x7f633c0fef10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6338009990 tx=0x7f6338009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 -- 172.21.15.93:0/4289096855 shutdown_connections 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 --2- 172.21.15.93:0/4289096855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c0ff450 0x7f633c10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 --2- 172.21.15.93:0/4289096855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c104ab0 0x7f633c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 --2- 172.21.15.93:0/4289096855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0feaa0 0x7f633c0fef10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.359+0000 7f63445f4700 1 -- 172.21.15.93:0/4289096855 >> 172.21.15.93:0/4289096855 conn(0x7f633c075ba0 msgr2=0x7f633c075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.360+0000 7f63445f4700 1 -- 172.21.15.93:0/4289096855 shutdown_connections 2023-07-21T22:32:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.360+0000 7f63445f4700 1 -- 172.21.15.93:0/4289096855 wait complete. 2023-07-21T22:32:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.360+0000 7f63445f4700 1 Processor -- start 2023-07-21T22:32:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.360+0000 7f63445f4700 1 -- start start 2023-07-21T22:32:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c0feaa0 0x7f633c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c104ab0 0x7f633c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19cf60 con 0x7f633c0ff450 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19d0d0 con 0x7f633c104ab0 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19d290 con 0x7f633c0feaa0 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48678/0 (socket says 172.21.15.93:48678) 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 -- 172.21.15.93:0/1677534230 learned_addr learned my addr 172.21.15.93:0/1677534230 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6342390700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c0feaa0 0x7f633c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6342b91700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c104ab0 0x7f633c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 -- 172.21.15.93:0/1677534230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c0feaa0 msgr2=0x7f633c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c0feaa0 0x7f633c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 -- 172.21.15.93:0/1677534230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c104ab0 msgr2=0x7f633c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c104ab0 0x7f633c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 -- 172.21.15.93:0/1677534230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334009c40 con 0x7f633c0ff450 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f6341b8f700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6338000c00 tx=0x7f6338004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63337fe700 1 -- 172.21.15.93:0/1677534230 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633801d070 con 0x7f633c0ff450 2023-07-21T22:32:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63380095f0 con 0x7f633c0ff450 2023-07-21T22:32:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.361+0000 7f63337fe700 1 -- 172.21.15.93:0/1677534230 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6338021410 con 0x7f633c0ff450 2023-07-21T22:32:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.362+0000 7f63337fe700 1 -- 172.21.15.93:0/1677534230 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338017650 con 0x7f633c0ff450 2023-07-21T22:32:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.362+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1a4bd0 con 0x7f633c0ff450 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.364+0000 7f63337fe700 1 -- 172.21.15.93:0/1677534230 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6338021580 con 0x7f633c0ff450 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.365+0000 7f63337fe700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 0x7f63280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.365+0000 7f63337fe700 1 -- 172.21.15.93:0/1677534230 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f63380cddd0 con 0x7f633c0ff450 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.365+0000 7f6342390700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 0x7f63280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6342390700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 0x7f63280a3770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f633c0ffbe0 tx=0x7f632c009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 Processor -- start 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 -- start start 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f210 0x555a15838f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f660 0x555a158394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1573ec50 con 0x555a156956b0 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a157919e0 con 0x555a1583f660 2023-07-21T22:32:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.366+0000 7f6350779b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1578ae70 con 0x555a1583f210 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6341b8f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f210 0x555a15838f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48684/0 (socket says 172.21.15.93:48684) 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 -- 172.21.15.93:0/2268075374 learned_addr learned my addr 172.21.15.93:0/2268075374 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342b91700 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f660 0x555a158394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 -- 172.21.15.93:0/2268075374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f210 msgr2=0x555a15838f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f210 0x555a15838f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 -- 172.21.15.93:0/2268075374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f660 msgr2=0x555a158394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f660 0x555a158394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 -- 172.21.15.93:0/2268075374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a155b8d80 con 0x555a156956b0 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6342390700 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f632c011d50 tx=0x7f632c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8d8b19831a0bed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6330ff9700 1 -- 172.21.15.93:0/2268075374 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00ed20 con 0x555a156956b0 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6330ff9700 1 -- 172.21.15.93:0/2268075374 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632c021070 con 0x555a156956b0 2023-07-21T22:32:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.367+0000 7f6330ff9700 1 -- 172.21.15.93:0/2268075374 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c01d670 con 0x555a156956b0 2023-07-21T22:32:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 -- 172.21.15.93:0/2268075374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 msgr2=0x555a1583ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f632c011d50 tx=0x7f632c014000 comp rx=0 tx=0).stop 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 -- 172.21.15.93:0/2268075374 shutdown_connections 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f660 0x555a158394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a1583ecd0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 --2- 172.21.15.93:0/2268075374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f210 0x555a15838f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.368+0000 7f6350779b80 1 -- 172.21.15.93:0/2268075374 >> 172.21.15.93:0/2268075374 conn(0x555a1582eae0 msgr2=0x555a158340e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.369+0000 7f6350779b80 1 -- 172.21.15.93:0/2268075374 shutdown_connections 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.369+0000 7f6350779b80 1 -- 172.21.15.93:0/2268075374 wait complete. 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.369+0000 7f6350779b80 1 -- start start 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.369+0000 7f6350779b80 1 Processor -- start 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f210 0x555a15842810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15842f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1572aa60 con 0x555a1583f210 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1572a760 con 0x555a156956b0 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1560e8c0 con 0x555a1583f660 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6350779b80 10 client.0 _ll_register_callbacks cb 0x555a15728700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59852/0 (socket says 172.21.15.93:59852) 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 -- 172.21.15.93:0/3150518557 learned_addr learned my addr 172.21.15.93:0/3150518557 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342b91700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15842f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 -- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 msgr2=0x555a15842f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15842f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 -- 172.21.15.93:0/3150518557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f210 msgr2=0x555a15842810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f210 0x555a15842810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.370+0000 7f6342390700 1 -- 172.21.15.93:0/3150518557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c017070 con 0x555a156956b0 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.371+0000 7f6342390700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f632c011b30 tx=0x7f632c01cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1568433f0a8d04d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.371+0000 7f6325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.371+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00edb0 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.371+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f632c01c410 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.371+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a156e4490 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.372+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f632c01c5d0 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.372+0000 7f6350779b80 10 client.26768 fetch_fsmap learned FSMap version 73 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.372+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x555a155a1c00 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.372+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a156e4490 con 0x555a156956b0 2023-07-21T22:32:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.372+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 73) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f632c004810 con 0x555a156956b0 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 10 client.26768 fetch_fsmap finished waiting for FSMap version 73 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 10 client.26768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 20 client.26768 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a15852b10 0x555a15854f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 4 client.26768 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a15783e50 con 0x555a15852b10 2023-07-21T22:32:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6341b8f700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a15852b10 0x555a15854f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6325ffb700 10 client.26768 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.373+0000 7f6341b8f700 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a15852b10 0x555a15854f20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6413f7e925cde34 server_cookie=a603424327dc27d8 in_seq=0 out_seq=0 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.376+0000 7f6325ffb700 1 -- 172.21.15.93:0/3150518557 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a15783e50 con 0x555a15852b10 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.376+0000 7f6325ffb700 10 client.26768 handle_command_reply: tid=0 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 client.26768 shutdown 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 msgr2=0x555a15845600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f632c011b30 tx=0x7f632c01cde0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 shutdown_connections 2023-07-21T22:32:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a15852b10 0x555a15854f20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15842f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a156956b0 0x555a15845600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 --2- 172.21.15.93:0/3150518557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f210 0x555a15842810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 >> 172.21.15.93:0/3150518557 conn(0x555a1582eae0 msgr2=0x555a158333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6325ffb700 0 client.26768 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 shutdown_connections 2023-07-21T22:32:07.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 1 -- 172.21.15.93:0/3150518557 wait complete. 2023-07-21T22:32:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.377+0000 7f6350779b80 20 client.26768 trim_cache size 0 max 16384 2023-07-21T22:32:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.397+0000 7f6350779b80 1 Processor -- start 2023-07-21T22:32:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.397+0000 7f6350779b80 1 -- start start 2023-07-21T22:32:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a156956b0 0x555a15849ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a1583e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a15843ca0 con 0x555a1583f660 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1583d4e0 con 0x555a1583f210 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6350779b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a1583e8c0 con 0x555a156956b0 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48700/0 (socket says 172.21.15.93:48700) 2023-07-21T22:32:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 -- 172.21.15.93:0/1223899180 learned_addr learned my addr 172.21.15.93:0/1223899180 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342390700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a156956b0 0x555a15849ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6341b8f700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a1583e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 -- 172.21.15.93:0/1223899180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a156956b0 msgr2=0x555a15849ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a156956b0 0x555a15849ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 -- 172.21.15.93:0/1223899180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 msgr2=0x555a1583e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a1583e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 -- 172.21.15.93:0/1223899180 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a155df1f0 con 0x555a1583f660 2023-07-21T22:32:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6341b8f700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a1583e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.398+0000 7f6342b91700 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6334007580 tx=0x7f6334009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7820d28788468f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.399+0000 7f63317fa700 1 -- 172.21.15.93:0/1223899180 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400e040 con 0x555a1583f660 2023-07-21T22:32:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.399+0000 7f63317fa700 1 -- 172.21.15.93:0/1223899180 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f633401b070 con 0x555a1583f660 2023-07-21T22:32:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.399+0000 7f63317fa700 1 -- 172.21.15.93:0/1223899180 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633401f610 con 0x555a1583f660 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.399+0000 7f6350779b80 1 -- 172.21.15.93:0/1223899180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 msgr2=0x555a15845130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.399+0000 7f6350779b80 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6334007580 tx=0x7f6334009920 comp rx=0 tx=0).stop 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 -- 172.21.15.93:0/1223899180 shutdown_connections 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a1583f660 0x555a15845130 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a156956b0 0x555a15849ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 --2- 172.21.15.93:0/1223899180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a1583e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 -- 172.21.15.93:0/1223899180 >> 172.21.15.93:0/1223899180 conn(0x555a1582eae0 msgr2=0x555a1583d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 -- 172.21.15.93:0/1223899180 shutdown_connections 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 -- 172.21.15.93:0/1223899180 wait complete. 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 -- start start 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 Processor -- start 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.400+0000 7f6350779b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a158493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a15849900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a15846700 con 0x555a156956b0 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a15846870 con 0x555a1583f210 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a15846a30 con 0x555a1583f660 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6350779b80 10 client.0 _ll_register_callbacks cb 0x7f633400e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35782/0 (socket says 172.21.15.93:35782) 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 -- 172.21.15.93:0/3590478536 learned_addr learned my addr 172.21.15.93:0/3590478536 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342390700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a158493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6341b8f700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a15849900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 -- 172.21.15.93:0/3590478536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 msgr2=0x555a15849900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a15849900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 -- 172.21.15.93:0/3590478536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 msgr2=0x555a158493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a158493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 -- 172.21.15.93:0/3590478536 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334009260 con 0x555a1583f660 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342390700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a158493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6341b8f700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a15849900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6342b91700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6334000f80 tx=0x7f6334004d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8f38526b3adb12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.401+0000 7f6325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633402c070 con 0x555a1583f660 2023-07-21T22:32:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f632c021070 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6334003a00 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633400e050 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 10 client.26695 fetch_fsmap learned FSMap version 73 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x555a155b7250 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632c021070 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 73) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f633402f020 con 0x555a1583f660 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 10 client.26695 fetch_fsmap finished waiting for FSMap version 73 2023-07-21T22:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 20 client.26695 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a158a7a90 0x555a158a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 4 client.26695 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.402+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a15745ff0 con 0x555a158a7a90 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.403+0000 7f6342390700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a158a7a90 0x555a158a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.403+0000 7f6325ffb700 10 client.26695 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.403+0000 7f6342390700 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a158a7a90 0x555a158a9ea0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c1415d4c52883b server_cookie=86119f80c5b931b5 in_seq=0 out_seq=0 2023-07-21T22:32:07.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 108188 37.58 MB/sec execute 43 sec latency 51.911 ms 2023-07-21T22:32:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6325ffb700 1 -- 172.21.15.93:0/3590478536 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a15745ff0 con 0x555a158a7a90 2023-07-21T22:32:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6325ffb700 10 client.26695 handle_command_reply: tid=0 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 client.26695 shutdown 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 msgr2=0x555a15846130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6334000f80 tx=0x7f6334004d80 comp rx=0 tx=0).stop 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 shutdown_connections 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x555a158a7a90 0x555a158a9ea0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a1583f660 0x555a15846130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a156956b0 0x555a158493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 --2- 172.21.15.93:0/3590478536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1583f210 0x555a15849900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 >> 172.21.15.93:0/3590478536 conn(0x555a1582eae0 msgr2=0x555a1583e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.584+0000 7f6325ffb700 0 client.26695 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.587+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 shutdown_connections 2023-07-21T22:32:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.587+0000 7f6350779b80 1 -- 172.21.15.93:0/3590478536 wait complete. 2023-07-21T22:32:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.587+0000 7f6350779b80 20 client.26695 trim_cache size 0 max 16384 2023-07-21T22:32:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:32:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:32:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:32:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:32:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 msgr2=0x7f63280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 0x7f63280a3770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f633c0ffbe0 tx=0x7f632c009260 comp rx=0 tx=0).stop 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 msgr2=0x7f633c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6338000c00 tx=0x7f6338004790 comp rx=0 tx=0).stop 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 shutdown_connections 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c104ab0 0x7f633c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f633c0feaa0 0x7f633c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f63280a12c0 0x7f63280a3770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 --2- 172.21.15.93:0/1677534230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f633c0ff450 0x7f633c1982b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 >> 172.21.15.93:0/1677534230 conn(0x7f633c075ba0 msgr2=0x7f633c068850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 shutdown_connections 2023-07-21T22:32:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:07.588+0000 7f63445f4700 1 -- 172.21.15.93:0/1677534230 wait complete. 2023-07-21T22:32:07.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:07 smithi134 ceph-mon[97873]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 653 op/s 2023-07-21T22:32:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:07 smithi093 ceph-mon[96579]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 653 op/s 2023-07-21T22:32:07.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:07 smithi194 ceph-mon[98288]: pgmap v498: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 653 op/s 2023-07-21T22:32:08.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 108999 37.73 MB/sec execute 44 sec latency 105.590 ms 2023-07-21T22:32:09.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 109795 37.91 MB/sec execute 45 sec latency 48.607 ms 2023-07-21T22:32:09.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:09 smithi134 ceph-mon[97873]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:09 smithi093 ceph-mon[96579]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:09.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:09 smithi194 ceph-mon[98288]: pgmap v499: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:10.506 INFO:tasks.workunit.client.0.smithi093.stdout: 10 110596 38.00 MB/sec execute 46 sec latency 57.549 ms 2023-07-21T22:32:11.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 110912 37.54 MB/sec execute 47 sec latency 600.052 ms 2023-07-21T22:32:11.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:11 smithi093 ceph-mon[96579]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:11.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:11 smithi194 ceph-mon[98288]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:11.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:11 smithi134 ceph-mon[97873]: pgmap v500: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T22:32:12.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 110960 36.81 MB/sec execute 48 sec latency 1313.308 ms 2023-07-21T22:32:13.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 111373 36.51 MB/sec execute 49 sec latency 572.646 ms 2023-07-21T22:32:13.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:13 smithi194 ceph-mon[98288]: pgmap v501: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 972 op/s 2023-07-21T22:32:13.592 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:13 smithi194 ceph-mon[98288]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:32:13.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:13 smithi134 ceph-mon[97873]: pgmap v501: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 972 op/s 2023-07-21T22:32:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:13 smithi134 ceph-mon[97873]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:32:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:13 smithi093 ceph-mon[96579]: pgmap v501: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 22 MiB/s wr, 972 op/s 2023-07-21T22:32:13.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:13 smithi093 ceph-mon[96579]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:32:14.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 111887 36.32 MB/sec execute 50 sec latency 99.955 ms 2023-07-21T22:32:15.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 112803 36.58 MB/sec execute 51 sec latency 47.844 ms 2023-07-21T22:32:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:15 smithi194 ceph-mon[98288]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 28 MiB/s wr, 1.11k op/s 2023-07-21T22:32:15.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:15 smithi134 ceph-mon[97873]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 28 MiB/s wr, 1.11k op/s 2023-07-21T22:32:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:15 smithi093 ceph-mon[96579]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 28 MiB/s wr, 1.11k op/s 2023-07-21T22:32:16.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 113706 36.85 MB/sec execute 52 sec latency 47.684 ms 2023-07-21T22:32:17.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 114308 36.78 MB/sec execute 53 sec latency 83.334 ms 2023-07-21T22:32:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:17 smithi194 ceph-mon[98288]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T22:32:17.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:17 smithi134 ceph-mon[97873]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T22:32:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:17 smithi093 ceph-mon[96579]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 28 MiB/s wr, 1.10k op/s 2023-07-21T22:32:18.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 115027 36.83 MB/sec execute 54 sec latency 64.664 ms 2023-07-21T22:32:19.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 115854 36.76 MB/sec execute 55 sec latency 53.926 ms 2023-07-21T22:32:19.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:19 smithi194 ceph-mon[98288]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:19.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:19 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:19.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:19 smithi134 ceph-mon[97873]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:19.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:19 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:19 smithi093 ceph-mon[96579]: pgmap v505: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:19.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:19 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:20.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 116823 37.30 MB/sec execute 56 sec latency 53.376 ms 2023-07-21T22:32:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 -- 172.21.15.93:0/2795146867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 msgr2=0x7f9cec10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 --2- 172.21.15.93:0/2795146867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec10bd40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80099c0 tx=0x7f9ce8009cd0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 -- 172.21.15.93:0/2795146867 shutdown_connections 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 --2- 172.21.15.93:0/2795146867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10cff0 0x7f9cec1137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 --2- 172.21.15.93:0/2795146867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec10bd40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 --2- 172.21.15.93:0/2795146867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cec10c640 0x7f9cec10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 -- 172.21.15.93:0/2795146867 >> 172.21.15.93:0/2795146867 conn(0x7f9cec06bd70 msgr2=0x7f9cec070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.733+0000 7f9cf3773700 1 -- 172.21.15.93:0/2795146867 shutdown_connections 2023-07-21T22:32:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.735+0000 7f9cf3773700 1 -- 172.21.15.93:0/2795146867 wait complete. 2023-07-21T22:32:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.735+0000 7f9cf3773700 1 Processor -- start 2023-07-21T22:32:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 -- start start 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 0x7f9cec1a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cec10cff0 0x7f9cec19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19c8e0 con 0x7f9cec10cff0 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19ca50 con 0x7f9cec10b970 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf3773700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec19cc10 con 0x7f9cec10c640 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 0x7f9cec1a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf150f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf150f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59876/0 (socket says 172.21.15.93:59876) 2023-07-21T22:32:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf150f700 1 -- 172.21.15.93:0/1076665460 learned_addr learned my addr 172.21.15.93:0/1076665460 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.736+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 0x7f9cec1a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35796/0 (socket says 172.21.15.93:35796) 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 -- 172.21.15.93:0/1076665460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 msgr2=0x7f9cec1a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 0x7f9cec1a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 -- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cec10cff0 msgr2=0x7f9cec19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cec10cff0 0x7f9cec19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 -- 172.21.15.93:0/1076665460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80095f0 con 0x7f9cec10b970 2023-07-21T22:32:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9cf150f700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009990 tx=0x7f9ce800bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.737+0000 7f9ce27fc700 1 -- 172.21.15.93:0/1076665460 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce801d070 con 0x7f9cec10b970 2023-07-21T22:32:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.738+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec19ce90 con 0x7f9cec10b970 2023-07-21T22:32:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.738+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec1a1130 con 0x7f9cec10b970 2023-07-21T22:32:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.738+0000 7f9ce27fc700 1 -- 172.21.15.93:0/1076665460 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce8004cb0 con 0x7f9cec10b970 2023-07-21T22:32:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.738+0000 7f9ce27fc700 1 -- 172.21.15.93:0/1076665460 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8003a10 con 0x7f9cec10b970 2023-07-21T22:32:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.741+0000 7f9ce27fc700 1 -- 172.21.15.93:0/1076665460 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9ce8004e20 con 0x7f9cec10b970 2023-07-21T22:32:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.742+0000 7f9ce27fc700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 0x7f9cd80a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.742+0000 7f9ce27fc700 1 -- 172.21.15.93:0/1076665460 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9ce80cdc60 con 0x7f9cec10b970 2023-07-21T22:32:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.743+0000 7f9cff8f8b80 1 Processor -- start 2023-07-21T22:32:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.743+0000 7f9cff8f8b80 1 -- start start 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.743+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c289f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c28a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c18fc50 con 0x56432c290660 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c1e29e0 con 0x56432c0e66b0 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c1dbe70 con 0x56432c290210 2023-07-21T22:32:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf150f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf150f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59882/0 (socket says 172.21.15.93:59882) 2023-07-21T22:32:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf150f700 1 -- 172.21.15.93:0/4249231527 learned_addr learned my addr 172.21.15.93:0/4249231527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c28a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf0d0e700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 0x7f9cd80a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf0d0e700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c289f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4249231527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 msgr2=0x56432c289f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c289f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4249231527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 msgr2=0x56432c28fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4249231527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56432c009d80 con 0x56432c290660 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.744+0000 7f9cf150f700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.745+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c28a4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00ac00 tx=0x7f9cdc00af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf3723f71d8f7fc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.745+0000 7f9cf0d0e700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 0x7f9cd80a3820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009420 tx=0x7f9ce4009630 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.745+0000 7f9cd77fe700 1 -- 172.21.15.93:0/4249231527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004230 con 0x56432c290660 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.745+0000 7f9cd77fe700 1 -- 172.21.15.93:0/4249231527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc00edc0 con 0x56432c290660 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.745+0000 7f9cd77fe700 1 -- 172.21.15.93:0/4249231527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc005020 con 0x56432c290660 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4249231527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 msgr2=0x56432c28a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c28a4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00ac00 tx=0x7f9cdc00af10 comp rx=0 tx=0).stop 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4249231527 shutdown_connections 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c28a4a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c28fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4249231527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c289f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4249231527 >> 172.21.15.93:0/4249231527 conn(0x56432c27fae0 msgr2=0x56432c2850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4249231527 shutdown_connections 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.746+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4249231527 wait complete. 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.747+0000 7f9cff8f8b80 1 -- start start 2023-07-21T22:32:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.747+0000 7f9cff8f8b80 1 Processor -- start 2023-07-21T22:32:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c296530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c293740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c17ba60 con 0x56432c0e66b0 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c17b760 con 0x56432c290660 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c05f8c0 con 0x56432c290210 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cff8f8b80 10 client.0 _ll_register_callbacks cb 0x56432c179700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c293740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c293740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35808/0 (socket says 172.21.15.93:35808) 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59892/0 (socket says 172.21.15.93:59892) 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf0d0e700 1 -- 172.21.15.93:0/2137359239 learned_addr learned my addr 172.21.15.93:0/2137359239 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf150f700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c296530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 -- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 msgr2=0x56432c293740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c293740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 -- 172.21.15.93:0/2137359239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 msgr2=0x56432c296530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c296530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 -- 172.21.15.93:0/2137359239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec1a1130 con 0x56432c290660 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e6d0 tx=0x7f9cdc0048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bb094091d1229 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.748+0000 7f9cd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc005430 con 0x56432c290660 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9cdc01d040 con 0x56432c290660 2023-07-21T22:32:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56432c135490 con 0x56432c290660 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cdc004940 con 0x56432c290660 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cff8f8b80 10 client.26786 fetch_fsmap learned FSMap version 74 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f9cdc00a8e0 con 0x56432c290660 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.749+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56432c135490 con 0x56432c290660 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 74) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f9cdc028050 con 0x56432c290660 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 10 client.26786 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 10 client.26786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 20 client.26786 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2a3a30 0x56432c2a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 4 client.26786 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56432c1d4e50 con 0x56432c2a3a30 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cf150f700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2a3a30 0x56432c2a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cd4ff9700 10 client.26786 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.750+0000 7f9cf150f700 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2a3a30 0x56432c2a5e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472cb521739dcfca server_cookie=a8c680bf5d45f02b in_seq=0 out_seq=0 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/2137359239 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56432c1d4e50 con 0x56432c2a3a30 2023-07-21T22:32:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cd4ff9700 10 client.26786 handle_command_reply: tid=0 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 client.26786 shutdown 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 msgr2=0x56432c293d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e6d0 tx=0x7f9cdc0048a0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 shutdown_connections 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2a3a30 0x56432c2a5e40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c293d00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c296530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/2137359239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c293740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 >> 172.21.15.93:0/2137359239 conn(0x56432c27fae0 msgr2=0x56432c2843e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.751+0000 7f9cd4ff9700 0 client.26786 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.752+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 shutdown_connections 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.752+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/2137359239 wait complete. 2023-07-21T22:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.752+0000 7f9cff8f8b80 20 client.26786 trim_cache size 0 max 16384 2023-07-21T22:32:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.773+0000 7f9cff8f8b80 1 Processor -- start 2023-07-21T22:32:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.773+0000 7f9cff8f8b80 1 -- start start 2023-07-21T22:32:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c29ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c292f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c286760 con 0x56432c290660 2023-07-21T22:32:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c291ca0 con 0x56432c0e66b0 2023-07-21T22:32:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.774+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c28e380 con 0x56432c290210 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48736/0 (socket says 172.21.15.93:48736) 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4122986906 learned_addr learned my addr 172.21.15.93:0/4122986906 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf0d0e700 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c292f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf150f700 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c29ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4122986906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 msgr2=0x56432c292f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c292f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4122986906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 msgr2=0x56432c29ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c29ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.776+0000 7f9cf1d10700 1 -- 172.21.15.93:0/4122986906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56432c2aaa70 con 0x56432c290660 2023-07-21T22:32:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.777+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000c00 tx=0x7f9cdc00a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2034fe816d8e389f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.777+0000 7f9cd7fff700 1 -- 172.21.15.93:0/4122986906 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc030070 con 0x56432c290660 2023-07-21T22:32:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.777+0000 7f9cd7fff700 1 -- 172.21.15.93:0/4122986906 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc00a250 con 0x56432c290660 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.777+0000 7f9cd7fff700 1 -- 172.21.15.93:0/4122986906 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc021810 con 0x56432c290660 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4122986906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 msgr2=0x56432c296060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000c00 tx=0x7f9cdc00a6d0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4122986906 shutdown_connections 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c290660 0x56432c296060 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c0e66b0 0x56432c29ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/4122986906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c292f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4122986906 >> 172.21.15.93:0/4122986906 conn(0x56432c27fae0 msgr2=0x56432c293700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4122986906 shutdown_connections 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/4122986906 wait complete. 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 -- start start 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.778+0000 7f9cff8f8b80 1 Processor -- start 2023-07-21T22:32:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c29ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c28b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c28bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c28c240 con 0x56432c0e66b0 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c28c3b0 con 0x56432c290660 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56432c28c570 con 0x56432c290210 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cff8f8b80 10 client.0 _ll_register_callbacks cb 0x56432c179700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c28b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cf0d0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c28b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35824/0 (socket says 172.21.15.93:35824) 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cf0d0e700 1 -- 172.21.15.93:0/911280437 learned_addr learned my addr 172.21.15.93:0/911280437 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cf150f700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c29ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.779+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c28bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 -- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 msgr2=0x56432c28b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c28b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 -- 172.21.15.93:0/911280437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 msgr2=0x56432c29ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c29ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 -- 172.21.15.93:0/911280437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc00a8e0 con 0x56432c290660 2023-07-21T22:32:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cf1d10700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c28bc70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc004790 tx=0x7f9cdc0054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f18ee491f50672 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:32:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc030070 con 0x56432c290660 2023-07-21T22:32:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cdc00edc0 con 0x56432c290660 2023-07-21T22:32:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9cdc000e40 con 0x56432c290660 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cdc0215d0 con 0x56432c290660 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cff8f8b80 10 client.26792 fetch_fsmap learned FSMap version 74 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.780+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x56432bff2c00 con 0x56432c290660 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc042020 con 0x56432c290660 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 74) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f9cdc00e3f0 con 0x56432c290660 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 10 client.26792 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T22:32:20.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 10 client.26792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 20 client.26792 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2f7ba0 0x56432c2f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 4 client.26792 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:20.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56432c196ff0 con 0x56432c2f7ba0 2023-07-21T22:32:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.781+0000 7f9cf150f700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2f7ba0 0x56432c2f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.782+0000 7f9cd4ff9700 10 client.26792 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:20.782+0000 7f9cf150f700 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2f7ba0 0x56432c2f9f80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6400afa34214dcf0 server_cookie=4a0e0ffd9a40aa56 in_seq=0 out_seq=0 2023-07-21T22:32:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cd4ff9700 1 -- 172.21.15.93:0/911280437 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56432c196ff0 con 0x56432c2f7ba0 2023-07-21T22:32:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cd4ff9700 10 client.26792 handle_command_reply: tid=0 2023-07-21T22:32:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 client.26792 shutdown 2023-07-21T22:32:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 msgr2=0x56432c28bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c28bc70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc004790 tx=0x7f9cdc0054e0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 shutdown_connections 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56432c2f7ba0 0x56432c2f9f80 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56432c290660 0x56432c28bc70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56432c0e66b0 0x56432c29ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cd4ff9700 0 client.26792 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 --2- 172.21.15.93:0/911280437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56432c290210 0x56432c28b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 >> 172.21.15.93:0/911280437 conn(0x56432c27fae0 msgr2=0x56432c290a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 shutdown_connections 2023-07-21T22:32:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 1 -- 172.21.15.93:0/911280437 wait complete. 2023-07-21T22:32:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.007+0000 7f9cff8f8b80 20 client.26792 trim_cache size 0 max 16384 2023-07-21T22:32:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:32:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:32:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:32:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 msgr2=0x7f9cd80a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 0x7f9cd80a3820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009420 tx=0x7f9ce4009630 comp rx=0 tx=0).stop 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 msgr2=0x7f9cec1a3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009990 tx=0x7f9ce800bae0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 shutdown_connections 2023-07-21T22:32:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9cd80a1370 0x7f9cd80a3820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cec10cff0 0x7f9cec19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cec10b970 0x7f9cec1a3310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 --2- 172.21.15.93:0/1076665460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9cec10c640 0x7f9cec1a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 >> 172.21.15.93:0/1076665460 conn(0x7f9cec06bd70 msgr2=0x7f9cec109630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 shutdown_connections 2023-07-21T22:32:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:21.008+0000 7f9cf3773700 1 -- 172.21.15.93:0/1076665460 wait complete. 2023-07-21T22:32:21.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 117493 37.33 MB/sec execute 57 sec latency 140.242 ms 2023-07-21T22:32:21.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:21 smithi194 ceph-mon[98288]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:21.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:21 smithi134 ceph-mon[97873]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:21 smithi093 ceph-mon[96579]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 28 MiB/s wr, 1.28k op/s 2023-07-21T22:32:22.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 118059 37.11 MB/sec execute 58 sec latency 78.493 ms 2023-07-21T22:32:23.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 118924 37.14 MB/sec execute 59 sec latency 54.294 ms 2023-07-21T22:32:23.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:23 smithi194 ceph-mon[98288]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-07-21T22:32:23.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:23 smithi134 ceph-mon[97873]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-07-21T22:32:23.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:23 smithi093 ceph-mon[96579]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-07-21T22:32:24.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 119742 37.24 MB/sec execute 60 sec latency 88.578 ms 2023-07-21T22:32:25.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 120605 37.68 MB/sec execute 61 sec latency 69.152 ms 2023-07-21T22:32:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:25 smithi194 ceph-mon[98288]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-07-21T22:32:25.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:25 smithi134 ceph-mon[97873]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-07-21T22:32:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:25 smithi093 ceph-mon[96579]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-07-21T22:32:26.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 121196 37.41 MB/sec execute 62 sec latency 48.077 ms 2023-07-21T22:32:27.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 121919 37.43 MB/sec execute 63 sec latency 52.752 ms 2023-07-21T22:32:27.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:27 smithi194 ceph-mon[98288]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 811 op/s 2023-07-21T22:32:27.631 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:27 smithi134 ceph-mon[97873]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 811 op/s 2023-07-21T22:32:27.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:27 smithi093 ceph-mon[96579]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 6.3 MiB/s wr, 811 op/s 2023-07-21T22:32:28.507 INFO:tasks.workunit.client.0.smithi093.stdout: 10 122898 37.58 MB/sec execute 64 sec latency 69.237 ms 2023-07-21T22:32:29.508 INFO:tasks.workunit.client.0.smithi093.stdout: 10 123691 37.85 MB/sec execute 65 sec latency 84.606 ms 2023-07-21T22:32:29.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:29 smithi194 ceph-mon[98288]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-07-21T22:32:29.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:29 smithi134 ceph-mon[97873]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-07-21T22:32:29.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:29 smithi093 ceph-mon[96579]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-07-21T22:32:30.508 INFO:tasks.workunit.client.0.smithi093.stdout: 10 124356 37.82 MB/sec execute 66 sec latency 36.838 ms 2023-07-21T22:32:30.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:30 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:30.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:30 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:30.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:30 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:31.509 INFO:tasks.workunit.client.0.smithi093.stdout: 10 124886 37.56 MB/sec execute 67 sec latency 165.104 ms 2023-07-21T22:32:31.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:31 smithi134 ceph-mon[97873]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 587 op/s 2023-07-21T22:32:31.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:31 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:31.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:31 smithi093 ceph-mon[96579]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 587 op/s 2023-07-21T22:32:31.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:31 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:31.845 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:31 smithi194 ceph-mon[98288]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 587 op/s 2023-07-21T22:32:31.845 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:31 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:32.509 INFO:tasks.workunit.client.0.smithi093.stdout: 10 125045 37.17 MB/sec execute 68 sec latency 744.051 ms 2023-07-21T22:32:33.509 INFO:tasks.workunit.client.0.smithi093.stdout: 10 125594 37.01 MB/sec execute 69 sec latency 87.195 ms 2023-07-21T22:32:33.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:33 smithi134 ceph-mon[97873]: pgmap v512: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 16 MiB/s wr, 864 op/s 2023-07-21T22:32:33.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:33 smithi093 ceph-mon[96579]: pgmap v512: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 16 MiB/s wr, 864 op/s 2023-07-21T22:32:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:33 smithi194 ceph-mon[98288]: pgmap v512: 97 pgs: 97 active+clean; 1.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 16 MiB/s wr, 864 op/s 2023-07-21T22:32:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 -- 172.21.15.93:0/3292924611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c0ff3c0 msgr2=0x7f400c10b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 --2- 172.21.15.93:0/3292924611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c0ff3c0 0x7f400c10b850 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4004009f90 tx=0x7f400400b330 comp rx=0 tx=0).stop 2023-07-21T22:32:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 -- 172.21.15.93:0/3292924611 shutdown_connections 2023-07-21T22:32:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 --2- 172.21.15.93:0/3292924611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c0ff3c0 0x7f400c10b850 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 --2- 172.21.15.93:0/3292924611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f400c104a20 0x7f400c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 --2- 172.21.15.93:0/3292924611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 -- 172.21.15.93:0/3292924611 >> 172.21.15.93:0/3292924611 conn(0x7f400c0fa4f0 msgr2=0x7f400c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 -- 172.21.15.93:0/3292924611 shutdown_connections 2023-07-21T22:32:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.146+0000 7f401358c700 1 -- 172.21.15.93:0/3292924611 wait complete. 2023-07-21T22:32:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 Processor -- start 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 -- start start 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f400c0ff3c0 0x7f400c194f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 0x7f400c1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c195690 con 0x7f400c104a20 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a43e0 con 0x7f400c0fea10 2023-07-21T22:32:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.147+0000 7f401358c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c1a4570 con 0x7f400c0ff3c0 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011b29700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 0x7f400c1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59918/0 (socket says 172.21.15.93:59918) 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 -- 172.21.15.93:0/3417388633 learned_addr learned my addr 172.21.15.93:0/3417388633 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 -- 172.21.15.93:0/3417388633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f400c0ff3c0 msgr2=0x7f400c194f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f400c0ff3c0 0x7f400c194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 -- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 msgr2=0x7f400c1a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 0x7f400c1a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 -- 172.21.15.93:0/3417388633 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009c40 con 0x7f400c0fea10 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011b29700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 0x7f400c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f4011328700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9f0 tx=0x7f400800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f40027fc700 1 -- 172.21.15.93:0/3417388633 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008004220 con 0x7f400c0fea10 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f40027fc700 1 -- 172.21.15.93:0/3417388633 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4008004380 con 0x7f400c0fea10 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c1a4850 con 0x7f400c0fea10 2023-07-21T22:32:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f40027fc700 1 -- 172.21.15.93:0/3417388633 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008003c00 con 0x7f400c0fea10 2023-07-21T22:32:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.148+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a4db0 con 0x7f400c0fea10 2023-07-21T22:32:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.151+0000 7f40027fc700 1 -- 172.21.15.93:0/3417388633 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4008004870 con 0x7f400c0fea10 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.152+0000 7f40027fc700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 0x7f3ff80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.152+0000 7f4010b27700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 0x7f3ff80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.152+0000 7f40027fc700 1 -- 172.21.15.93:0/3417388633 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4008014070 con 0x7f400c0fea10 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.152+0000 7f4010b27700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 0x7f3ff80a37f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005950 tx=0x7f3ffc0058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 Processor -- start 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 -- start start 2023-07-21T22:32:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70de83c50 con 0x55c70df84210 2023-07-21T22:32:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70ded69e0 con 0x55c70ddda6b0 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f401f711b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70decfe70 con 0x55c70df84660 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f4011328700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f4011328700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59926/0 (socket says 172.21.15.93:59926) 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f4011328700 1 -- 172.21.15.93:0/1446812561 learned_addr learned my addr 172.21.15.93:0/1446812561 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.153+0000 7f4010b27700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011b29700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 -- 172.21.15.93:0/1446812561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 msgr2=0x55c70df7e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df7e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 -- 172.21.15.93:0/1446812561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 msgr2=0x55c70df7df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df7df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 -- 172.21.15.93:0/1446812561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c70dcfdd80 con 0x55c70ddda6b0 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011b29700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.154+0000 7f4011328700 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4008095db0 tx=0x7f40080044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8c263e6c186dec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f3ff77fe700 1 -- 172.21.15.93:0/1446812561 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40080a1d00 con 0x55c70ddda6b0 2023-07-21T22:32:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f3ff77fe700 1 -- 172.21.15.93:0/1446812561 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4008095fc0 con 0x55c70ddda6b0 2023-07-21T22:32:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f3ff77fe700 1 -- 172.21.15.93:0/1446812561 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008098da0 con 0x55c70ddda6b0 2023-07-21T22:32:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 -- 172.21.15.93:0/1446812561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 msgr2=0x55c70df83cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4008095db0 tx=0x7f40080044f0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 -- 172.21.15.93:0/1446812561 shutdown_connections 2023-07-21T22:32:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df83cd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 --2- 172.21.15.93:0/1446812561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df7df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.155+0000 7f401f711b80 1 -- 172.21.15.93:0/1446812561 >> 172.21.15.93:0/1446812561 conn(0x55c70df73ae0 msgr2=0x55c70df790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.156+0000 7f401f711b80 1 -- 172.21.15.93:0/1446812561 shutdown_connections 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.156+0000 7f401f711b80 1 -- 172.21.15.93:0/1446812561 wait complete. 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.156+0000 7f401f711b80 1 -- start start 2023-07-21T22:32:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.156+0000 7f401f711b80 1 Processor -- start 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.156+0000 7f401f711b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df87ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70de6fa60 con 0x55c70df84210 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70de6f760 con 0x55c70ddda6b0 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70dd538c0 con 0x55c70df84660 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 10 client.0 _ll_register_callbacks cb 0x55c70de6d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4011328700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48772/0 (socket says 172.21.15.93:48772) 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 -- 172.21.15.93:0/3109180302 learned_addr learned my addr 172.21.15.93:0/3109180302 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4011b29700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df87ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 -- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 msgr2=0x55c70df87ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df87ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 -- 172.21.15.93:0/3109180302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 msgr2=0x55c70df8a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df8a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 -- 172.21.15.93:0/3109180302 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1a4db0 con 0x55c70df84210 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4011328700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df8a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f4010b27700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc01d6e0 tx=0x7f3ffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a53a95b0bda3a9d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f3ff4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc004a30 con 0x55c70df84210 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3ffc004b90 con 0x55c70df84210 2023-07-21T22:32:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.157+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c70de29490 con 0x55c70df84210 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.158+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ffc00eec0 con 0x55c70df84210 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.158+0000 7f401f711b80 10 client.17424 fetch_fsmap learned FSMap version 76 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.158+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55c70dce6c00 con 0x55c70df84210 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.158+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c70de29490 con 0x55c70df84210 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 76) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ffc018070 con 0x55c70df84210 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 10 client.17424 fetch_fsmap finished waiting for FSMap version 76 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 20 client.17424 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dfa0320 0x55c70dfa2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 4 client.17424 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c70dec8e50 con 0x55c70dfa0320 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f4011328700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dfa0320 0x55c70dfa2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f3ff4ff9700 10 client.17424 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.159+0000 7f4011328700 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dfa0320 0x55c70dfa2730 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae217b5177d289b server_cookie=304489ad86444976 in_seq=0 out_seq=0 2023-07-21T22:32:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.161+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/3109180302 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c70dec8e50 con 0x55c70dfa0320 2023-07-21T22:32:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.161+0000 7f3ff4ff9700 10 client.17424 handle_command_reply: tid=0 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 client.17424 shutdown 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 msgr2=0x55c70df878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc01d6e0 tx=0x7f3ffc004750 comp rx=0 tx=0).stop 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 shutdown_connections 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dfa0320 0x55c70dfa2730 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84660 0x55c70df87ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f3ff4ff9700 0 client.17424 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70ddda6b0 0x55c70df8a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 --2- 172.21.15.93:0/3109180302 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70df84210 0x55c70df878b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 >> 172.21.15.93:0/3109180302 conn(0x55c70df73ae0 msgr2=0x55c70df783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 shutdown_connections 2023-07-21T22:32:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 1 -- 172.21.15.93:0/3109180302 wait complete. 2023-07-21T22:32:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.162+0000 7f401f711b80 20 client.17424 trim_cache size 0 max 16384 2023-07-21T22:32:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 Processor -- start 2023-07-21T22:32:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 -- start start 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df9da50 con 0x55c70ddda6b0 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df79cb0 con 0x55c70df84660 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.179+0000 7f401f711b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df7ea30 con 0x55c70df84210 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48778/0 (socket says 172.21.15.93:48778) 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 -- 172.21.15.93:0/1788294304 learned_addr learned my addr 172.21.15.93:0/1788294304 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011b29700 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4010b27700 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 -- 172.21.15.93:0/1788294304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 msgr2=0x55c70df83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 -- 172.21.15.93:0/1788294304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 msgr2=0x55c70df851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 -- 172.21.15.93:0/1788294304 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c70dd241f0 con 0x55c70ddda6b0 2023-07-21T22:32:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f4011328700 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f40080cc900 tx=0x7f4008078040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887b4340870dc9af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f3ff7fff700 1 -- 172.21.15.93:0/1788294304 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40080b20c0 con 0x55c70ddda6b0 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f3ff7fff700 1 -- 172.21.15.93:0/1788294304 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4008096130 con 0x55c70ddda6b0 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.180+0000 7f3ff7fff700 1 -- 172.21.15.93:0/1788294304 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008099000 con 0x55c70ddda6b0 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 -- 172.21.15.93:0/1788294304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 msgr2=0x55c70df9c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f40080cc900 tx=0x7f4008078040 comp rx=0 tx=0).stop 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 -- 172.21.15.93:0/1788294304 shutdown_connections 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df9c880 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 --2- 172.21.15.93:0/1788294304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 -- 172.21.15.93:0/1788294304 >> 172.21.15.93:0/1788294304 conn(0x55c70df73ae0 msgr2=0x55c70df84aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 -- 172.21.15.93:0/1788294304 shutdown_connections 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.181+0000 7f401f711b80 1 -- 172.21.15.93:0/1788294304 wait complete. 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 -- start start 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 Processor -- start 2023-07-21T22:32:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df80eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df814b0 con 0x55c70ddda6b0 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df7fc70 con 0x55c70df84660 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70df7fe30 con 0x55c70df84210 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f401f711b80 10 client.0 _ll_register_callbacks cb 0x7f3ffc00ed20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.182+0000 7f4011328700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48784/0 (socket says 172.21.15.93:48784) 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 -- 172.21.15.93:0/423933626 learned_addr learned my addr 172.21.15.93:0/423933626 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4010b27700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df80eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011b29700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 -- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 msgr2=0x55c70df80eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df80eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 -- 172.21.15.93:0/423933626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 msgr2=0x55c70df7f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df7f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 -- 172.21.15.93:0/423933626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c70dcfdd80 con 0x55c70ddda6b0 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f4011328700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f400800c5b0 tx=0x7f4008009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed41c67b3d98160e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f3ff4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40080b7c40 con 0x55c70ddda6b0 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4008099b10 con 0x55c70ddda6b0 2023-07-21T22:32:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4008095fc0 con 0x55c70ddda6b0 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4008099cd0 con 0x55c70ddda6b0 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f401f711b80 10 client.17430 fetch_fsmap learned FSMap version 76 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.183+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x55c70dcfc250 con 0x55c70ddda6b0 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4008095fc0 con 0x55c70ddda6b0 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 76) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f400807e020 con 0x55c70ddda6b0 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 76 2023-07-21T22:32:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 10 client.17430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 20 client.17430 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dff4a30 0x55c70dff6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 4 client.17430 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c70de8aff0 con 0x55c70dff4a30 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.184+0000 7f4010b27700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dff4a30 0x55c70dff6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.185+0000 7f3ff4ff9700 10 client.17430 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.185+0000 7f4010b27700 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dff4a30 0x55c70dff6e10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f835d4b34e0355 server_cookie=4f5750af1d883791 in_seq=0 out_seq=0 2023-07-21T22:32:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f3ff4ff9700 1 -- 172.21.15.93:0/423933626 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c70de8aff0 con 0x55c70dff4a30 2023-07-21T22:32:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f3ff4ff9700 10 client.17430 handle_command_reply: tid=0 2023-07-21T22:32:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 client.17430 shutdown 2023-07-21T22:32:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 msgr2=0x55c70df80970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f400800c5b0 tx=0x7f4008009530 comp rx=0 tx=0).stop 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 shutdown_connections 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c70dff4a30 0x55c70dff6e10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c70df84660 0x55c70df7f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c70ddda6b0 0x55c70df80970 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 --2- 172.21.15.93:0/423933626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c70df84210 0x55c70df80eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 >> 172.21.15.93:0/423933626 conn(0x55c70df73ae0 msgr2=0x55c70df84aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f3ff4ff9700 0 client.17430 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 shutdown_connections 2023-07-21T22:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 1 -- 172.21.15.93:0/423933626 wait complete. 2023-07-21T22:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.346+0000 7f401f711b80 20 client.17430 trim_cache size 0 max 16384 2023-07-21T22:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:32:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:32:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:32:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:32:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 msgr2=0x7f3ff80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 0x7f3ff80a37f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005950 tx=0x7f3ffc0058e0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 msgr2=0x7f400c194a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9f0 tx=0x7f400800ed00 comp rx=0 tx=0).stop 2023-07-21T22:32:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 shutdown_connections 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3ff80a1340 0x7f3ff80a37f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f400c104a20 0x7f400c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0fea10 0x7f400c194a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 --2- 172.21.15.93:0/3417388633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f400c0ff3c0 0x7f400c194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 >> 172.21.15.93:0/3417388633 conn(0x7f400c0fa4f0 msgr2=0x7f400c105870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 shutdown_connections 2023-07-21T22:32:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:34.347+0000 7f401358c700 1 -- 172.21.15.93:0/3417388633 wait complete. 2023-07-21T22:32:34.509 INFO:tasks.workunit.client.0.smithi093.stdout: 10 126490 37.10 MB/sec execute 70 sec latency 91.341 ms 2023-07-21T22:32:35.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 127295 37.36 MB/sec execute 71 sec latency 76.371 ms 2023-07-21T22:32:35.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:35 smithi134 ceph-mon[97873]: pgmap v513: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 741 op/s 2023-07-21T22:32:35.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:35 smithi093 ceph-mon[96579]: pgmap v513: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 741 op/s 2023-07-21T22:32:35.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:35 smithi194 ceph-mon[98288]: pgmap v513: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 741 op/s 2023-07-21T22:32:36.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 127955 37.33 MB/sec execute 72 sec latency 58.184 ms 2023-07-21T22:32:37.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 128694 37.31 MB/sec execute 73 sec latency 44.035 ms 2023-07-21T22:32:37.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:37 smithi134 ceph-mon[97873]: pgmap v514: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 668 op/s 2023-07-21T22:32:37.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:37 smithi093 ceph-mon[96579]: pgmap v514: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 668 op/s 2023-07-21T22:32:37.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:37 smithi194 ceph-mon[98288]: pgmap v514: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 17 MiB/s wr, 668 op/s 2023-07-21T22:32:38.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 129647 37.44 MB/sec execute 74 sec latency 51.810 ms 2023-07-21T22:32:39.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 130410 37.56 MB/sec execute 75 sec latency 43.665 ms 2023-07-21T22:32:39.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:39 smithi134 ceph-mon[97873]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2023-07-21T22:32:39.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:39 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:39.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:39 smithi093 ceph-mon[96579]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2023-07-21T22:32:39.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:39 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:39.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:39 smithi194 ceph-mon[98288]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2023-07-21T22:32:39.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:39 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:40.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 131164 37.61 MB/sec execute 76 sec latency 50.840 ms 2023-07-21T22:32:41.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 131796 37.54 MB/sec execute 77 sec latency 132.939 ms 2023-07-21T22:32:41.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:41 smithi134 ceph-mon[97873]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 833 op/s 2023-07-21T22:32:41.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:41 smithi093 ceph-mon[96579]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 833 op/s 2023-07-21T22:32:41.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:41 smithi194 ceph-mon[98288]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 833 op/s 2023-07-21T22:32:42.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 132728 37.65 MB/sec execute 78 sec latency 52.441 ms 2023-07-21T22:32:43.510 INFO:tasks.workunit.client.0.smithi093.stdout: 10 133428 37.79 MB/sec execute 79 sec latency 46.580 ms 2023-07-21T22:32:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:43 smithi134 ceph-mon[97873]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-07-21T22:32:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:43 smithi093 ceph-mon[96579]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-07-21T22:32:43.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:43 smithi194 ceph-mon[98288]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2023-07-21T22:32:44.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 134136 37.71 MB/sec execute 80 sec latency 56.905 ms 2023-07-21T22:32:45.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 134885 37.70 MB/sec execute 81 sec latency 46.465 ms 2023-07-21T22:32:45.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:45 smithi134 ceph-mon[97873]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 820 op/s 2023-07-21T22:32:45.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:45 smithi093 ceph-mon[96579]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 820 op/s 2023-07-21T22:32:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:45 smithi194 ceph-mon[98288]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 820 op/s 2023-07-21T22:32:46.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 135766 37.92 MB/sec execute 82 sec latency 43.790 ms 2023-07-21T22:32:47.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 136464 37.88 MB/sec execute 83 sec latency 69.912 ms 2023-07-21T22:32:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 -- 172.21.15.93:0/2215344604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 msgr2=0x7f7b1410b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/2215344604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b1410b860 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00b0a0 tx=0x7f7b0c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 -- 172.21.15.93:0/2215344604 shutdown_connections 2023-07-21T22:32:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/2215344604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b1410b860 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/2215344604 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b14104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/2215344604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b140feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.493+0000 7f7b18ca0700 1 -- 172.21.15.93:0/2215344604 >> 172.21.15.93:0/2215344604 conn(0x7f7b14074760 msgr2=0x7f7b14074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.495+0000 7f7b18ca0700 1 -- 172.21.15.93:0/2215344604 shutdown_connections 2023-07-21T22:32:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.495+0000 7f7b18ca0700 1 -- 172.21.15.93:0/2215344604 wait complete. 2023-07-21T22:32:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 Processor -- start 2023-07-21T22:32:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 -- start start 2023-07-21T22:32:47.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b141a8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14106690 con 0x7f7b140ff3e0 2023-07-21T22:32:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141a8da0 con 0x7f7b140fea30 2023-07-21T22:32:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b18ca0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b141a8ee0 con 0x7f7b14104a30 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59960/0 (socket says 172.21.15.93:59960) 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48800/0 (socket says 172.21.15.93:48800) 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b1259c700 1 -- 172.21.15.93:0/561274133 learned_addr learned my addr 172.21.15.93:0/561274133 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.496+0000 7f7b12d9d700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b141a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 -- 172.21.15.93:0/561274133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 msgr2=0x7f7b141a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b141a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 -- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 msgr2=0x7f7b14105e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 -- 172.21.15.93:0/561274133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c009d00 con 0x7f7b140fea30 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b12d9d700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b141a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b1259c700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b530 tx=0x7f7b0400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b037fe700 1 -- 172.21.15.93:0/561274133 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011830 con 0x7f7b140fea30 2023-07-21T22:32:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b141a9140 con 0x7f7b140fea30 2023-07-21T22:32:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b037fe700 1 -- 172.21.15.93:0/561274133 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b04011e50 con 0x7f7b140fea30 2023-07-21T22:32:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b037fe700 1 -- 172.21.15.93:0/561274133 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400f2e0 con 0x7f7b140fea30 2023-07-21T22:32:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.497+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b141a96a0 con 0x7f7b140fea30 2023-07-21T22:32:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.502+0000 7f7b24e25b80 1 Processor -- start 2023-07-21T22:32:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.502+0000 7f7b24e25b80 1 -- start start 2023-07-21T22:32:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e281e6b0 0x55b9e29c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e28c7c50 con 0x55b9e29c8660 2023-07-21T22:32:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e291a9e0 con 0x55b9e29c8210 2023-07-21T22:32:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e2913e70 con 0x55b9e281e6b0 2023-07-21T22:32:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48804/0 (socket says 172.21.15.93:48804) 2023-07-21T22:32:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 -- 172.21.15.93:0/4118409973 learned_addr learned my addr 172.21.15.93:0/4118409973 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 -- 172.21.15.93:0/4118409973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e281e6b0 msgr2=0x55b9e29c7cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e281e6b0 0x55b9e29c7cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 -- 172.21.15.93:0/4118409973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 msgr2=0x55b9e29c1f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.503+0000 7f7b11d9b700 1 -- 172.21.15.93:0/4118409973 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e2741d80 con 0x55b9e29c8660 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.504+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e90 tx=0x7f7b0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3362df4272d697a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.504+0000 7f7b00ff9700 1 -- 172.21.15.93:0/4118409973 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800ecd0 con 0x55b9e29c8660 2023-07-21T22:32:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.504+0000 7f7b00ff9700 1 -- 172.21.15.93:0/4118409973 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b0800ee30 con 0x55b9e29c8660 2023-07-21T22:32:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.504+0000 7f7b037fe700 1 -- 172.21.15.93:0/561274133 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b04010580 con 0x7f7b140fea30 2023-07-21T22:32:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.504+0000 7f7b00ff9700 1 -- 172.21.15.93:0/4118409973 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080127a0 con 0x55b9e29c8660 2023-07-21T22:32:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b037fe700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 0x7f7afc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 -- 172.21.15.93:0/4118409973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 msgr2=0x55b9e29c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e90 tx=0x7f7b0800e4b0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b037fe700 1 -- 172.21.15.93:0/561274133 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b040cc6b0 con 0x7f7b140fea30 2023-07-21T22:32:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 -- 172.21.15.93:0/4118409973 shutdown_connections 2023-07-21T22:32:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8660 0x55b9e29c24a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e281e6b0 0x55b9e29c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/4118409973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 -- 172.21.15.93:0/4118409973 >> 172.21.15.93:0/4118409973 conn(0x55b9e29b7ae0 msgr2=0x55b9e29bd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 -- 172.21.15.93:0/4118409973 shutdown_connections 2023-07-21T22:32:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b12d9d700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 0x7f7afc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.505+0000 7f7b24e25b80 1 -- 172.21.15.93:0/4118409973 wait complete. 2023-07-21T22:32:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 -- start start 2023-07-21T22:32:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b12d9d700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 0x7f7afc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009fd0 tx=0x7f7b0c00b7b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 Processor -- start 2023-07-21T22:32:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e28b3a60 con 0x55b9e29c8210 2023-07-21T22:32:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e28b3760 con 0x55b9e281e6b0 2023-07-21T22:32:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.506+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e27978c0 con 0x55b9e29c8660 2023-07-21T22:32:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b24e25b80 10 client.0 _ll_register_callbacks cb 0x55b9e28b1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b12d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:59970/0 (socket says 172.21.15.93:59970) 2023-07-21T22:32:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b12d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35890/0 (socket says 172.21.15.93:35890) 2023-07-21T22:32:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 -- 172.21.15.93:0/2819833135 learned_addr learned my addr 172.21.15.93:0/2819833135 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 -- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 msgr2=0x55b9e29cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 -- 172.21.15.93:0/2819833135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 msgr2=0x55b9e29cb7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cb7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 -- 172.21.15.93:0/2819833135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b08009ac0 con 0x55b9e281e6b0 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bc40 tx=0x7f7b04099460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7034dd777e28e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0409d860 con 0x55b9e281e6b0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b040a1c80 con 0x55b9e281e6b0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.507+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e286d490 con 0x55b9e281e6b0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b04099a90 con 0x55b9e281e6b0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 10 client.26816 fetch_fsmap learned FSMap version 77 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55b9e272ac00 con 0x55b9e281e6b0 2023-07-21T22:32:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9e286d490 con 0x55b9e281e6b0 2023-07-21T22:32:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 77) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7b040b4020 con 0x55b9e281e6b0 2023-07-21T22:32:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 77 2023-07-21T22:32:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 20 client.26816 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e29eca60 0x55b9e29eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 4 client.26816 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.508+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9e290ce50 con 0x55b9e29eca60 2023-07-21T22:32:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.509+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e29eca60 0x55b9e29eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.509+0000 7f7af9ffb700 10 client.26816 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.509+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e29eca60 0x55b9e29eee70 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de1eb661ec493f8 server_cookie=35b4a885f876031f in_seq=0 out_seq=0 2023-07-21T22:32:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.510+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2819833135 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b9e290ce50 con 0x55b9e29eca60 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.510+0000 7f7af9ffb700 10 client.26816 handle_command_reply: tid=0 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.510+0000 7f7b24e25b80 1 client.26816 shutdown 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 msgr2=0x55b9e29ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400bc40 tx=0x7f7b04099460 comp rx=0 tx=0).stop 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 shutdown_connections 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e29eca60 0x55b9e29eee70 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29ce560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7af9ffb700 0 client.26816 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2819833135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 >> 172.21.15.93:0/2819833135 conn(0x55b9e29b7ae0 msgr2=0x55b9e29bc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 shutdown_connections 2023-07-21T22:32:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2819833135 wait complete. 2023-07-21T22:32:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.511+0000 7f7b24e25b80 20 client.26816 trim_cache size 0 max 16384 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 Processor -- start 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 -- start start 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29ccdc0 con 0x55b9e29c8210 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29c6420 con 0x55b9e281e6b0 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.534+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29be9d0 con 0x55b9e29c8660 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b12d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48816/0 (socket says 172.21.15.93:48816) 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 -- 172.21.15.93:0/2140386190 learned_addr learned my addr 172.21.15.93:0/2140386190 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 -- 172.21.15.93:0/2140386190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 msgr2=0x55b9e29ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 -- 172.21.15.93:0/2140386190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 msgr2=0x55b9e29d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 -- 172.21.15.93:0/2140386190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9e27681f0 con 0x55b9e29c8210 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e90 tx=0x7f7b08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b074106ee5f84f46 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b017fa700 1 -- 172.21.15.93:0/2140386190 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08022070 con 0x55b9e29c8210 2023-07-21T22:32:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b017fa700 1 -- 172.21.15.93:0/2140386190 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b08006c10 con 0x55b9e29c8210 2023-07-21T22:32:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.535+0000 7f7b017fa700 1 -- 172.21.15.93:0/2140386190 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08026780 con 0x55b9e29c8210 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2140386190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 msgr2=0x55b9e29cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e90 tx=0x7f7b08000bc0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2140386190 shutdown_connections 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e281e6b0 0x55b9e29d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2140386190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e29c8210 0x55b9e29cafc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2140386190 >> 172.21.15.93:0/2140386190 conn(0x55b9e29b7ae0 msgr2=0x55b9e29c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2140386190 shutdown_connections 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.536+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2140386190 wait complete. 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 -- start start 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 Processor -- start 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29c41b0 con 0x55b9e281e6b0 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29c4320 con 0x55b9e29c8210 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9e29c4460 con 0x55b9e29c8660 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b24e25b80 10 client.0 _ll_register_callbacks cb 0x55b9e28b1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b1259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48820/0 (socket says 172.21.15.93:48820) 2023-07-21T22:32:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.537+0000 7f7b1259c700 1 -- 172.21.15.93:0/2969260735 learned_addr learned my addr 172.21.15.93:0/2969260735 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 -- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 msgr2=0x55b9e29c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 -- 172.21.15.93:0/2969260735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 msgr2=0x55b9e29c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 -- 172.21.15.93:0/2969260735 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9e2741d80 con 0x55b9e281e6b0 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7b1259c700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7b040a1750 tx=0x7f7b04010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35365e80344f8ca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7af9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.538+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040a1ab0 con 0x55b9e281e6b0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9e28ceff0 con 0x55b9e281e6b0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7b040a1c80 con 0x55b9e281e6b0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b04010ee0 con 0x55b9e281e6b0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 10 client.17445 fetch_fsmap learned FSMap version 77 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55b9e2740250 con 0x55b9e281e6b0 2023-07-21T22:32:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b04063080 con 0x55b9e281e6b0 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 77) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7b040b4070 con 0x55b9e281e6b0 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 10 client.17445 fetch_fsmap finished waiting for FSMap version 77 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 20 client.17445 populate_metadata read hostname 'smithi093' 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e2a52830 0x55b9e2a54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 4 client.17445 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9e2762f10 con 0x55b9e2a52830 2023-07-21T22:32:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.539+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e2a52830 0x55b9e2a54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.540+0000 7f7b11d9b700 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e2a52830 0x55b9e2a54c10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e039e76e725109d server_cookie=68731da0197123a2 in_seq=0 out_seq=0 2023-07-21T22:32:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.540+0000 7f7af9ffb700 10 client.17445 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:47.637 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:47 smithi134 ceph-mon[97873]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.9 MiB/s wr, 702 op/s 2023-07-21T22:32:47.702 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:47 smithi093 ceph-mon[96579]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.9 MiB/s wr, 702 op/s 2023-07-21T22:32:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7af9ffb700 1 -- 172.21.15.93:0/2969260735 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9e2762f10 con 0x55b9e2a52830 2023-07-21T22:32:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7af9ffb700 10 client.17445 handle_command_reply: tid=0 2023-07-21T22:32:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 client.17445 shutdown 2023-07-21T22:32:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 msgr2=0x55b9e29c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7b040a1750 tx=0x7f7b04010980 comp rx=0 tx=0).stop 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 shutdown_connections 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b9e2a52830 0x55b9e2a54c10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9e29c8660 0x55b9e29c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b9e281e6b0 0x55b9e29c4e30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 --2- 172.21.15.93:0/2969260735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b9e29c8210 0x55b9e29c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 >> 172.21.15.93:0/2969260735 conn(0x55b9e29b7ae0 msgr2=0x55b9e29c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7af9ffb700 0 client.17445 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:32:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 shutdown_connections 2023-07-21T22:32:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 1 -- 172.21.15.93:0/2969260735 wait complete. 2023-07-21T22:32:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.731+0000 7f7b24e25b80 20 client.17445 trim_cache size 0 max 16384 2023-07-21T22:32:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:32:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:32:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:32:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:32:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 msgr2=0x7f7afc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 0x7f7afc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009fd0 tx=0x7f7b0c00b7b0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 msgr2=0x7f7b14105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b530 tx=0x7f7b0400b840 comp rx=0 tx=0).stop 2023-07-21T22:32:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 shutdown_connections 2023-07-21T22:32:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b14104a30 0x7f7b141a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140fea30 0x7f7b14105900 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7afc0a12c0 0x7f7afc0a3770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 --2- 172.21.15.93:0/561274133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b140ff3e0 0x7f7b14105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 >> 172.21.15.93:0/561274133 conn(0x7f7b14074760 msgr2=0x7f7b140fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 shutdown_connections 2023-07-21T22:32:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:32:47.732+0000 7f7b18ca0700 1 -- 172.21.15.93:0/561274133 wait complete. 2023-07-21T22:32:47.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:47 smithi194 ceph-mon[98288]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.9 MiB/s wr, 702 op/s 2023-07-21T22:32:48.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 137047 37.80 MB/sec execute 84 sec latency 162.569 ms 2023-07-21T22:32:49.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 137839 37.76 MB/sec execute 85 sec latency 47.667 ms 2023-07-21T22:32:49.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:49 smithi134 ceph-mon[97873]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 953 op/s 2023-07-21T22:32:49.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:49 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:49.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:49 smithi093 ceph-mon[96579]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 953 op/s 2023-07-21T22:32:49.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:49 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:49 smithi194 ceph-mon[98288]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 953 op/s 2023-07-21T22:32:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:49 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:50.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 138720 37.94 MB/sec execute 86 sec latency 60.231 ms 2023-07-21T22:32:51.511 INFO:tasks.workunit.client.0.smithi093.stdout: 10 139528 38.07 MB/sec execute 87 sec latency 40.929 ms 2023-07-21T22:32:51.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:51 smithi093 ceph-mon[96579]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.4 MiB/s wr, 515 op/s 2023-07-21T22:32:51.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:51 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:51.879 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:51 smithi194 ceph-mon[98288]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.4 MiB/s wr, 515 op/s 2023-07-21T22:32:51.879 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:51 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:51.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:51 smithi134 ceph-mon[97873]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.4 MiB/s wr, 515 op/s 2023-07-21T22:32:51.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:51 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:52.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 140158 37.97 MB/sec execute 88 sec latency 57.425 ms 2023-07-21T22:32:52.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:52 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:52.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:52 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:52.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:52 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:32:52.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:52 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:52.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:52 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:52.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:52 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:32:52.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:52 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:52.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:52 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:52.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:52 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:32:53.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 140998 38.04 MB/sec execute 89 sec latency 41.800 ms 2023-07-21T22:32:53.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:53 smithi194 ceph-mon[98288]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.0 MiB/s wr, 768 op/s 2023-07-21T22:32:53.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:53 smithi134 ceph-mon[97873]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.0 MiB/s wr, 768 op/s 2023-07-21T22:32:53.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:53 smithi093 ceph-mon[96579]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.0 MiB/s wr, 768 op/s 2023-07-21T22:32:54.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 141929 38.15 MB/sec execute 90 sec latency 46.305 ms 2023-07-21T22:32:55.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 142774 38.31 MB/sec execute 91 sec latency 43.970 ms 2023-07-21T22:32:55.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:55 smithi194 ceph-mon[98288]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-07-21T22:32:55.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:55 smithi134 ceph-mon[97873]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-07-21T22:32:55.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:55 smithi093 ceph-mon[96579]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 646 op/s 2023-07-21T22:32:56.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 143531 38.34 MB/sec execute 92 sec latency 44.629 ms 2023-07-21T22:32:57.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 144203 38.25 MB/sec execute 93 sec latency 77.072 ms 2023-07-21T22:32:57.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:57 smithi194 ceph-mon[98288]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 561 op/s 2023-07-21T22:32:57.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:57 smithi134 ceph-mon[97873]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 561 op/s 2023-07-21T22:32:57.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:57 smithi093 ceph-mon[96579]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.0 MiB/s wr, 561 op/s 2023-07-21T22:32:58.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 144674 38.13 MB/sec execute 94 sec latency 445.785 ms 2023-07-21T22:32:59.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 145599 38.27 MB/sec execute 95 sec latency 42.226 ms 2023-07-21T22:32:59.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:32:59 smithi194 ceph-mon[98288]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 832 op/s 2023-07-21T22:32:59.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:32:59 smithi134 ceph-mon[97873]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 832 op/s 2023-07-21T22:32:59.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:32:59 smithi093 ceph-mon[96579]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 832 op/s 2023-07-21T22:33:00.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 146439 38.39 MB/sec execute 96 sec latency 71.493 ms 2023-07-21T22:33:00.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:00 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.877+0000 7feac5032700 1 -- 172.21.15.93:0/3939195525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac0103430 msgr2=0x7feac010b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.877+0000 7feac5032700 1 --2- 172.21.15.93:0/3939195525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac0103430 0x7feac010b000 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feab800c240 tx=0x7feab800c550 comp rx=0 tx=0).stop 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 -- 172.21.15.93:0/3939195525 shutdown_connections 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 --2- 172.21.15.93:0/3939195525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac0103430 0x7feac010b000 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 --2- 172.21.15.93:0/3939195525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fe3f0 0x7feac00fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 --2- 172.21.15.93:0/3939195525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac00fed90 0x7feac0102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 -- 172.21.15.93:0/3939195525 >> 172.21.15.93:0/3939195525 conn(0x7feac0075f90 msgr2=0x7feac0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 -- 172.21.15.93:0/3939195525 shutdown_connections 2023-07-21T22:33:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.878+0000 7feac5032700 1 -- 172.21.15.93:0/3939195525 wait complete. 2023-07-21T22:33:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 Processor -- start 2023-07-21T22:33:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 -- start start 2023-07-21T22:33:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fed90 0x7feac010f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac0103430 0x7feac010f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01131f0 con 0x7feac00fe3f0 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01a88e0 con 0x7feac0103430 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feac5032700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01a8aa0 con 0x7feac00fed90 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feabed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feabed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48834/0 (socket says 172.21.15.93:48834) 2023-07-21T22:33:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feabed9d700 1 -- 172.21.15.93:0/3197863039 learned_addr learned my addr 172.21.15.93:0/3197863039 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feabf59e700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac0103430 0x7feac010f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.879+0000 7feabe59c700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fed90 0x7feac010f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 -- 172.21.15.93:0/3197863039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fed90 msgr2=0x7feac010f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fed90 0x7feac010f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 -- 172.21.15.93:0/3197863039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac0103430 msgr2=0x7feac010f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac0103430 0x7feac010f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 -- 172.21.15.93:0/3197863039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab800ae10 con 0x7feac00fe3f0 2023-07-21T22:33:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feabed9d700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feab000b530 tx=0x7feab000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feaa7fff700 1 -- 172.21.15.93:0/3197863039 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0011830 con 0x7feac00fe3f0 2023-07-21T22:33:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feaa7fff700 1 -- 172.21.15.93:0/3197863039 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab0011e50 con 0x7feac00fe3f0 2023-07-21T22:33:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feaa7fff700 1 -- 172.21.15.93:0/3197863039 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab000f2e0 con 0x7feac00fe3f0 2023-07-21T22:33:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac01a8d80 con 0x7feac00fe3f0 2023-07-21T22:33:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.880+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a91e0 con 0x7feac00fe3f0 2023-07-21T22:33:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.884+0000 7feaa7fff700 1 -- 172.21.15.93:0/3197863039 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7feab00105a0 con 0x7feac00fe3f0 2023-07-21T22:33:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.884+0000 7feaa7fff700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 0x7feaa80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.884+0000 7feabe59c700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 0x7feaa80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.884+0000 7feaa7fff700 1 -- 172.21.15.93:0/3197863039 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feab00cc510 con 0x7feac00fe3f0 2023-07-21T22:33:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.886+0000 7feabe59c700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 0x7feaa80a37f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feac00767c0 tx=0x7feab4009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 Processor -- start 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 -- start start 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531843bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531843c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x565318330c50 con 0x565318442220 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653183839e0 con 0x565318442670 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7fead11b7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531837ce70 con 0x5653182876b0 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35922/0 (socket says 172.21.15.93:35922) 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7feabed9d700 1 -- 172.21.15.93:0/2192040573 learned_addr learned my addr 172.21.15.93:0/2192040573 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7feabe59c700 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531843bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.888+0000 7feabf59e700 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531843c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 -- 172.21.15.93:0/2192040573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 msgr2=0x56531843c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531843c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 -- 172.21.15.93:0/2192040573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 msgr2=0x56531843bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531843bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 -- 172.21.15.93:0/2192040573 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653181aad80 con 0x5653182876b0 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feabed9d700 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feab0099680 tx=0x7feab0095b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24651b704280e8c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feaa57fa700 1 -- 172.21.15.93:0/2192040573 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab009d720 con 0x5653182876b0 2023-07-21T22:33:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feaa57fa700 1 -- 172.21.15.93:0/2192040573 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab00a1b40 con 0x5653182876b0 2023-07-21T22:33:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.889+0000 7feaa57fa700 1 -- 172.21.15.93:0/2192040573 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0099950 con 0x5653182876b0 2023-07-21T22:33:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 -- 172.21.15.93:0/2192040573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 msgr2=0x565318441ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feab0099680 tx=0x7feab0095b40 comp rx=0 tx=0).stop 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 -- 172.21.15.93:0/2192040573 shutdown_connections 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531843c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318441ce0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2192040573 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531843bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 -- 172.21.15.93:0/2192040573 >> 172.21.15.93:0/2192040573 conn(0x565318420ae0 msgr2=0x5653184260e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 -- 172.21.15.93:0/2192040573 shutdown_connections 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.890+0000 7fead11b7b80 1 -- 172.21.15.93:0/2192040573 wait complete. 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 -- start start 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 Processor -- start 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x565318433af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531842e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531831ca60 con 0x565318442220 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531831c760 con 0x565318442670 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653182008c0 con 0x5653182876b0 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7fead11b7b80 10 client.0 _ll_register_callbacks cb 0x56531831a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35928/0 (socket says 172.21.15.93:35928) 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.891+0000 7feabed9d700 1 -- 172.21.15.93:0/2806898538 learned_addr learned my addr 172.21.15.93:0/2806898538 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabe59c700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x565318433af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 -- 172.21.15.93:0/2806898538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 msgr2=0x56531842e8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531842e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 -- 172.21.15.93:0/2806898538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 msgr2=0x565318433af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x565318433af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 -- 172.21.15.93:0/2806898538 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01a91e0 con 0x5653182876b0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabe59c700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x565318433af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feabed9d700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feab00a1610 tx=0x7feab00b9270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972359e9473ce092 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feaa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.892+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0097a90 con 0x5653182876b0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.893+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feab009d720 con 0x5653182876b0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.893+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653182d6490 con 0x5653182876b0 2023-07-21T22:33:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.893+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab0098be0 con 0x5653182876b0 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.893+0000 7fead11b7b80 10 client.26746 fetch_fsmap learned FSMap version 79 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.893+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x565318193c00 con 0x5653182876b0 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653182d6490 con 0x5653182876b0 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 79) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feab00a16b0 con 0x5653182876b0 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 79 2023-07-21T22:33:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 10 client.26746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 20 client.26746 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56531843c5d0 0x565318446de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 4 client.26746 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565318375e50 con 0x56531843c5d0 2023-07-21T22:33:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.894+0000 7feabe59c700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56531843c5d0 0x565318446de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.895+0000 7feaa2ff5700 10 client.26746 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.895+0000 7feabe59c700 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56531843c5d0 0x565318446de0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd2d20a806d7d5f server_cookie=8f930494bbd9af3d in_seq=0 out_seq=0 2023-07-21T22:33:00.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:00 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7feaa2ff5700 1 -- 172.21.15.93:0/2806898538 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565318375e50 con 0x56531843c5d0 2023-07-21T22:33:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7feaa2ff5700 10 client.26746 handle_command_reply: tid=0 2023-07-21T22:33:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 client.26746 shutdown 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 msgr2=0x5653184335b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feab00a1610 tx=0x7feab00b9270 comp rx=0 tx=0).stop 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 shutdown_connections 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56531843c5d0 0x565318446de0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565318442670 0x56531842e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x5653184335b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7feaa2ff5700 0 client.26746 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 --2- 172.21.15.93:0/2806898538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x565318433af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 >> 172.21.15.93:0/2806898538 conn(0x565318420ae0 msgr2=0x5653184253e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.896+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 shutdown_connections 2023-07-21T22:33:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.897+0000 7fead11b7b80 1 -- 172.21.15.93:0/2806898538 wait complete. 2023-07-21T22:33:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.897+0000 7fead11b7b80 20 client.26746 trim_cache size 0 max 16384 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 Processor -- start 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 -- start start 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 0x56531842d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x565318440140 con 0x565318442220 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531842eed0 con 0x5653182876b0 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.912+0000 7fead11b7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531842fe30 con 0x56531843c5d0 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabf59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35938/0 (socket says 172.21.15.93:35938) 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabf59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48860/0 (socket says 172.21.15.93:48860) 2023-07-21T22:33:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 -- 172.21.15.93:0/1279434701 learned_addr learned my addr 172.21.15.93:0/1279434701 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabed9d700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 0x56531842d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 -- 172.21.15.93:0/1279434701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 msgr2=0x56531842d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 0x56531842d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 -- 172.21.15.93:0/1279434701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 msgr2=0x56531842bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 -- 172.21.15.93:0/1279434701 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653184470b0 con 0x56531843c5d0 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabed9d700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 0x56531842d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabf59e700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feabe59c700 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feab4005d90 tx=0x7feab4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7268e2f2be75b0d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feaa5ffb700 1 -- 172.21.15.93:0/1279434701 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4022070 con 0x56531843c5d0 2023-07-21T22:33:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feaa5ffb700 1 -- 172.21.15.93:0/1279434701 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4023040 con 0x56531843c5d0 2023-07-21T22:33:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.913+0000 7feaa5ffb700 1 -- 172.21.15.93:0/1279434701 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4003ff0 con 0x56531843c5d0 2023-07-21T22:33:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 -- 172.21.15.93:0/1279434701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 msgr2=0x56531842db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feab4005d90 tx=0x7feab4014000 comp rx=0 tx=0).stop 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 -- 172.21.15.93:0/1279434701 shutdown_connections 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531842bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653182876b0 0x56531842d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 --2- 172.21.15.93:0/1279434701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56531843c5d0 0x56531842db40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 -- 172.21.15.93:0/1279434701 >> 172.21.15.93:0/1279434701 conn(0x565318420ae0 msgr2=0x56531842cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 -- 172.21.15.93:0/1279434701 shutdown_connections 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.914+0000 7fead11b7b80 1 -- 172.21.15.93:0/1279434701 wait complete. 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 -- start start 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 Processor -- start 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56531843c5d0 0x565318492130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531848c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531848cd50 con 0x565318442220 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531848cec0 con 0x56531843c5d0 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56531848d080 con 0x5653182876b0 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7fead11b7b80 10 client.0 _ll_register_callbacks cb 0x7feab40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7feabed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35942/0 (socket says 172.21.15.93:35942) 2023-07-21T22:33:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7feabed9d700 1 -- 172.21.15.93:0/4124366434 learned_addr learned my addr 172.21.15.93:0/4124366434 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7feabf59e700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531848c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.915+0000 7feabe59c700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56531843c5d0 0x565318492130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 -- 172.21.15.93:0/4124366434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56531843c5d0 msgr2=0x565318492130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56531843c5d0 0x565318492130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 -- 172.21.15.93:0/4124366434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 msgr2=0x56531848c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531848c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 -- 172.21.15.93:0/4124366434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac010b840 con 0x5653182876b0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feabed9d700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feab00b79c0 tx=0x7feab00844c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d7e6d015858135 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab00840b0 con 0x5653182876b0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8527+0+0 (secure 0 0 0) 0x7feab00b7ac0 con 0x5653182876b0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab00a1b40 con 0x5653182876b0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab009ce20 con 0x5653182876b0 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7fead11b7b80 10 client.26752 fetch_fsmap learned FSMap version 79 2023-07-21T22:33:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) v3 -- 0x7feab4017070 con 0x5653182876b0 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab00a1b40 con 0x5653182876b0 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.916+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 79) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feab00999e0 con 0x5653182876b0 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 10 client.26752 fetch_fsmap finished waiting for FSMap version 79 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 10 client.26752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 20 client.26752 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x565318442670 0x56531849b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 4 client.26752 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565318337ff0 con 0x565318442670 2023-07-21T22:33:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7feabe59c700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x565318442670 0x56531849b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7feabe59c700 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x565318442670 0x56531849b490 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0528220d07394a8 server_cookie=afd94e5707e9ebb7 in_seq=0 out_seq=0 2023-07-21T22:33:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:00.917+0000 7feaa2ff5700 10 client.26752 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:01.094 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:00 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7feaa2ff5700 1 -- 172.21.15.93:0/4124366434 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565318337ff0 con 0x565318442670 2023-07-21T22:33:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7feaa2ff5700 10 client.26752 handle_command_reply: tid=0 2023-07-21T22:33:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 client.26752 shutdown 2023-07-21T22:33:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 msgr2=0x565318491bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feab00b79c0 tx=0x7feab00844c0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 shutdown_connections 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x565318442670 0x56531849b490 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x565318442220 0x56531848c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5653182876b0 0x565318491bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 --2- 172.21.15.93:0/4124366434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56531843c5d0 0x565318492130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 >> 172.21.15.93:0/4124366434 conn(0x565318420ae0 msgr2=0x56531842d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7feaa2ff5700 0 client.26752 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 shutdown_connections 2023-07-21T22:33:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.140+0000 7fead11b7b80 1 -- 172.21.15.93:0/4124366434 wait complete. 2023-07-21T22:33:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.143+0000 7fead11b7b80 20 client.26752 trim_cache size 0 max 16384 2023-07-21T22:33:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:33:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:33:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:33:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:33:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 msgr2=0x7feaa80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 0x7feaa80a37f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feac00767c0 tx=0x7feab4009260 comp rx=0 tx=0).stop 2023-07-21T22:33:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 msgr2=0x7feac010ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feab000b530 tx=0x7feab000b840 comp rx=0 tx=0).stop 2023-07-21T22:33:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 shutdown_connections 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feac0103430 0x7feac010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7feaa80a1340 0x7feaa80a37f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feac00fe3f0 0x7feac010ef30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 --2- 172.21.15.93:0/3197863039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feac00fed90 0x7feac010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.144+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 >> 172.21.15.93:0/3197863039 conn(0x7feac0075f90 msgr2=0x7feac0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.145+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 shutdown_connections 2023-07-21T22:33:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:01.145+0000 7feac5032700 1 -- 172.21.15.93:0/3197863039 wait complete. 2023-07-21T22:33:01.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 146877 38.14 MB/sec execute 97 sec latency 160.620 ms 2023-07-21T22:33:01.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:01 smithi134 ceph-mon[97873]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 580 op/s 2023-07-21T22:33:01.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:01 smithi093 ceph-mon[96579]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 580 op/s 2023-07-21T22:33:02.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:01 smithi194 ceph-mon[98288]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.3 MiB/s wr, 580 op/s 2023-07-21T22:33:02.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 147261 37.98 MB/sec execute 98 sec latency 106.821 ms 2023-07-21T22:33:03.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 147683 37.82 MB/sec execute 99 sec latency 231.015 ms 2023-07-21T22:33:03.593 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:03 smithi194 ceph-mon[98288]: pgmap v527: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 871 op/s 2023-07-21T22:33:03.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:03 smithi134 ceph-mon[97873]: pgmap v527: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 871 op/s 2023-07-21T22:33:03.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:03 smithi093 ceph-mon[96579]: pgmap v527: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 871 op/s 2023-07-21T22:33:04.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 148338 37.86 MB/sec execute 100 sec latency 124.346 ms 2023-07-21T22:33:05.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 148761 37.66 MB/sec execute 101 sec latency 139.414 ms 2023-07-21T22:33:05.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:05 smithi194 ceph-mon[98288]: pgmap v528: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-07-21T22:33:05.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:05 smithi194 ceph-mon[98288]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:33:05.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:05 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:05.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:05 smithi134 ceph-mon[97873]: pgmap v528: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-07-21T22:33:05.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:05 smithi134 ceph-mon[97873]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:33:05.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:05 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:05.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:05 smithi093 ceph-mon[96579]: pgmap v528: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.8 MiB/s wr, 713 op/s 2023-07-21T22:33:05.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:05 smithi093 ceph-mon[96579]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:33:05.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:05 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:06.513 INFO:tasks.workunit.client.0.smithi093.stdout: 10 149448 37.70 MB/sec execute 102 sec latency 72.366 ms 2023-07-21T22:33:07.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 149828 37.54 MB/sec execute 103 sec latency 404.393 ms 2023-07-21T22:33:07.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:07 smithi134 ceph-mon[97873]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 788 op/s 2023-07-21T22:33:07.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:07 smithi093 ceph-mon[96579]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 788 op/s 2023-07-21T22:33:07.845 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:07 smithi194 ceph-mon[98288]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.9 MiB/s wr, 788 op/s 2023-07-21T22:33:08.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 149835 37.19 MB/sec execute 104 sec latency 1404.451 ms 2023-07-21T22:33:09.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 150007 36.92 MB/sec execute 105 sec latency 1770.170 ms 2023-07-21T22:33:09.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:09 smithi134 ceph-mon[97873]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:09.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:09 smithi093 ceph-mon[96579]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:09.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:09 smithi194 ceph-mon[98288]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:10.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 150272 36.73 MB/sec execute 106 sec latency 679.359 ms 2023-07-21T22:33:11.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 150689 36.52 MB/sec execute 107 sec latency 1679.421 ms 2023-07-21T22:33:11.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:11 smithi134 ceph-mon[97873]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:11.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:11 smithi093 ceph-mon[96579]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:11.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:11 smithi194 ceph-mon[98288]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 15 MiB/s wr, 950 op/s 2023-07-21T22:33:12.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 151259 36.55 MB/sec execute 108 sec latency 2679.449 ms 2023-07-21T22:33:13.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 151627 36.39 MB/sec execute 109 sec latency 3679.483 ms 2023-07-21T22:33:13.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:13 smithi194 ceph-mon[98288]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 811 op/s 2023-07-21T22:33:13.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:13 smithi194 ceph-mon[98288]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:33:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:13 smithi134 ceph-mon[97873]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 811 op/s 2023-07-21T22:33:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:13 smithi134 ceph-mon[97873]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:33:13.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:13 smithi093 ceph-mon[96579]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 811 op/s 2023-07-21T22:33:13.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:13 smithi093 ceph-mon[96579]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:33:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 -- 172.21.15.93:0/1901129991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98740ff470 msgr2=0x7f987410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 --2- 172.21.15.93:0/1901129991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98740ff470 0x7f987410b900 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b0a0 tx=0x7f986c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 -- 172.21.15.93:0/1901129991 shutdown_connections 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 --2- 172.21.15.93:0/1901129991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98740ff470 0x7f987410b900 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 --2- 172.21.15.93:0/1901129991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9874104ad0 0x7f9874104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 --2- 172.21.15.93:0/1901129991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f98740fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.289+0000 7f9879b90700 1 -- 172.21.15.93:0/1901129991 >> 172.21.15.93:0/1901129991 conn(0x7f9874075bd0 msgr2=0x7f9874075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.290+0000 7f9879b90700 1 -- 172.21.15.93:0/1901129991 shutdown_connections 2023-07-21T22:33:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.290+0000 7f9879b90700 1 -- 172.21.15.93:0/1901129991 wait complete. 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.290+0000 7f9879b90700 1 Processor -- start 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 -- start start 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 0x7f9874107370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 0x7f98741078b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874107ef0 con 0x7f9874104ad0 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874108060 con 0x7f98740ff470 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9879b90700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874108220 con 0x7f98740feac0 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9873fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 0x7f98741078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f98737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f98737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35954/0 (socket says 172.21.15.93:35954) 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9873fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 0x7f98741078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48878/0 (socket says 172.21.15.93:48878) 2023-07-21T22:33:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f98737fe700 1 -- 172.21.15.93:0/1730272912 learned_addr learned my addr 172.21.15.93:0/1730272912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f9872ffd700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 0x7f9874107370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.291+0000 7f98737fe700 1 -- 172.21.15.93:0/1730272912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 msgr2=0x7f9874107370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f98737fe700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 0x7f9874107370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f98737fe700 1 -- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 msgr2=0x7f98741078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f98737fe700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 0x7f98741078b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f98737fe700 1 -- 172.21.15.93:0/1730272912 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c009d00 con 0x7f98740feac0 2023-07-21T22:33:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f9872ffd700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 0x7f9874107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.292+0000 7f98737fe700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f986400e970 tx=0x7f986400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.294+0000 7f9870ff9700 1 -- 172.21.15.93:0/1730272912 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009790 con 0x7f98740feac0 2023-07-21T22:33:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.294+0000 7f9870ff9700 1 -- 172.21.15.93:0/1730272912 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f986400c7e0 con 0x7f98740feac0 2023-07-21T22:33:14.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.294+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98741a8cf0 con 0x7f98740feac0 2023-07-21T22:33:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.294+0000 7f9870ff9700 1 -- 172.21.15.93:0/1730272912 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003c70 con 0x7f98740feac0 2023-07-21T22:33:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.294+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98741a91f0 con 0x7f98740feac0 2023-07-21T22:33:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.297+0000 7f9870ff9700 1 -- 172.21.15.93:0/1730272912 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9864004220 con 0x7f98740feac0 2023-07-21T22:33:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.298+0000 7f9870ff9700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 0x7f985c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.298+0000 7f9872ffd700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 0x7f985c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.298+0000 7f9870ff9700 1 -- 172.21.15.93:0/1730272912 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9864014070 con 0x7f98740feac0 2023-07-21T22:33:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.299+0000 7f9872ffd700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 0x7f985c0a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9874108d70 tx=0x7f9868009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 Processor -- start 2023-07-21T22:33:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 -- start start 2023-07-21T22:33:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3596cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c3590f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c35914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3496c50 con 0x5643c3597660 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c34e99e0 con 0x5643c3597210 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.300+0000 7f9885d15b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c34e2e70 con 0x5643c33ed6b0 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f98737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3596cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c3590f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f98737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3596cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:35962/0 (socket says 172.21.15.93:35962) 2023-07-21T22:33:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f98737fe700 1 -- 172.21.15.93:0/3809411471 learned_addr learned my addr 172.21.15.93:0/3809411471 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 -- 172.21.15.93:0/3809411471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 msgr2=0x5643c3596cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3596cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 -- 172.21.15.93:0/3809411471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 msgr2=0x5643c35914a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c35914a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 -- 172.21.15.93:0/3809411471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c3310d80 con 0x5643c3597210 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9872ffd700 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c3590f40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98680042f0 tx=0x7f9868014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432a46180c17d66e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9859ffb700 1 -- 172.21.15.93:0/3809411471 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868003d10 con 0x5643c3597210 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9859ffb700 1 -- 172.21.15.93:0/3809411471 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9868003e70 con 0x5643c3597210 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.301+0000 7f9859ffb700 1 -- 172.21.15.93:0/3809411471 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986801e7b0 con 0x5643c3597210 2023-07-21T22:33:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 -- 172.21.15.93:0/3809411471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 msgr2=0x5643c3590f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c3590f40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98680042f0 tx=0x7f9868014000 comp rx=0 tx=0).stop 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 -- 172.21.15.93:0/3809411471 shutdown_connections 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c35914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3596cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 --2- 172.21.15.93:0/3809411471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c3590f40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 -- 172.21.15.93:0/3809411471 >> 172.21.15.93:0/3809411471 conn(0x5643c3586ae0 msgr2=0x5643c358c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.302+0000 7f9885d15b80 1 -- 172.21.15.93:0/3809411471 shutdown_connections 2023-07-21T22:33:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.303+0000 7f9885d15b80 1 -- 172.21.15.93:0/3809411471 wait complete. 2023-07-21T22:33:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.303+0000 7f9885d15b80 1 -- start start 2023-07-21T22:33:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.303+0000 7f9885d15b80 1 Processor -- start 2023-07-21T22:33:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c359d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c359a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3482a60 con 0x5643c3597660 2023-07-21T22:33:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3482760 con 0x5643c3597210 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c33668c0 con 0x5643c33ed6b0 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9885d15b80 10 client.0 _ll_register_callbacks cb 0x5643c3480700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48888/0 (socket says 172.21.15.93:48888) 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 -- 172.21.15.93:0/730006933 learned_addr learned my addr 172.21.15.93:0/730006933 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9872ffd700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c359a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f98737fe700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c359d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 -- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 msgr2=0x5643c359d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c359d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 -- 172.21.15.93:0/730006933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 msgr2=0x5643c359a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c359a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.304+0000 7f9873fff700 1 -- 172.21.15.93:0/730006933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9868017070 con 0x5643c3597660 2023-07-21T22:33:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9873fff700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f986c009610 tx=0x7f986c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4011a9758f0a03f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9852ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00e040 con 0x5643c3597660 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f986c003e10 con 0x5643c3597660 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643c343c490 con 0x5643c3597660 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f986c0074e0 con 0x5643c3597660 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9885d15b80 10 client.17472 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5643c32f9c00 con 0x5643c3597660 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.305+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643c343c490 con 0x5643c3597660 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f986c019030 con 0x5643c3597660 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 20 client.17472 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35aac60 0x5643c35ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 4 client.17472 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.306+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643c34dbe50 con 0x5643c35aac60 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.307+0000 7f98737fe700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35aac60 0x5643c35ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.307+0000 7f9852ffd700 10 client.17472 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.307+0000 7f98737fe700 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35aac60 0x5643c35ad070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a638b58713a8fc server_cookie=77c4e008dd8f20b3 in_seq=0 out_seq=0 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9852ffd700 1 -- 172.21.15.93:0/730006933 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5643c34dbe50 con 0x5643c35aac60 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9852ffd700 10 client.17472 handle_command_reply: tid=0 2023-07-21T22:33:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 client.17472 shutdown 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 msgr2=0x5643c359af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f986c009610 tx=0x7f986c00bd50 comp rx=0 tx=0).stop 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 shutdown_connections 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35aac60 0x5643c35ad070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597660 0x5643c359af60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c359d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 --2- 172.21.15.93:0/730006933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597210 0x5643c359a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 >> 172.21.15.93:0/730006933 conn(0x5643c3586ae0 msgr2=0x5643c358b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.308+0000 7f9852ffd700 0 client.17472 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.309+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 shutdown_connections 2023-07-21T22:33:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.310+0000 7f9885d15b80 1 -- 172.21.15.93:0/730006933 wait complete. 2023-07-21T22:33:14.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.312+0000 7f9885d15b80 20 client.17472 trim_cache size 0 max 16384 2023-07-21T22:33:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 Processor -- start 2023-07-21T22:33:14.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 -- start start 2023-07-21T22:33:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3593740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597210 0x5643c35a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3595480 con 0x5643c3597210 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c358d6b0 con 0x5643c3597660 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9885d15b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c35a2170 con 0x5643c33ed6b0 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.385 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f98737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3593740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60056/0 (socket says 172.21.15.93:60056) 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 -- 172.21.15.93:0/2558158485 learned_addr learned my addr 172.21.15.93:0/2558158485 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 -- 172.21.15.93:0/2558158485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 msgr2=0x5643c3593740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3593740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 -- 172.21.15.93:0/2558158485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597210 msgr2=0x5643c35a1c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.336+0000 7f9873fff700 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597210 0x5643c35a1c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.386 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.337+0000 7f9873fff700 1 -- 172.21.15.93:0/2558158485 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c33371f0 con 0x5643c3597660 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.337+0000 7f9873fff700 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f986c01ce10 tx=0x7f986c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5186daac92e2d596 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.337+0000 7f985a7fc700 1 -- 172.21.15.93:0/2558158485 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00e040 con 0x5643c3597660 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.337+0000 7f985a7fc700 1 -- 172.21.15.93:0/2558158485 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f986c007960 con 0x5643c3597660 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.337+0000 7f985a7fc700 1 -- 172.21.15.93:0/2558158485 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c005370 con 0x5643c3597660 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 -- 172.21.15.93:0/2558158485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 msgr2=0x5643c359d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f986c01ce10 tx=0x7f986c0099e0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 -- 172.21.15.93:0/2558158485 shutdown_connections 2023-07-21T22:33:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c3597660 0x5643c359d140 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c33ed6b0 0x5643c3593740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 --2- 172.21.15.93:0/2558158485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c3597210 0x5643c35a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.388 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.338+0000 7f9885d15b80 1 -- 172.21.15.93:0/2558158485 >> 172.21.15.93:0/2558158485 conn(0x5643c3586ae0 msgr2=0x5643c3594910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.339+0000 7f9885d15b80 1 -- 172.21.15.93:0/2558158485 shutdown_connections 2023-07-21T22:33:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.341+0000 7f9885d15b80 1 -- 172.21.15.93:0/2558158485 wait complete. 2023-07-21T22:33:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.341+0000 7f9885d15b80 1 -- start start 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 Processor -- start 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c3597660 0x5643c3590470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 0x5643c3590e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3591360 con 0x5643c33ed6b0 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c35914d0 con 0x5643c35909b0 2023-07-21T22:33:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c3591640 con 0x5643c3597660 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9885d15b80 10 client.0 _ll_register_callbacks cb 0x7f986c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f98737fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f98737fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48898/0 (socket says 172.21.15.93:48898) 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f98737fe700 1 -- 172.21.15.93:0/1660375157 learned_addr learned my addr 172.21.15.93:0/1660375157 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9872ffd700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c3597660 0x5643c3590470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.342+0000 7f9873fff700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 0x5643c3590e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 -- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c3597660 msgr2=0x5643c3590470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c3597660 0x5643c3590470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 -- 172.21.15.93:0/1660375157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 msgr2=0x5643c358ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 -- 172.21.15.93:0/1660375157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643c32f9c00 con 0x5643c35909b0 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f98737fe700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9873fff700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 0x5643c3590e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f986c01cd90 tx=0x7f986c01c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf844ecf30961a2f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9852ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c01cb40 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9868003e70 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f986c005320 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f986c0054e0 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9885d15b80 10 client.26855 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.343+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x5643c330f250 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9868003e70 con 0x5643c35909b0 2023-07-21T22:33:14.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f986c03e070 con 0x5643c35909b0 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 10 client.26855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 20 client.26855 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35ff7a0 0x5643c3601b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 4 client.26855 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:14.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5643c349dff0 con 0x5643c35ff7a0 2023-07-21T22:33:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f98737fe700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35ff7a0 0x5643c3601b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f98737fe700 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35ff7a0 0x5643c3601b80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7fe7f624ae4cd2 server_cookie=df21a8f5b9342e7a in_seq=0 out_seq=0 2023-07-21T22:33:14.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:14.344+0000 7f9852ffd700 10 client.26855 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:14.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 151981 36.17 MB/sec execute 110 sec latency 4679.525 ms 2023-07-21T22:33:15.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 152598 36.20 MB/sec execute 111 sec latency 5679.562 ms 2023-07-21T22:33:15.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:15 smithi194 ceph-mon[98288]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.24k op/s 2023-07-21T22:33:15.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:15 smithi134 ceph-mon[97873]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.24k op/s 2023-07-21T22:33:15.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:15 smithi093 ceph-mon[96579]: pgmap v535: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.24k op/s 2023-07-21T22:33:16.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 152970 36.06 MB/sec execute 112 sec latency 6679.597 ms 2023-07-21T22:33:17.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 153324 35.87 MB/sec execute 113 sec latency 7679.624 ms 2023-07-21T22:33:17.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:17 smithi194 ceph-mon[98288]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.23k op/s 2023-07-21T22:33:17.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:17 smithi134 ceph-mon[97873]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.23k op/s 2023-07-21T22:33:17.696 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:17 smithi093 ceph-mon[96579]: pgmap v536: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 29 MiB/s wr, 1.23k op/s 2023-07-21T22:33:18.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 153791 35.71 MB/sec execute 114 sec latency 8178.178 ms 2023-07-21T22:33:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.757+0000 7f9852ffd700 1 -- 172.21.15.93:0/1660375157 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5643c349dff0 con 0x5643c35ff7a0 2023-07-21T22:33:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.757+0000 7f9852ffd700 10 client.26855 handle_command_reply: tid=0 2023-07-21T22:33:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.757+0000 7f9885d15b80 1 client.26855 shutdown 2023-07-21T22:33:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.757+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 msgr2=0x5643c3590e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.757+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 0x5643c3590e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f986c01cd90 tx=0x7f986c01c670 comp rx=0 tx=0).stop 2023-07-21T22:33:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 shutdown_connections 2023-07-21T22:33:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5643c35ff7a0 0x5643c3601b80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5643c35909b0 0x5643c3590e20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5643c33ed6b0 0x5643c358ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 --2- 172.21.15.93:0/1660375157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5643c3597660 0x5643c3590470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 >> 172.21.15.93:0/1660375157 conn(0x5643c3586ae0 msgr2=0x5643c359a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9852ffd700 0 client.26855 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.758+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 shutdown_connections 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.759+0000 7f9885d15b80 1 -- 172.21.15.93:0/1660375157 wait complete. 2023-07-21T22:33:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.759+0000 7f9885d15b80 20 client.26855 trim_cache size 0 max 16384 2023-07-21T22:33:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:33:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:33:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:33:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:33:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 msgr2=0x7f985c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 0x7f985c0a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9874108d70 tx=0x7f9868009260 comp rx=0 tx=0).stop 2023-07-21T22:33:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 msgr2=0x7f987410b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f986400e970 tx=0x7f986400ec80 comp rx=0 tx=0).stop 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 shutdown_connections 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f985c0a1340 0x7f985c0a37f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9874104ad0 0x7f98741078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98740feac0 0x7f987410b580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 --2- 172.21.15.93:0/1730272912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98740ff470 0x7f9874107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.761+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 >> 172.21.15.93:0/1730272912 conn(0x7f9874075bd0 msgr2=0x7f98740fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.762+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 shutdown_connections 2023-07-21T22:33:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:18.762+0000 7f9879b90700 1 -- 172.21.15.93:0/1730272912 wait complete. 2023-07-21T22:33:19.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 154342 35.76 MB/sec execute 115 sec latency 8593.579 ms 2023-07-21T22:33:19.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:19 smithi194 ceph-mon[98288]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:19.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:19 smithi134 ceph-mon[97873]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:19.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:19 smithi093 ceph-mon[96579]: pgmap v537: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:20.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 154806 35.65 MB/sec execute 116 sec latency 9419.785 ms 2023-07-21T22:33:21.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 155343 35.56 MB/sec execute 117 sec latency 10419.854 ms 2023-07-21T22:33:21.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:21 smithi194 ceph-mon[98288]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:21.633 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:21 smithi134 ceph-mon[97873]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:21.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:21 smithi093 ceph-mon[96579]: pgmap v538: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.02k op/s 2023-07-21T22:33:22.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 155979 35.54 MB/sec execute 118 sec latency 10879.401 ms 2023-07-21T22:33:23.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 156690 35.50 MB/sec execute 119 sec latency 91.808 ms 2023-07-21T22:33:23.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:23 smithi194 ceph-mon[98288]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 853 op/s 2023-07-21T22:33:23.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:23 smithi134 ceph-mon[97873]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 853 op/s 2023-07-21T22:33:23.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:23 smithi093 ceph-mon[96579]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 16 MiB/s wr, 853 op/s 2023-07-21T22:33:24.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 157529 35.67 MB/sec execute 120 sec latency 63.157 ms 2023-07-21T22:33:25.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 158118 35.70 MB/sec execute 121 sec latency 103.736 ms 2023-07-21T22:33:25.589 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:25 smithi194 ceph-mon[98288]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 892 op/s 2023-07-21T22:33:25.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:25 smithi134 ceph-mon[97873]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 892 op/s 2023-07-21T22:33:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:25 smithi093 ceph-mon[96579]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 16 MiB/s wr, 892 op/s 2023-07-21T22:33:26.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 158790 35.67 MB/sec execute 122 sec latency 54.927 ms 2023-07-21T22:33:27.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 159400 35.59 MB/sec execute 123 sec latency 96.509 ms 2023-07-21T22:33:27.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:27 smithi134 ceph-mon[97873]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2023-07-21T22:33:27.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:27 smithi093 ceph-mon[96579]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2023-07-21T22:33:27.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:27 smithi194 ceph-mon[98288]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2023-07-21T22:33:28.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 160131 35.60 MB/sec execute 124 sec latency 65.023 ms 2023-07-21T22:33:29.514 INFO:tasks.workunit.client.0.smithi093.stdout: 10 161032 35.73 MB/sec execute 125 sec latency 57.159 ms 2023-07-21T22:33:29.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:29 smithi134 ceph-mon[97873]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 676 op/s 2023-07-21T22:33:29.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:29 smithi093 ceph-mon[96579]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 676 op/s 2023-07-21T22:33:29.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:29 smithi194 ceph-mon[98288]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.8 MiB/s wr, 676 op/s 2023-07-21T22:33:30.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 161761 35.79 MB/sec execute 126 sec latency 69.200 ms 2023-07-21T22:33:31.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 162459 35.83 MB/sec execute 127 sec latency 76.799 ms 2023-07-21T22:33:31.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:31 smithi134 ceph-mon[97873]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2023-07-21T22:33:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:31 smithi093 ceph-mon[96579]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2023-07-21T22:33:31.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:31 smithi194 ceph-mon[98288]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.7 MiB/s wr, 450 op/s 2023-07-21T22:33:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 -- 172.21.15.93:0/4158222582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e20076980 msgr2=0x7f3e200fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 --2- 172.21.15.93:0/4158222582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e20076980 0x7f3e200fed20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009990 tx=0x7f3e10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 -- 172.21.15.93:0/4158222582 shutdown_connections 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 --2- 172.21.15.93:0/4158222582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e2010a140 0x7f3e2010c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 --2- 172.21.15.93:0/4158222582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e20076980 0x7f3e200fed20 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 --2- 172.21.15.93:0/4158222582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e200ff2f0 0x7f3e200ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 -- 172.21.15.93:0/4158222582 >> 172.21.15.93:0/4158222582 conn(0x7f3e20075270 msgr2=0x7f3e20075670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 -- 172.21.15.93:0/4158222582 shutdown_connections 2023-07-21T22:33:31.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.902+0000 7f3e24d44700 1 -- 172.21.15.93:0/4158222582 wait complete. 2023-07-21T22:33:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 Processor -- start 2023-07-21T22:33:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 -- start start 2023-07-21T22:33:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e20076980 0x7f3e2019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e2010a140 0x7f3e20195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20196080 con 0x7f3e200ff2f0 2023-07-21T22:33:31.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201961f0 con 0x7f3e2010a140 2023-07-21T22:33:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.903+0000 7f3e24d44700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201963b0 con 0x7f3e20076980 2023-07-21T22:33:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48914/0 (socket says 172.21.15.93:48914) 2023-07-21T22:33:31.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/351823209 learned_addr learned my addr 172.21.15.93:0/351823209 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e20076980 0x7f3e2019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1ed9d700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e2010a140 0x7f3e20195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/351823209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e20076980 msgr2=0x7f3e2019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e20076980 0x7f3e2019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/351823209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e2010a140 msgr2=0x7f3e20195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e2010a140 0x7f3e20195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/351823209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100095f0 con 0x7f3e200ff2f0 2023-07-21T22:33:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9a0 tx=0x7f3e1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e0f7fe700 1 -- 172.21.15.93:0/351823209 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14009790 con 0x7f3e200ff2f0 2023-07-21T22:33:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e20196690 con 0x7f3e200ff2f0 2023-07-21T22:33:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e0f7fe700 1 -- 172.21.15.93:0/351823209 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e14010430 con 0x7f3e200ff2f0 2023-07-21T22:33:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e0f7fe700 1 -- 172.21.15.93:0/351823209 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14018680 con 0x7f3e200ff2f0 2023-07-21T22:33:31.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.904+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201a2ba0 con 0x7f3e200ff2f0 2023-07-21T22:33:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.907+0000 7f3e0f7fe700 1 -- 172.21.15.93:0/351823209 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3e140105a0 con 0x7f3e200ff2f0 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.908+0000 7f3e0f7fe700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 0x7f3e080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.908+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 0x7f3e080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.908+0000 7f3e0f7fe700 1 -- 172.21.15.93:0/351823209 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e14014070 con 0x7f3e200ff2f0 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.908+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 0x7f3e080a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000b2e0 tx=0x7f3e10005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 Processor -- start 2023-07-21T22:33:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 -- start start 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf80e8c50 con 0x562cf81ed540 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf813b9e0 con 0x562cf81ed9b0 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf8134e70 con 0x562cf803f6b0 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.909+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48922/0 (socket says 172.21.15.93:48922) 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/109931525 learned_addr learned my addr 172.21.15.93:0/109931525 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1ed9d700 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/109931525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 msgr2=0x562cf81ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/109931525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 msgr2=0x562cf81e3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/109931525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cf7f62d80 con 0x562cf81ed540 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3e140a1ed0 tx=0x7f3e14099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a294c13f1f0ec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e0cff9700 1 -- 172.21.15.93:0/109931525 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14010880 con 0x562cf81ed540 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e0cff9700 1 -- 172.21.15.93:0/109931525 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e140109e0 con 0x562cf81ed540 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.910+0000 7f3e0cff9700 1 -- 172.21.15.93:0/109931525 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140a1f70 con 0x562cf81ed540 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/109931525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 msgr2=0x562cf81e2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3e140a1ed0 tx=0x7f3e14099fb0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/109931525 shutdown_connections 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/109931525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81e2f40 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/109931525 >> 172.21.15.93:0/109931525 conn(0x562cf81d8ae0 msgr2=0x562cf81de0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/109931525 shutdown_connections 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.911+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/109931525 wait complete. 2023-07-21T22:33:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 -- start start 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 Processor -- start 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf81e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf80d4a60 con 0x562cf81ed540 2023-07-21T22:33:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf80d4760 con 0x562cf803f6b0 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf7fb88c0 con 0x562cf81ed9b0 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e30ec9b80 10 client.0 _ll_register_callbacks cb 0x562cf80d2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e1e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e1e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60088/0 (socket says 172.21.15.93:60088) 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e1ed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf81e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.912+0000 7f3e1e59c700 1 -- 172.21.15.93:0/3985312416 learned_addr learned my addr 172.21.15.93:0/3985312416 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 -- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 msgr2=0x562cf81e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf81e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 -- 172.21.15.93:0/3985312416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 msgr2=0x562cf81f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 -- 172.21.15.93:0/3985312416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201a2ba0 con 0x562cf803f6b0 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005460 tx=0x7f3e10009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ef12920b9e7c9e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:33:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.913+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10004da0 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e10005960 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cf808e490 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e1000f460 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 10 client.26867 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x562cf7f4bc00 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e1003c020 con 0x562cf803f6b0 2023-07-21T22:33:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3e1002f7a0 con 0x562cf803f6b0 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 10 client.26867 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 10 client.26867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 20 client.26867 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf820db90 0x562cf820ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 4 client.26867 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.914+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cf812de50 con 0x562cf820db90 2023-07-21T22:33:31.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.915+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf820db90 0x562cf820ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.915+0000 7f3e05ffb700 10 client.26867 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:31.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.915+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf820db90 0x562cf820ffa0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35d9b827fa62485 server_cookie=d9cec8fcc6a3000a in_seq=0 out_seq=0 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e05ffb700 1 -- 172.21.15.93:0/3985312416 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562cf812de50 con 0x562cf820db90 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e05ffb700 10 client.26867 handle_command_reply: tid=0 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 client.26867 shutdown 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 msgr2=0x562cf81f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005460 tx=0x7f3e10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 shutdown_connections 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf820db90 0x562cf820ffa0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf81e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf81f1570 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e05ffb700 0 client.26867 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3985312416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.916+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 >> 172.21.15.93:0/3985312416 conn(0x562cf81d8ae0 msgr2=0x562cf81dd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.917+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 shutdown_connections 2023-07-21T22:33:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.917+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3985312416 wait complete. 2023-07-21T22:33:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.917+0000 7f3e30ec9b80 20 client.26867 trim_cache size 0 max 16384 2023-07-21T22:33:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.937+0000 7f3e30ec9b80 1 Processor -- start 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.937+0000 7f3e30ec9b80 1 -- start start 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf81e3950 con 0x562cf81ed540 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf81e7d10 con 0x562cf81ed9b0 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf81df760 con 0x562cf803f6b0 2023-07-21T22:33:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48936/0 (socket says 172.21.15.93:48936) 2023-07-21T22:33:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/3925359053 learned_addr learned my addr 172.21.15.93:0/3925359053 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1ed9d700 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/3925359053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 msgr2=0x562cf81e43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/3925359053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 msgr2=0x562cf81e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.938+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/3925359053 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cf81e77e0 con 0x562cf81ed540 2023-07-21T22:33:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.939+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3e14099f40 tx=0x7f3e14096370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360aae6816565ccb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.939+0000 7f3e0d7fa700 1 -- 172.21.15.93:0/3925359053 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140b6070 con 0x562cf81ed540 2023-07-21T22:33:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.939+0000 7f3e0d7fa700 1 -- 172.21.15.93:0/3925359053 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e14098350 con 0x562cf81ed540 2023-07-21T22:33:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.939+0000 7f3e0d7fa700 1 -- 172.21.15.93:0/3925359053 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140a1f70 con 0x562cf81ed540 2023-07-21T22:33:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.940+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3925359053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 msgr2=0x562cf81ddb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.940+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3e14099f40 tx=0x7f3e14096370 comp rx=0 tx=0).stop 2023-07-21T22:33:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3925359053 shutdown_connections 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf81ed9b0 0x562cf81e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf803f6b0 0x562cf81e43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/3925359053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf81ddb60 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3925359053 >> 172.21.15.93:0/3925359053 conn(0x562cf81d8ae0 msgr2=0x562cf81ec5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3925359053 shutdown_connections 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.941+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/3925359053 wait complete. 2023-07-21T22:33:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 -- start start 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 Processor -- start 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf821b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf82611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf821c650 con 0x562cf81ed540 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf8261700 con 0x562cf803f6b0 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf8261840 con 0x562cf81ed9b0 2023-07-21T22:33:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e30ec9b80 10 client.0 _ll_register_callbacks cb 0x562cf80d2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.942+0000 7f3e1dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48942/0 (socket says 172.21.15.93:48942) 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/992233795 learned_addr learned my addr 172.21.15.93:0/992233795 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1ed9d700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf82611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 msgr2=0x562cf82611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf82611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/992233795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 msgr2=0x562cf821b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf821b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 -- 172.21.15.93:0/992233795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cf7f62d80 con 0x562cf81ed540 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1ed9d700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf82611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e1dd9b700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3e140a1ed0 tx=0x7f3e14099eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad579dab71051512 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140b6030 con 0x562cf81ed540 2023-07-21T22:33:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e140109e0 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e140b7070 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.943+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e140ca310 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 10 client.17493 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x562cf7f891f0 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e14063080 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3e14094c50 con 0x562cf81ed540 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 10 client.17493 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 10 client.17493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 20 client.17493 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf8262a50 0x562cf8264e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 4 client.17493 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562cf80efff0 con 0x562cf8262a50 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf8262a50 0x562cf8264e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e05ffb700 10 client.17493 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:31.944+0000 7f3e1e59c700 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf8262a50 0x562cf8264e30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f9074f8eb9e9330 server_cookie=e4575d3035f43c3b in_seq=0 out_seq=0 2023-07-21T22:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e05ffb700 1 -- 172.21.15.93:0/992233795 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f3e1003c040 con 0x562cf8262a50 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e05ffb700 10 client.17493 handle_command_reply: tid=0 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 client.17493 shutdown 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 msgr2=0x562cf821bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3e140a1ed0 tx=0x7f3e14099eb0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 shutdown_connections 2023-07-21T22:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562cf8262a50 0x562cf8264e30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562cf81ed9b0 0x562cf82611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.238+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562cf803f6b0 0x562cf821b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e30ec9b80 1 --2- 172.21.15.93:0/992233795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562cf81ed540 0x562cf821bd30 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 >> 172.21.15.93:0/992233795 conn(0x562cf81d8ae0 msgr2=0x562cf81e43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e05ffb700 0 client.17493 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 shutdown_connections 2023-07-21T22:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e30ec9b80 1 -- 172.21.15.93:0/992233795 wait complete. 2023-07-21T22:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.239+0000 7f3e30ec9b80 20 client.17493 trim_cache size 0 max 16384 2023-07-21T22:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:33:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.240+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 msgr2=0x7f3e080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.240+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 0x7f3e080a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000b2e0 tx=0x7f3e10005fd0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.240+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 msgr2=0x7f3e2019c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.240+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400e9a0 tx=0x7f3e1400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 shutdown_connections 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e2010a140 0x7f3e20195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e20076980 0x7f3e2019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f3e080a12c0 0x7f3e080a3770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 --2- 172.21.15.93:0/351823209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e200ff2f0 0x7f3e2019c0f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 >> 172.21.15.93:0/351823209 conn(0x7f3e20075270 msgr2=0x7f3e20102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.241+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 shutdown_connections 2023-07-21T22:33:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:32.242+0000 7f3e24d44700 1 -- 172.21.15.93:0/351823209 wait complete. 2023-07-21T22:33:32.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 162936 35.75 MB/sec execute 128 sec latency 150.847 ms 2023-07-21T22:33:33.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 163641 35.71 MB/sec execute 129 sec latency 68.441 ms 2023-07-21T22:33:33.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:33 smithi134 ceph-mon[97873]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 484 op/s 2023-07-21T22:33:33.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:33 smithi093 ceph-mon[96579]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 484 op/s 2023-07-21T22:33:33.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:33 smithi194 ceph-mon[98288]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.2 MiB/s wr, 484 op/s 2023-07-21T22:33:34.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 164483 35.78 MB/sec execute 130 sec latency 57.224 ms 2023-07-21T22:33:35.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 165246 35.87 MB/sec execute 131 sec latency 84.522 ms 2023-07-21T22:33:35.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:35 smithi134 ceph-mon[97873]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 710 op/s 2023-07-21T22:33:35.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:35 smithi093 ceph-mon[96579]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 710 op/s 2023-07-21T22:33:35.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:35 smithi194 ceph-mon[98288]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.2 MiB/s wr, 710 op/s 2023-07-21T22:33:36.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 165693 35.78 MB/sec execute 132 sec latency 197.500 ms 2023-07-21T22:33:37.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 166223 35.71 MB/sec execute 133 sec latency 234.329 ms 2023-07-21T22:33:37.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:37 smithi134 ceph-mon[97873]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.4 MiB/s wr, 531 op/s 2023-07-21T22:33:37.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:37 smithi093 ceph-mon[96579]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.4 MiB/s wr, 531 op/s 2023-07-21T22:33:37.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:37 smithi194 ceph-mon[98288]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.4 MiB/s wr, 531 op/s 2023-07-21T22:33:38.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 166923 35.71 MB/sec execute 134 sec latency 56.512 ms 2023-07-21T22:33:39.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 167794 35.74 MB/sec execute 135 sec latency 46.389 ms 2023-07-21T22:33:39.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:39 smithi134 ceph-mon[97873]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-07-21T22:33:39.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:39 smithi093 ceph-mon[96579]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-07-21T22:33:39.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:39 smithi194 ceph-mon[98288]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 779 op/s 2023-07-21T22:33:40.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 168652 35.88 MB/sec execute 136 sec latency 72.226 ms 2023-07-21T22:33:41.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 169225 35.91 MB/sec execute 137 sec latency 67.772 ms 2023-07-21T22:33:41.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:41 smithi134 ceph-mon[97873]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.9 MiB/s wr, 547 op/s 2023-07-21T22:33:41.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:41 smithi093 ceph-mon[96579]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.9 MiB/s wr, 547 op/s 2023-07-21T22:33:41.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:41 smithi194 ceph-mon[98288]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.9 MiB/s wr, 547 op/s 2023-07-21T22:33:42.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 169689 35.71 MB/sec execute 138 sec latency 130.015 ms 2023-07-21T22:33:43.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 170277 35.72 MB/sec execute 139 sec latency 192.019 ms 2023-07-21T22:33:43.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:43 smithi134 ceph-mon[97873]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-07-21T22:33:43.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:43 smithi093 ceph-mon[96579]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-07-21T22:33:43.843 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:43 smithi194 ceph-mon[98288]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-07-21T22:33:44.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 170992 35.70 MB/sec execute 140 sec latency 70.656 ms 2023-07-21T22:33:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.391+0000 7fafc6100700 1 -- 172.21.15.93:0/882629273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0067c10 msgr2=0x7fafc0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.391+0000 7fafc6100700 1 --2- 172.21.15.93:0/882629273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0067c10 0x7fafc0067fe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fafb00099c0 tx=0x7fafb0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 -- 172.21.15.93:0/882629273 shutdown_connections 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 --2- 172.21.15.93:0/882629273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc0104e50 0x7fafc010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 --2- 172.21.15.93:0/882629273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0067c10 0x7fafc0067fe0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 --2- 172.21.15.93:0/882629273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafc00685b0 0x7fafc0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 -- 172.21.15.93:0/882629273 >> 172.21.15.93:0/882629273 conn(0x7fafc0074760 msgr2=0x7fafc0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 -- 172.21.15.93:0/882629273 shutdown_connections 2023-07-21T22:33:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 -- 172.21.15.93:0/882629273 wait complete. 2023-07-21T22:33:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.392+0000 7fafc6100700 1 Processor -- start 2023-07-21T22:33:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 -- start start 2023-07-21T22:33:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafc0067c10 0x7fafc019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc00685b0 0x7fafc019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0196080 con 0x7fafc0104e50 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01961f0 con 0x7fafc00685b0 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafc6100700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01963b0 con 0x7fafc0067c10 2023-07-21T22:33:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafbffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafbffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48956/0 (socket says 172.21.15.93:48956) 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafbffff700 1 -- 172.21.15.93:0/818328183 learned_addr learned my addr 172.21.15.93:0/818328183 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.393+0000 7fafbeffd700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc00685b0 0x7fafc019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 -- 172.21.15.93:0/818328183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafc0067c10 msgr2=0x7fafc019bbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafc0067c10 0x7fafc019bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 -- 172.21.15.93:0/818328183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc00685b0 msgr2=0x7fafc019c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc00685b0 0x7fafc019c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 -- 172.21.15.93:0/818328183 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00095f0 con 0x7fafc0104e50 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbffff700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007ee0 tx=0x7fafb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbcff9700 1 -- 172.21.15.93:0/818328183 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800ddd0 con 0x7fafc0104e50 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc0196690 con 0x7fafc0104e50 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc01a2ba0 con 0x7fafc0104e50 2023-07-21T22:33:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbcff9700 1 -- 172.21.15.93:0/818328183 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb800f040 con 0x7fafc0104e50 2023-07-21T22:33:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.394+0000 7fafbcff9700 1 -- 172.21.15.93:0/818328183 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8015400 con 0x7fafc0104e50 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.398+0000 7fafbcff9700 1 -- 172.21.15.93:0/818328183 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fafb8004020 con 0x7fafc0104e50 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.398+0000 7fafd2285b80 1 Processor -- start 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 -- start start 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6210 0x557c209bff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6660 0x557c209c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c208c5c50 con 0x557c209c6660 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209189e0 con 0x557c2081c6b0 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafd2285b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c20911e70 con 0x557c209c6210 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbcff9700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 0x7fafa80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60122/0 (socket says 172.21.15.93:60122) 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbeffd700 1 -- 172.21.15.93:0/634728430 learned_addr learned my addr 172.21.15.93:0/634728430 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6210 0x557c209bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbeffd700 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6660 0x557c209c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.399+0000 7fafbcff9700 1 -- 172.21.15.93:0/818328183 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fafb80cc490 con 0x7fafc0104e50 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 -- 172.21.15.93:0/634728430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6210 msgr2=0x557c209bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6210 0x557c209bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 -- 172.21.15.93:0/634728430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6660 msgr2=0x557c209c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6660 0x557c209c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 -- 172.21.15.93:0/634728430 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2073fd80 con 0x557c2081c6b0 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbffff700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 0x7fafa80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafbeffd700 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4011820 tx=0x7fafb40058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c969a1cb465704c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafa5ffb700 1 -- 172.21.15.93:0/634728430 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004830 con 0x557c2081c6b0 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafa5ffb700 1 -- 172.21.15.93:0/634728430 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb4004e50 con 0x557c2081c6b0 2023-07-21T22:33:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.400+0000 7fafa5ffb700 1 -- 172.21.15.93:0/634728430 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb401e650 con 0x557c2081c6b0 2023-07-21T22:33:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- 172.21.15.93:0/634728430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 msgr2=0x557c209c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafbffff700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 0x7fafa80a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafb80994f0 tx=0x7fafb8009550 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4011820 tx=0x7fafb40058e0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- 172.21.15.93:0/634728430 shutdown_connections 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6660 0x557c209c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c2081c6b0 0x557c209c5cd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 --2- 172.21.15.93:0/634728430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6210 0x557c209bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- 172.21.15.93:0/634728430 >> 172.21.15.93:0/634728430 conn(0x557c209b5ae0 msgr2=0x557c209bb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- 172.21.15.93:0/634728430 shutdown_connections 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- 172.21.15.93:0/634728430 wait complete. 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.401+0000 7fafd2285b80 1 -- start start 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.402+0000 7fafd2285b80 1 Processor -- start 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 0x557c209cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 0x557c209c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 0x557c209c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c208b1a60 con 0x557c209c6210 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c208b1760 con 0x557c209c6660 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c207958c0 con 0x557c2081c6b0 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafd2285b80 10 client.0 _ll_register_callbacks cb 0x557c208af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 0x557c209c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 0x557c209c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60134/0 (socket says 172.21.15.93:60134) 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 -- 172.21.15.93:0/2267849452 learned_addr learned my addr 172.21.15.93:0/2267849452 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 0x557c209cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 0x557c209c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 -- 172.21.15.93:0/2267849452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 msgr2=0x557c209c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 0x557c209c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 -- 172.21.15.93:0/2267849452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 msgr2=0x557c209c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 0x557c209c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbeffd700 1 -- 172.21.15.93:0/2267849452 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4019040 con 0x557c2081c6b0 2023-07-21T22:33:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.403+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 0x557c209c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 0x557c209cc540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005820 tx=0x7fafb4005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e991b3c71496b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004be0 con 0x557c2081c6b0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c2086b490 con 0x557c2081c6b0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fafb401e3f0 con 0x557c2081c6b0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb4004020 con 0x557c2081c6b0 2023-07-21T22:33:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 10 client.26791 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x557c20728c00 con 0x557c2081c6b0 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c2086b490 con 0x557c2081c6b0 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fafb401e5b0 con 0x557c2081c6b0 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 10 client.26791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 20 client.26791 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.404+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c209d9950 0x557c209e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.405+0000 7fafd2285b80 4 client.26791 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.405+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c2090ae50 con 0x557c209d9950 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.405+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c209d9950 0x557c209e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.405+0000 7faf9effd700 10 client.26791 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.405+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c209d9950 0x557c209e48e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602cf884d02d4eb2 server_cookie=9e75a37e00ad2a81 in_seq=0 out_seq=0 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7faf9effd700 1 -- 172.21.15.93:0/2267849452 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c2090ae50 con 0x557c209d9950 2023-07-21T22:33:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7faf9effd700 10 client.26791 handle_command_reply: tid=0 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 client.26791 shutdown 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 msgr2=0x557c209cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 0x557c209cc540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005820 tx=0x7fafb4005470 comp rx=0 tx=0).stop 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 shutdown_connections 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c209d9950 0x557c209e48e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6660 0x557c209c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c2081c6b0 0x557c209cc540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2267849452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c209c6210 0x557c209c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 >> 172.21.15.93:0/2267849452 conn(0x557c209b5ae0 msgr2=0x557c209ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7faf9effd700 0 client.26791 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 shutdown_connections 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 1 -- 172.21.15.93:0/2267849452 wait complete. 2023-07-21T22:33:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.406+0000 7fafd2285b80 20 client.26791 trim_cache size 0 max 16384 2023-07-21T22:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 Processor -- start 2023-07-21T22:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 -- start start 2023-07-21T22:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209bc760 con 0x557c2081c6b0 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209bff40 con 0x557c209c6210 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafd2285b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209c7d90 con 0x557c209c6660 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48976/0 (socket says 172.21.15.93:48976) 2023-07-21T22:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.427+0000 7fafbeffd700 1 -- 172.21.15.93:0/1291654676 learned_addr learned my addr 172.21.15.93:0/1291654676 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 -- 172.21.15.93:0/1291654676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 msgr2=0x557c209c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 -- 172.21.15.93:0/1291654676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 msgr2=0x557c209c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 -- 172.21.15.93:0/1291654676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c207661f0 con 0x557c209c6210 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209d0c30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fafb00098e0 tx=0x7fafb000b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad68a3deac1cf8e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafa67fc700 1 -- 172.21.15.93:0/1291654676 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb001c040 con 0x557c209c6210 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafa67fc700 1 -- 172.21.15.93:0/1291654676 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb000bda0 con 0x557c209c6210 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.428+0000 7fafa67fc700 1 -- 172.21.15.93:0/1291654676 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0004ab0 con 0x557c209c6210 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- 172.21.15.93:0/1291654676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 msgr2=0x557c209d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209d0c30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fafb00098e0 tx=0x7fafb000b630 comp rx=0 tx=0).stop 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- 172.21.15.93:0/1291654676 shutdown_connections 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 --2- 172.21.15.93:0/1291654676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209d0c30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- 172.21.15.93:0/1291654676 >> 172.21.15.93:0/1291654676 conn(0x557c209b5ae0 msgr2=0x557c209d5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- 172.21.15.93:0/1291654676 shutdown_connections 2023-07-21T22:33:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- 172.21.15.93:0/1291654676 wait complete. 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.429+0000 7fafd2285b80 1 -- start start 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 Processor -- start 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209c2530 con 0x557c2081c6b0 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209c26a0 con 0x557c209c6210 2023-07-21T22:33:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c209c2810 con 0x557c209c6660 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafd2285b80 10 client.0 _ll_register_callbacks cb 0x7fafb4005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48982/0 (socket says 172.21.15.93:48982) 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 -- 172.21.15.93:0/2292756691 learned_addr learned my addr 172.21.15.93:0/2292756691 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 -- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 msgr2=0x557c209c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 -- 172.21.15.93:0/2292756691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 msgr2=0x557c209c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.430+0000 7fafbeffd700 1 -- 172.21.15.93:0/2292756691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc010d210 con 0x557c2081c6b0 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafbeffd700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fafb401e9c0 tx=0x7fafb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d2ec0eebc44ca8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:33:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb401ec30 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fafb4004bf0 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb4004e50 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb401e3f0 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 10 client.17511 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x557c2073e250 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb4004e50 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fafb4003680 con 0x557c2081c6b0 2023-07-21T22:33:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 10 client.17511 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 20 client.17511 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c20a47500 0x557c20a498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 4 client.17511 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.431+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c208ccff0 con 0x557c20a47500 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.432+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c20a47500 0x557c20a498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.432+0000 7faf9effd700 10 client.17511 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.432+0000 7fafbf7fe700 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c20a47500 0x557c20a498e0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18961719e9355663 server_cookie=c4e0b78be6e4cb42 in_seq=0 out_seq=0 2023-07-21T22:33:45.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 171704 35.77 MB/sec execute 141 sec latency 46.911 ms 2023-07-21T22:33:45.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:45 smithi134 ceph-mon[97873]: pgmap v550: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 751 op/s 2023-07-21T22:33:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:45 smithi093 ceph-mon[96579]: pgmap v550: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 751 op/s 2023-07-21T22:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7faf9effd700 1 -- 172.21.15.93:0/2292756691 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c208ccff0 con 0x557c20a47500 2023-07-21T22:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7faf9effd700 10 client.17511 handle_command_reply: tid=0 2023-07-21T22:33:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 client.17511 shutdown 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 msgr2=0x557c209d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fafb401e9c0 tx=0x7fafb4000bc0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 shutdown_connections 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x557c20a47500 0x557c20a498e0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557c209c6660 0x557c209c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7faf9effd700 0 client.17511 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:45.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c2081c6b0 0x557c209d0e30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 --2- 172.21.15.93:0/2292756691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c209c6210 0x557c209c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 >> 172.21.15.93:0/2292756691 conn(0x557c209b5ae0 msgr2=0x557c209c3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 shutdown_connections 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.674+0000 7fafd2285b80 1 -- 172.21.15.93:0/2292756691 wait complete. 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.675+0000 7fafd2285b80 20 client.17511 trim_cache size 0 max 16384 2023-07-21T22:33:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:33:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:33:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:33:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 msgr2=0x7fafa80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 0x7fafa80a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fafb80994f0 tx=0x7fafb8009550 comp rx=0 tx=0).stop 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 msgr2=0x7fafc0195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007ee0 tx=0x7fafb800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 shutdown_connections 2023-07-21T22:33:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafa80a1340 0x7fafa80a37f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fafc0104e50 0x7fafc0195a10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafc0067c10 0x7fafc019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 --2- 172.21.15.93:0/818328183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafc00685b0 0x7fafc019c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 >> 172.21.15.93:0/818328183 conn(0x7fafc0074760 msgr2=0x7fafc01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 shutdown_connections 2023-07-21T22:33:45.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:45.678+0000 7fafc6100700 1 -- 172.21.15.93:0/818328183 wait complete. 2023-07-21T22:33:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:45 smithi194 ceph-mon[98288]: pgmap v550: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 751 op/s 2023-07-21T22:33:46.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 172271 35.78 MB/sec execute 142 sec latency 212.204 ms 2023-07-21T22:33:47.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 172804 35.70 MB/sec execute 143 sec latency 67.234 ms 2023-07-21T22:33:47.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:47 smithi134 ceph-mon[97873]: pgmap v551: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T22:33:47.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:47 smithi093 ceph-mon[96579]: pgmap v551: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T22:33:47.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:47 smithi194 ceph-mon[98288]: pgmap v551: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.1 MiB/s wr, 486 op/s 2023-07-21T22:33:48.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 173215 35.58 MB/sec execute 144 sec latency 372.996 ms 2023-07-21T22:33:49.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 173234 35.34 MB/sec execute 145 sec latency 1261.752 ms 2023-07-21T22:33:49.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:49 smithi093 ceph-mon[96579]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 784 op/s 2023-07-21T22:33:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:49 smithi194 ceph-mon[98288]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 784 op/s 2023-07-21T22:33:49.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:49 smithi134 ceph-mon[97873]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 784 op/s 2023-07-21T22:33:50.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 173582 35.23 MB/sec execute 146 sec latency 1317.855 ms 2023-07-21T22:33:51.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 173937 35.12 MB/sec execute 147 sec latency 149.640 ms 2023-07-21T22:33:51.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:51 smithi093 ceph-mon[96579]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 580 op/s 2023-07-21T22:33:51.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:51 smithi194 ceph-mon[98288]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 580 op/s 2023-07-21T22:33:51.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:51 smithi134 ceph-mon[97873]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.4 MiB/s wr, 580 op/s 2023-07-21T22:33:52.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 174657 35.11 MB/sec execute 148 sec latency 101.443 ms 2023-07-21T22:33:53.515 INFO:tasks.workunit.client.0.smithi093.stdout: 10 175366 35.18 MB/sec execute 149 sec latency 90.772 ms 2023-07-21T22:33:53.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:53 smithi093 ceph-mon[96579]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-07-21T22:33:53.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:53 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:53.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:53 smithi194 ceph-mon[98288]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-07-21T22:33:53.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:53 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:53.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:53 smithi134 ceph-mon[97873]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 745 op/s 2023-07-21T22:33:53.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:53 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:54.516 INFO:tasks.workunit.client.0.smithi093.stdout: 10 176048 35.22 MB/sec execute 150 sec latency 60.803 ms 2023-07-21T22:33:55.516 INFO:tasks.workunit.client.0.smithi093.stdout: 10 176723 35.22 MB/sec execute 151 sec latency 56.503 ms 2023-07-21T22:33:55.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:55 smithi093 ceph-mon[96579]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-07-21T22:33:55.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:55 smithi194 ceph-mon[98288]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-07-21T22:33:55.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:55 smithi134 ceph-mon[97873]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 25 MiB/s wr, 1.15k op/s 2023-07-21T22:33:56.516 INFO:tasks.workunit.client.0.smithi093.stdout: 10 177340 35.17 MB/sec execute 152 sec latency 96.620 ms 2023-07-21T22:33:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:33:56.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:33:56.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:33:57.516 INFO:tasks.workunit.client.0.smithi093.stdout: 10 178049 35.18 MB/sec execute 153 sec latency 66.041 ms 2023-07-21T22:33:57.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:57 smithi093 ceph-mon[96579]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 969 op/s 2023-07-21T22:33:57.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:57 smithi194 ceph-mon[98288]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 969 op/s 2023-07-21T22:33:57.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:57 smithi134 ceph-mon[97873]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 969 op/s 2023-07-21T22:33:58.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 178776 35.20 MB/sec execute 154 sec latency 109.679 ms 2023-07-21T22:33:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 -- 172.21.15.93:0/2907158259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 msgr2=0x7fbe901137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 --2- 172.21.15.93:0/2907158259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901137c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b0a0 tx=0x7fbe8800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 -- 172.21.15.93:0/2907158259 shutdown_connections 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 --2- 172.21.15.93:0/2907158259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901137c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 --2- 172.21.15.93:0/2907158259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe9010b970 0x7fbe9010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 --2- 172.21.15.93:0/2907158259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.831+0000 7fbe949a1700 1 -- 172.21.15.93:0/2907158259 >> 172.21.15.93:0/2907158259 conn(0x7fbe9006bd70 msgr2=0x7fbe90070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.833+0000 7fbe949a1700 1 -- 172.21.15.93:0/2907158259 shutdown_connections 2023-07-21T22:33:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.833+0000 7fbe949a1700 1 -- 172.21.15.93:0/2907158259 wait complete. 2023-07-21T22:33:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 Processor -- start 2023-07-21T22:33:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 -- start start 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe9010b970 0x7fbe9019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d050 con 0x7fbe9010cff0 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d1c0 con 0x7fbe9010c640 2023-07-21T22:33:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe949a1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d380 con 0x7fbe9010b970 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe8dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe8dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60160/0 (socket says 172.21.15.93:60160) 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.834+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/3499468535 learned_addr learned my addr 172.21.15.93:0/3499468535 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8ed9d700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/3499468535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe9010b970 msgr2=0x7fbe9019f440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe9010b970 0x7fbe9019f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 msgr2=0x7fbe901aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/3499468535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88009d00 con 0x7fbe9010c640 2023-07-21T22:33:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e9a0 tx=0x7fbe8000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe7f7fe700 1 -- 172.21.15.93:0/3499468535 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8000cbd0 con 0x7fbe9010c640 2023-07-21T22:33:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9019d660 con 0x7fbe9010c640 2023-07-21T22:33:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe7f7fe700 1 -- 172.21.15.93:0/3499468535 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8000cd30 con 0x7fbe9010c640 2023-07-21T22:33:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.835+0000 7fbe7f7fe700 1 -- 172.21.15.93:0/3499468535 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80018680 con 0x7fbe9010c640 2023-07-21T22:33:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.836+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9019db70 con 0x7fbe9010c640 2023-07-21T22:33:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.840+0000 7fbea0b26b80 1 Processor -- start 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.840+0000 7fbea0b26b80 1 -- start start 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865210 0x56062b85ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865660 0x56062b85f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b764c50 con 0x56062b6bb6b0 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b7b79e0 con 0x56062b865660 2023-07-21T22:33:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b7b0e70 con 0x56062b865210 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49004/0 (socket says 172.21.15.93:49004) 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 -- 172.21.15.93:0/2214615848 learned_addr learned my addr 172.21.15.93:0/2214615848 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865210 0x56062b85ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 -- 172.21.15.93:0/2214615848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865210 msgr2=0x56062b85ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865210 0x56062b85ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 -- 172.21.15.93:0/2214615848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865660 msgr2=0x56062b85f4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865660 0x56062b85f4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 -- 172.21.15.93:0/2214615848 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062b5ded80 con 0x56062b6bb6b0 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.841+0000 7fbe8ed9d700 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009c70 tx=0x7fbe880092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc1f876fc3c490 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.842+0000 7fbe7f7fe700 1 -- 172.21.15.93:0/3499468535 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbe80010430 con 0x7fbe9010c640 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.842+0000 7fbe7cff9700 1 -- 172.21.15.93:0/2214615848 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800e040 con 0x56062b6bb6b0 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.842+0000 7fbe7cff9700 1 -- 172.21.15.93:0/2214615848 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8800bcb0 con 0x56062b6bb6b0 2023-07-21T22:33:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.842+0000 7fbe7cff9700 1 -- 172.21.15.93:0/2214615848 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8803e6b0 con 0x56062b6bb6b0 2023-07-21T22:33:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbe7f7fe700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 0x7fbe780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbe7f7fe700 1 -- 172.21.15.93:0/3499468535 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe80014070 con 0x7fbe9010c640 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 -- 172.21.15.93:0/2214615848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 msgr2=0x56062b864cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009c70 tx=0x7fbe880092e0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 -- 172.21.15.93:0/2214615848 shutdown_connections 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865660 0x56062b85f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b864cd0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/2214615848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865210 0x56062b85ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbea0b26b80 1 -- 172.21.15.93:0/2214615848 >> 172.21.15.93:0/2214615848 conn(0x56062b854ae0 msgr2=0x56062b85a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.843+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 0x7fbe780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.844+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 0x7fbe780a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d9d0 tx=0x7fbe8400d410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.845+0000 7fbea0b26b80 1 -- 172.21.15.93:0/2214615848 shutdown_connections 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.845+0000 7fbea0b26b80 1 -- 172.21.15.93:0/2214615848 wait complete. 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 -- start start 2023-07-21T22:33:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 Processor -- start 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b86ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b868950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b750a60 con 0x56062b865210 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b750760 con 0x56062b6bb6b0 2023-07-21T22:33:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.846+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b6348c0 con 0x56062b865660 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.847+0000 7fbea0b26b80 10 client.0 _ll_register_callbacks cb 0x56062b74e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.847+0000 7fbe8e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.847+0000 7fbe8e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60172/0 (socket says 172.21.15.93:60172) 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.847+0000 7fbe8e59c700 1 -- 172.21.15.93:0/1739920277 learned_addr learned my addr 172.21.15.93:0/1739920277 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 -- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 msgr2=0x56062b868950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b868950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 -- 172.21.15.93:0/1739920277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 msgr2=0x56062b86ebd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b86ebd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 -- 172.21.15.93:0/1739920277 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90114080 con 0x56062b6bb6b0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005c50 tx=0x7fbe84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27fcc472e7984e8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84011e60 con 0x56062b6bb6b0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe8401e450 con 0x56062b6bb6b0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56062b70a490 con 0x56062b6bb6b0 2023-07-21T22:33:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8401e610 con 0x56062b6bb6b0 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbea0b26b80 10 client.26897 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.848+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56062b5c7c00 con 0x56062b6bb6b0 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56062b70a490 con 0x56062b6bb6b0 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbe84026820 con 0x56062b6bb6b0 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 10 client.26897 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 20 client.26897 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b889bd0 0x56062b88c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 4 client.26897 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.849+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56062b7a9e50 con 0x56062b889bd0 2023-07-21T22:33:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.852+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b889bd0 0x56062b88c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.854+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b889bd0 0x56062b88c040 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce428131a75910de server_cookie=5adf3c4980626d8e in_seq=0 out_seq=0 2023-07-21T22:33:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.854+0000 7fbe75ffb700 10 client.26897 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.858+0000 7fbe75ffb700 1 -- 172.21.15.93:0/1739920277 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56062b7a9e50 con 0x56062b889bd0 2023-07-21T22:33:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.858+0000 7fbe75ffb700 10 client.26897 handle_command_reply: tid=0 2023-07-21T22:33:58.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.858+0000 7fbea0b26b80 1 client.26897 shutdown 2023-07-21T22:33:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.858+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 msgr2=0x56062b86e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.858+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005c50 tx=0x7fbe84004750 comp rx=0 tx=0).stop 2023-07-21T22:33:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 shutdown_connections 2023-07-21T22:33:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b889bd0 0x56062b88c040 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b868950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b86e690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbe75ffb700 0 client.26897 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1739920277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b86ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 >> 172.21.15.93:0/1739920277 conn(0x56062b854ae0 msgr2=0x56062b85df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 shutdown_connections 2023-07-21T22:33:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.859+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1739920277 wait complete. 2023-07-21T22:33:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.860+0000 7fbea0b26b80 20 client.26897 trim_cache size 0 max 16384 2023-07-21T22:33:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.881+0000 7fbea0b26b80 1 Processor -- start 2023-07-21T22:33:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.881+0000 7fbea0b26b80 1 -- start start 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b85a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b859260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b859a10 con 0x56062b865210 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b860050 con 0x56062b6bb6b0 2023-07-21T22:33:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b866ed0 con 0x56062b865660 2023-07-21T22:33:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49014/0 (socket says 172.21.15.93:49014) 2023-07-21T22:33:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/1970921735 learned_addr learned my addr 172.21.15.93:0/1970921735 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/1970921735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 msgr2=0x56062b859260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8ed9d700 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b859260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b859260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.882+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/1970921735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 msgr2=0x56062b85a240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b85a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe8dd9b700 1 -- 172.21.15.93:0/1970921735 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062b6051f0 con 0x56062b865210 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbe80096c50 tx=0x7fbe8000e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06df37a4363d9ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe7d7fa700 1 -- 172.21.15.93:0/1970921735 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8009e210 con 0x56062b865210 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe7d7fa700 1 -- 172.21.15.93:0/1970921735 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe800a2630 con 0x56062b865210 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.883+0000 7fbe7d7fa700 1 -- 172.21.15.93:0/1970921735 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8009a6a0 con 0x56062b865210 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1970921735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 msgr2=0x56062b868a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbe80096c50 tx=0x7fbe8000e8f0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1970921735 shutdown_connections 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b859260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b6bb6b0 0x56062b85a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/1970921735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b865210 0x56062b868a80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1970921735 >> 172.21.15.93:0/1970921735 conn(0x56062b854ae0 msgr2=0x56062b858b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1970921735 shutdown_connections 2023-07-21T22:33:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.884+0000 7fbea0b26b80 1 -- 172.21.15.93:0/1970921735 wait complete. 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 -- start start 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 Processor -- start 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865210 0x56062b8d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b8dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b8dd080 con 0x56062b6bb6b0 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b8dd1c0 con 0x56062b865210 2023-07-21T22:33:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062b8dd300 con 0x56062b865660 2023-07-21T22:33:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbea0b26b80 10 client.0 _ll_register_callbacks cb 0x7fbe8400f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbe8e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbe8e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49020/0 (socket says 172.21.15.93:49020) 2023-07-21T22:33:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbe8e59c700 1 -- 172.21.15.93:0/133652874 learned_addr learned my addr 172.21.15.93:0/133652874 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:33:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.885+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865210 0x56062b8d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 -- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 msgr2=0x56062b8dcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b8dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 -- 172.21.15.93:0/133652874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865210 msgr2=0x56062b8d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865210 0x56062b8d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 -- 172.21.15.93:0/133652874 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9019db70 con 0x56062b6bb6b0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe8e59c700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005e10 tx=0x7fbe84026a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9c606bde8b8445 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe840265b0 con 0x56062b6bb6b0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbe84011ab0 con 0x56062b6bb6b0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe8800bcb0 con 0x56062b6bb6b0 2023-07-21T22:33:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe84011c90 con 0x56062b6bb6b0 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbea0b26b80 10 client.17526 fetch_fsmap learned FSMap version 80 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.886+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56062b5dd250 con 0x56062b6bb6b0 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe84041020 con 0x56062b6bb6b0 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbe840048e0 con 0x56062b6bb6b0 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 10 client.17526 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:33:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 20 client.17526 populate_metadata read hostname 'smithi093' 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b8de9a0 0x56062b8e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 4 client.17526 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56062b76bff0 con 0x56062b8de9a0 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.887+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b8de9a0 0x56062b8e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.891+0000 7fbe75ffb700 10 client.17526 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:58.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:58.891+0000 7fbe8dd9b700 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b8de9a0 0x56062b8e0db0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e6295dc9b37292 server_cookie=325f2915418897af in_seq=0 out_seq=0 2023-07-21T22:33:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbe75ffb700 1 -- 172.21.15.93:0/133652874 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56062b76bff0 con 0x56062b8de9a0 2023-07-21T22:33:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbe75ffb700 10 client.17526 handle_command_reply: tid=0 2023-07-21T22:33:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 client.17526 shutdown 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 msgr2=0x56062b8d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005e10 tx=0x7fbe84026a30 comp rx=0 tx=0).stop 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 shutdown_connections 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56062b8de9a0 0x56062b8e0db0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56062b865660 0x56062b8dcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbe75ffb700 0 client.17526 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56062b6bb6b0 0x56062b8d1cb0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 --2- 172.21.15.93:0/133652874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56062b865210 0x56062b8d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 >> 172.21.15.93:0/133652874 conn(0x56062b854ae0 msgr2=0x56062b8662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 shutdown_connections 2023-07-21T22:33:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 1 -- 172.21.15.93:0/133652874 wait complete. 2023-07-21T22:33:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.126+0000 7fbea0b26b80 20 client.17526 trim_cache size 0 max 16384 2023-07-21T22:33:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:33:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:33:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:33:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 msgr2=0x7fbe780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 0x7fbe780a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d9d0 tx=0x7fbe8400d410 comp rx=0 tx=0).stop 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 msgr2=0x7fbe9019f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e9a0 tx=0x7fbe8000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 shutdown_connections 2023-07-21T22:33:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbe780a12c0 0x7fbe780a3770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe9010cff0 0x7fbe901aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe9010b970 0x7fbe9019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 --2- 172.21.15.93:0/3499468535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe9010c640 0x7fbe9019f980 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 >> 172.21.15.93:0/3499468535 conn(0x7fbe9006bd70 msgr2=0x7fbe90109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 shutdown_connections 2023-07-21T22:33:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:33:59.127+0000 7fbe949a1700 1 -- 172.21.15.93:0/3499468535 wait complete. 2023-07-21T22:33:59.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 179372 35.20 MB/sec execute 155 sec latency 180.836 ms 2023-07-21T22:33:59.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:33:59 smithi093 ceph-mon[96579]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-07-21T22:33:59.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:33:59 smithi194 ceph-mon[98288]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-07-21T22:33:59.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:33:59 smithi134 ceph-mon[97873]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-07-21T22:34:00.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180095 35.26 MB/sec execute 156 sec latency 79.178 ms 2023-07-21T22:34:01.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180101 35.05 MB/sec execute 157 sec latency 1045.185 ms 2023-07-21T22:34:01.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:01 smithi093 ceph-mon[96579]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 915 op/s 2023-07-21T22:34:01.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:01 smithi194 ceph-mon[98288]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 915 op/s 2023-07-21T22:34:01.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:01 smithi134 ceph-mon[97873]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 21 MiB/s wr, 915 op/s 2023-07-21T22:34:02.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180101 34.83 MB/sec execute 158 sec latency 2045.317 ms 2023-07-21T22:34:03.518 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180340 34.64 MB/sec execute 159 sec latency 2419.959 ms 2023-07-21T22:34:03.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:03 smithi093 ceph-mon[96579]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 21 MiB/s wr, 939 op/s 2023-07-21T22:34:03.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:03 smithi194 ceph-mon[98288]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 21 MiB/s wr, 939 op/s 2023-07-21T22:34:03.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:03 smithi134 ceph-mon[97873]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 21 MiB/s wr, 939 op/s 2023-07-21T22:34:04.519 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180591 34.49 MB/sec execute 160 sec latency 158.931 ms 2023-07-21T22:34:05.519 INFO:tasks.workunit.client.0.smithi093.stdout: 10 180791 34.37 MB/sec execute 161 sec latency 453.430 ms 2023-07-21T22:34:05.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:05 smithi194 ceph-mon[98288]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 18 MiB/s wr, 946 op/s 2023-07-21T22:34:05.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:05 smithi194 ceph-mon[98288]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:34:05.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:05 smithi134 ceph-mon[97873]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 18 MiB/s wr, 946 op/s 2023-07-21T22:34:05.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:05 smithi134 ceph-mon[97873]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:34:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:05 smithi093 ceph-mon[96579]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 18 MiB/s wr, 946 op/s 2023-07-21T22:34:05.945 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:05 smithi093 ceph-mon[96579]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:34:06.520 INFO:tasks.workunit.client.0.smithi093.stdout: 10 181070 34.23 MB/sec execute 162 sec latency 582.650 ms 2023-07-21T22:34:07.519 INFO:tasks.workunit.client.0.smithi093.stdout: 10 181641 34.22 MB/sec execute 163 sec latency 210.017 ms 2023-07-21T22:34:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:07 smithi194 ceph-mon[98288]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 580 op/s 2023-07-21T22:34:07.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:07 smithi134 ceph-mon[97873]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 580 op/s 2023-07-21T22:34:07.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:07 smithi093 ceph-mon[96579]: pgmap v562: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 580 op/s 2023-07-21T22:34:08.520 INFO:tasks.workunit.client.0.smithi093.stdout: 10 182363 34.24 MB/sec execute 164 sec latency 65.261 ms 2023-07-21T22:34:09.519 INFO:tasks.workunit.client.0.smithi093.stdout: 10 182957 34.23 MB/sec execute 165 sec latency 77.118 ms 2023-07-21T22:34:09.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:09 smithi194 ceph-mon[98288]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 658 op/s 2023-07-21T22:34:09.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:09 smithi134 ceph-mon[97873]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 658 op/s 2023-07-21T22:34:09.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:09 smithi093 ceph-mon[96579]: pgmap v563: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.0 MiB/s wr, 658 op/s 2023-07-21T22:34:10.520 INFO:tasks.workunit.client.0.smithi093.stdout: 10 183685 34.29 MB/sec execute 166 sec latency 54.039 ms 2023-07-21T22:34:11.520 INFO:tasks.workunit.client.0.smithi093.stdout: 10 184425 34.32 MB/sec execute 167 sec latency 57.277 ms 2023-07-21T22:34:11.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:11 smithi194 ceph-mon[98288]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 661 op/s 2023-07-21T22:34:11.888 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:11 smithi134 ceph-mon[97873]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 661 op/s 2023-07-21T22:34:11.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:11 smithi093 ceph-mon[96579]: pgmap v564: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 661 op/s 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 -- 172.21.15.93:0/3613728855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 msgr2=0x7f9ae80fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 --2- 172.21.15.93:0/3613728855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae80fef30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098f0 tx=0x7f9ad8009c00 comp rx=0 tx=0).stop 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 -- 172.21.15.93:0/3613728855 shutdown_connections 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 --2- 172.21.15.93:0/3613728855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 --2- 172.21.15.93:0/3613728855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.283+0000 7f9aefc00700 1 --2- 172.21.15.93:0/3613728855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae80fef30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.284+0000 7f9aefc00700 1 -- 172.21.15.93:0/3613728855 >> 172.21.15.93:0/3613728855 conn(0x7f9ae8075bd0 msgr2=0x7f9ae8075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.284+0000 7f9aefc00700 1 -- 172.21.15.93:0/3613728855 shutdown_connections 2023-07-21T22:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.284+0000 7f9aefc00700 1 -- 172.21.15.93:0/3613728855 wait complete. 2023-07-21T22:34:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.284+0000 7f9aefc00700 1 Processor -- start 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 -- start start 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae8197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae819cf70 con 0x7f9ae8104ad0 2023-07-21T22:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae819d0e0 con 0x7f9ae80ff470 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aefc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae819d2a0 con 0x7f9ae80feac0 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aed19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aed19b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60198/0 (socket says 172.21.15.93:60198) 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aed19b700 1 -- 172.21.15.93:0/1113526664 learned_addr learned my addr 172.21.15.93:0/1113526664 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aed99c700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae8197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 -- 172.21.15.93:0/1113526664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 msgr2=0x7f9ae8197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae8197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 -- 172.21.15.93:0/1113526664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 msgr2=0x7f9ae81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aee19d700 1 -- 172.21.15.93:0/1113526664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40095f0 con 0x7f9ae8104ad0 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.285+0000 7f9aed19b700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9aee19d700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae819c930 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000efc0 tx=0x7f9ae000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9adeffd700 1 -- 172.21.15.93:0/1113526664 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0018070 con 0x7f9ae8104ad0 2023-07-21T22:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8009520 con 0x7f9ae8104ad0 2023-07-21T22:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9adeffd700 1 -- 172.21.15.93:0/1113526664 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ae000f040 con 0x7f9ae8104ad0 2023-07-21T22:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9adeffd700 1 -- 172.21.15.93:0/1113526664 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000cd40 con 0x7f9ae8104ad0 2023-07-21T22:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.286+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae81a4be0 con 0x7f9ae8104ad0 2023-07-21T22:34:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.288+0000 7f9adeffd700 1 -- 172.21.15.93:0/1113526664 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9ae0003a00 con 0x7f9ae8104ad0 2023-07-21T22:34:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.289+0000 7f9adeffd700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 0x7f9ad40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.289+0000 7f9adeffd700 1 -- 172.21.15.93:0/1113526664 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9ae00cc8f0 con 0x7f9ae8104ad0 2023-07-21T22:34:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.289+0000 7f9aed99c700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 0x7f9ad40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.289+0000 7f9aed99c700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 0x7f9ad40a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ffc00 tx=0x7f9ae4009310 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 Processor -- start 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 -- start start 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 0x55e33b1c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 0x55e33b1c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b0c9c50 con 0x55e33b0206b0 2023-07-21T22:34:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b11c9e0 con 0x55e33b1ca210 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.290+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b115e70 con 0x55e33b1ca660 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed99c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 0x55e33b1c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed99c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 0x55e33b1c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49042/0 (socket says 172.21.15.93:49042) 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed99c700 1 -- 172.21.15.93:0/3350644343 learned_addr learned my addr 172.21.15.93:0/3350644343 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 0x55e33b1c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aee19d700 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 -- 172.21.15.93:0/3350644343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 msgr2=0x55e33b1c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 -- 172.21.15.93:0/3350644343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 msgr2=0x55e33b1c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 0x55e33b1c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 -- 172.21.15.93:0/3350644343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33af43d80 con 0x55e33b1ca210 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.291+0000 7f9aed19b700 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 0x55e33b1c3f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098c0 tx=0x7f9ad8009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394d205d07dbaff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.292+0000 7f9ad3fff700 1 -- 172.21.15.93:0/3350644343 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801c050 con 0x55e33b1ca210 2023-07-21T22:34:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.292+0000 7f9ad3fff700 1 -- 172.21.15.93:0/3350644343 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad801d040 con 0x55e33b1ca210 2023-07-21T22:34:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.292+0000 7f9ad3fff700 1 -- 172.21.15.93:0/3350644343 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80052a0 con 0x55e33b1ca210 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3350644343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 msgr2=0x55e33b1c3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 0x55e33b1c3f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098c0 tx=0x7f9ad8009400 comp rx=0 tx=0).stop 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3350644343 shutdown_connections 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b0206b0 0x55e33b1c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3350644343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca210 0x55e33b1c3f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.293+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3350644343 >> 172.21.15.93:0/3350644343 conn(0x55e33b1b9ae0 msgr2=0x55e33b1bf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.295+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3350644343 shutdown_connections 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.295+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3350644343 wait complete. 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 -- start start 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 Processor -- start 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 0x55e33b1d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b0b5a60 con 0x55e33b1ca210 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b0b5760 con 0x55e33b0206b0 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33af998c0 con 0x55e33b1ca660 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.296+0000 7f9afbd85b80 10 client.0 _ll_register_callbacks cb 0x55e33b0b3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aee19d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aee19d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36128/0 (socket says 172.21.15.93:36128) 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aee19d700 1 -- 172.21.15.93:0/2285444818 learned_addr learned my addr 172.21.15.93:0/2285444818 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed99c700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 0x55e33b1d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 -- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 msgr2=0x55e33b1cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 -- 172.21.15.93:0/2285444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 msgr2=0x55e33b1d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 0x55e33b1d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 -- 172.21.15.93:0/2285444818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae810c200 con 0x55e33b1ca210 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed99c700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 0x55e33b1d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9aed19b700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d3b20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8914612d0da0349 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.297+0000 7f9ad17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800bb70 con 0x55e33b1ca210 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e33b06f490 con 0x55e33b1ca210 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9ad80050c0 con 0x55e33b1ca210 2023-07-21T22:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad80052c0 con 0x55e33b1ca210 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 10 client.17538 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55e33af2cc00 con 0x55e33b1ca210 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e33b06f490 con 0x55e33b1ca210 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9ad8009c70 con 0x55e33b1ca210 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 10 client.17538 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 20 client.17538 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b1dd560 0x55e33b1df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 4 client.17538 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e33b10ee50 con 0x55e33b1dd560 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.298+0000 7f9aed99c700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b1dd560 0x55e33b1df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.299+0000 7f9ad17fa700 10 client.17538 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.299+0000 7f9aed99c700 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b1dd560 0x55e33b1df970 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8393252ed491dd72 server_cookie=176a18f50f70ef7d in_seq=0 out_seq=0 2023-07-21T22:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9ad17fa700 1 -- 172.21.15.93:0/2285444818 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e33b10ee50 con 0x55e33b1dd560 2023-07-21T22:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9ad17fa700 10 client.17538 handle_command_reply: tid=0 2023-07-21T22:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 client.17538 shutdown 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 msgr2=0x55e33b1d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d3b20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad800b770 comp rx=0 tx=0).stop 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 shutdown_connections 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b1dd560 0x55e33b1df970 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b1ca660 0x55e33b1cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b0206b0 0x55e33b1d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/2285444818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d3b20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.301+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 >> 172.21.15.93:0/2285444818 conn(0x55e33b1b9ae0 msgr2=0x55e33b1be3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.302+0000 7f9ad17fa700 0 client.17538 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.302+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 shutdown_connections 2023-07-21T22:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.303+0000 7f9afbd85b80 1 -- 172.21.15.93:0/2285444818 wait complete. 2023-07-21T22:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.303+0000 7f9afbd85b80 20 client.17538 trim_cache size 0 max 16384 2023-07-21T22:34:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 Processor -- start 2023-07-21T22:34:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 -- start start 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1c9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b1bbc60 con 0x55e33b1ca210 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b1c8320 con 0x55e33b1ca660 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b1cee60 con 0x55e33b0206b0 2023-07-21T22:34:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9aed19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9aed19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49056/0 (socket says 172.21.15.93:49056) 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9aed19b700 1 -- 172.21.15.93:0/1042355415 learned_addr learned my addr 172.21.15.93:0/1042355415 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9aed99c700 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.324+0000 7f9aee19d700 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 -- 172.21.15.93:0/1042355415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 msgr2=0x55e33b1c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 -- 172.21.15.93:0/1042355415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 msgr2=0x55e33b1bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 -- 172.21.15.93:0/1042355415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e33b1cd3f0 con 0x55e33b1ca210 2023-07-21T22:34:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9aed19b700 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800a010 tx=0x7f9ad8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26a217be1d2e60b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9adcdac700 1 -- 172.21.15.93:0/1042355415 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801c040 con 0x55e33b1ca210 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9adcdac700 1 -- 172.21.15.93:0/1042355415 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad802c6a0 con 0x55e33b1ca210 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.325+0000 7f9adcdac700 1 -- 172.21.15.93:0/1042355415 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80050e0 con 0x55e33b1ca210 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- 172.21.15.93:0/1042355415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 msgr2=0x55e33b1beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800a010 tx=0x7f9ad8005a00 comp rx=0 tx=0).stop 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- 172.21.15.93:0/1042355415 shutdown_connections 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/1042355415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1beb60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- 172.21.15.93:0/1042355415 >> 172.21.15.93:0/1042355415 conn(0x55e33b1b9ae0 msgr2=0x55e33b1c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- 172.21.15.93:0/1042355415 shutdown_connections 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- 172.21.15.93:0/1042355415 wait complete. 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.326+0000 7f9afbd85b80 1 -- start start 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 Processor -- start 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b230420 con 0x55e33b1ca210 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b230560 con 0x55e33b1ca660 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e33b2306a0 con 0x55e33b0206b0 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9afbd85b80 10 client.0 _ll_register_callbacks cb 0x7f9ae4009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:12.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9aee19d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9aee19d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60226/0 (socket says 172.21.15.93:60226) 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9aee19d700 1 -- 172.21.15.93:0/3611212648 learned_addr learned my addr 172.21.15.93:0/3611212648 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.327+0000 7f9aed99c700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aed19b700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 -- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 msgr2=0x55e33b1d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 -- 172.21.15.93:0/3611212648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 msgr2=0x55e33b1d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 -- 172.21.15.93:0/3611212648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e33af2cc00 con 0x55e33b1ca660 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9aee19d700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fd0 tx=0x7f9ae0009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e255ef7be667267f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9ad17fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0099eb0 con 0x55e33b1ca660 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9ae0099a50 con 0x55e33b1ca660 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ad801d040 con 0x55e33b1ca660 2023-07-21T22:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae009da80 con 0x55e33b1ca660 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9afbd85b80 10 client.26918 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.328+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55e33af6a1f0 con 0x55e33b1ca660 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ae0063080 con 0x55e33b1ca660 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9ae00a1aa0 con 0x55e33b1ca660 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 20 client.26918 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b231c10 0x55e33b234020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 4 client.26918 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e33b0d0ff0 con 0x55e33b231c10 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9aed99c700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b231c10 0x55e33b234020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9ad17fa700 10 client.26918 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.329+0000 7f9aed99c700 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b231c10 0x55e33b234020 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6359f61d40f3a989 server_cookie=5ae3c61cd4397f36 in_seq=0 out_seq=0 2023-07-21T22:34:12.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 184970 34.28 MB/sec execute 168 sec latency 132.398 ms 2023-07-21T22:34:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9ad17fa700 1 -- 172.21.15.93:0/3611212648 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e33b0d0ff0 con 0x55e33b231c10 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9ad17fa700 10 client.26918 handle_command_reply: tid=0 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 client.26918 shutdown 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 msgr2=0x55e33b1d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009fd0 tx=0x7f9ae0009370 comp rx=0 tx=0).stop 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 shutdown_connections 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e33b231c10 0x55e33b234020 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e33b1ca660 0x55e33b1d1bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e33b0206b0 0x55e33b1d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 --2- 172.21.15.93:0/3611212648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e33b1ca210 0x55e33b1d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9ad17fa700 0 client.26918 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.555+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 >> 172.21.15.93:0/3611212648 conn(0x55e33b1b9ae0 msgr2=0x55e33b1c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.556+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 shutdown_connections 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.556+0000 7f9afbd85b80 1 -- 172.21.15.93:0/3611212648 wait complete. 2023-07-21T22:34:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.556+0000 7f9afbd85b80 20 client.26918 trim_cache size 0 max 16384 2023-07-21T22:34:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:34:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:34:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:34:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:34:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 msgr2=0x7f9ad40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 0x7f9ad40a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80ffc00 tx=0x7f9ae4009310 comp rx=0 tx=0).stop 2023-07-21T22:34:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 msgr2=0x7f9ae819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae819c930 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000efc0 tx=0x7f9ae000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 shutdown_connections 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9ad40a12c0 0x7f9ad40a3770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ae8104ad0 0x7f9ae819c930 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ae80feac0 0x7f9ae8197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 --2- 172.21.15.93:0/1113526664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae80ff470 0x7f9ae81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 >> 172.21.15.93:0/1113526664 conn(0x7f9ae8075bd0 msgr2=0x7f9ae80687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 shutdown_connections 2023-07-21T22:34:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:12.557+0000 7f9aefc00700 1 -- 172.21.15.93:0/1113526664 wait complete. 2023-07-21T22:34:13.521 INFO:tasks.workunit.client.0.smithi093.stdout: 10 185655 34.29 MB/sec execute 169 sec latency 179.944 ms 2023-07-21T22:34:13.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:13 smithi194 ceph-mon[98288]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 628 op/s 2023-07-21T22:34:13.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:13 smithi194 ceph-mon[98288]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:34:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:13 smithi134 ceph-mon[97873]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 628 op/s 2023-07-21T22:34:13.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:13 smithi134 ceph-mon[97873]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:34:13.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:13 smithi093 ceph-mon[96579]: pgmap v565: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 628 op/s 2023-07-21T22:34:13.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:13 smithi093 ceph-mon[96579]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:34:14.521 INFO:tasks.workunit.client.0.smithi093.stdout: 10 186357 34.31 MB/sec execute 170 sec latency 83.950 ms 2023-07-21T22:34:15.521 INFO:tasks.workunit.client.0.smithi093.stdout: 10 186888 34.30 MB/sec execute 171 sec latency 108.871 ms 2023-07-21T22:34:15.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:15 smithi194 ceph-mon[98288]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 709 op/s 2023-07-21T22:34:15.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:15 smithi134 ceph-mon[97873]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 709 op/s 2023-07-21T22:34:15.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:15 smithi093 ceph-mon[96579]: pgmap v567: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 709 op/s 2023-07-21T22:34:16.521 INFO:tasks.workunit.client.0.smithi093.stdout: 10 187593 34.29 MB/sec execute 172 sec latency 54.767 ms 2023-07-21T22:34:17.521 INFO:tasks.workunit.client.0.smithi093.stdout: 10 188303 34.32 MB/sec execute 173 sec latency 82.213 ms 2023-07-21T22:34:17.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:17 smithi194 ceph-mon[98288]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2023-07-21T22:34:17.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:17 smithi134 ceph-mon[97873]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2023-07-21T22:34:17.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:17 smithi093 ceph-mon[96579]: pgmap v568: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.1 MiB/s wr, 698 op/s 2023-07-21T22:34:18.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 188960 34.34 MB/sec execute 174 sec latency 110.004 ms 2023-07-21T22:34:19.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 189502 34.31 MB/sec execute 175 sec latency 96.998 ms 2023-07-21T22:34:19.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:19 smithi194 ceph-mon[98288]: pgmap v569: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2023-07-21T22:34:19.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:19 smithi134 ceph-mon[97873]: pgmap v569: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2023-07-21T22:34:19.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:19 smithi093 ceph-mon[96579]: pgmap v569: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 679 op/s 2023-07-21T22:34:20.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 190099 34.29 MB/sec execute 176 sec latency 146.048 ms 2023-07-21T22:34:21.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 190655 34.24 MB/sec execute 177 sec latency 267.830 ms 2023-07-21T22:34:21.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:21 smithi194 ceph-mon[98288]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 652 op/s 2023-07-21T22:34:21.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:21 smithi134 ceph-mon[97873]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 652 op/s 2023-07-21T22:34:21.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:21 smithi093 ceph-mon[96579]: pgmap v570: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.7 MiB/s wr, 652 op/s 2023-07-21T22:34:22.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 191337 34.29 MB/sec execute 178 sec latency 63.346 ms 2023-07-21T22:34:23.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 191996 34.26 MB/sec execute 179 sec latency 55.895 ms 2023-07-21T22:34:23.591 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:23 smithi194 ceph-mon[98288]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.9 MiB/s wr, 639 op/s 2023-07-21T22:34:23.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:23 smithi134 ceph-mon[97873]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.9 MiB/s wr, 639 op/s 2023-07-21T22:34:23.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:23 smithi093 ceph-mon[96579]: pgmap v571: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.9 MiB/s wr, 639 op/s 2023-07-21T22:34:24.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 192513 34.24 MB/sec execute 180 sec latency 234.198 ms 2023-07-21T22:34:25.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 193085 34.27 MB/sec execute 181 sec latency 96.054 ms 2023-07-21T22:34:25.590 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:25 smithi194 ceph-mon[98288]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 733 op/s 2023-07-21T22:34:25.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:25 smithi134 ceph-mon[97873]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 733 op/s 2023-07-21T22:34:25.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:25 smithi093 ceph-mon[96579]: pgmap v572: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 733 op/s 2023-07-21T22:34:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 -- 172.21.15.93:0/2986219906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c104db0 msgr2=0x7f494c10c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 --2- 172.21.15.93:0/2986219906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c104db0 0x7f494c10c980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f494400b1d0 tx=0x7f494400b4e0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 -- 172.21.15.93:0/2986219906 shutdown_connections 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 --2- 172.21.15.93:0/2986219906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c104db0 0x7f494c10c980 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 --2- 172.21.15.93:0/2986219906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fe260 0x7f494c0fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 --2- 172.21.15.93:0/2986219906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c0fec00 0x7f494c1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 -- 172.21.15.93:0/2986219906 >> 172.21.15.93:0/2986219906 conn(0x7f494c075f90 msgr2=0x7f494c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 -- 172.21.15.93:0/2986219906 shutdown_connections 2023-07-21T22:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.697+0000 7f4951fa9700 1 -- 172.21.15.93:0/2986219906 wait complete. 2023-07-21T22:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 Processor -- start 2023-07-21T22:34:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 -- start start 2023-07-21T22:34:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 0x7f494c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c104db0 0x7f494c193820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c193e80 con 0x7f494c0fe260 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c193ff0 con 0x7f494c0fec00 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f4951fa9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1941b0 con 0x7f494c104db0 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f494b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f494b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49074/0 (socket says 172.21.15.93:49074) 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f494b7fe700 1 -- 172.21.15.93:0/4046443327 learned_addr learned my addr 172.21.15.93:0/4046443327 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.698+0000 7f494affd700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 0x7f494c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494bfff700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c104db0 0x7f494c193820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 -- 172.21.15.93:0/4046443327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c104db0 msgr2=0x7f494c193820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c104db0 0x7f494c193820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 -- 172.21.15.93:0/4046443327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 msgr2=0x7f494c199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 0x7f494c199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 -- 172.21.15.93:0/4046443327 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c009520 con 0x7f494c0fe260 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494affd700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 0x7f494c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f494b7fe700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f493c00eb30 tx=0x7f493c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f4948ff9700 1 -- 172.21.15.93:0/4046443327 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c009880 con 0x7f494c0fe260 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.699+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4944009d00 con 0x7f494c0fe260 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.700+0000 7f4948ff9700 1 -- 172.21.15.93:0/4046443327 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493c0044e0 con 0x7f494c0fe260 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.700+0000 7f4948ff9700 1 -- 172.21.15.93:0/4046443327 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c003d80 con 0x7f494c0fe260 2023-07-21T22:34:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.700+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a0760 con 0x7f494c0fe260 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.702+0000 7f4948ff9700 1 -- 172.21.15.93:0/4046443327 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f493c003fe0 con 0x7f494c0fe260 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.703+0000 7f4948ff9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 0x7f49340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.703+0000 7f494affd700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 0x7f49340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.703+0000 7f4948ff9700 1 -- 172.21.15.93:0/4046443327 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f493c014070 con 0x7f494c0fe260 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.704+0000 7f494affd700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 0x7f49340a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f494c194a30 tx=0x7f4940009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 Processor -- start 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 -- start start 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05677a6b0 0x55b056923cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 0x55b05691df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b056823c50 con 0x55b056924660 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0568769e0 con 0x55b05677a6b0 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f495e12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05686fe70 con 0x55b056924210 2023-07-21T22:34:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49084/0 (socket says 172.21.15.93:49084) 2023-07-21T22:34:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 -- 172.21.15.93:0/2905879453 learned_addr learned my addr 172.21.15.93:0/2905879453 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05677a6b0 0x55b056923cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494affd700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 0x55b05691df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 -- 172.21.15.93:0/2905879453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 msgr2=0x55b05691df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 0x55b05691df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 -- 172.21.15.93:0/2905879453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05677a6b0 msgr2=0x55b056923cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05677a6b0 0x55b056923cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494bfff700 1 -- 172.21.15.93:0/2905879453 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05669dd80 con 0x55b056924660 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.705+0000 7f494affd700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 0x55b05691df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.706+0000 7f494bfff700 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f494400b160 tx=0x7f49440061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db99e5e93b3065 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.706+0000 7f4931ffb700 1 -- 172.21.15.93:0/2905879453 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400bb00 con 0x55b056924660 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.706+0000 7f4931ffb700 1 -- 172.21.15.93:0/2905879453 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f494400be90 con 0x55b056924660 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.706+0000 7f4931ffb700 1 -- 172.21.15.93:0/2905879453 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944003c20 con 0x55b056924660 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- 172.21.15.93:0/2905879453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 msgr2=0x55b05691e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f494400b160 tx=0x7f49440061f0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- 172.21.15.93:0/2905879453 shutdown_connections 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b05691e4a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05677a6b0 0x55b056923cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2905879453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b056924210 0x55b05691df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- 172.21.15.93:0/2905879453 >> 172.21.15.93:0/2905879453 conn(0x55b056913ae0 msgr2=0x55b0569190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- 172.21.15.93:0/2905879453 shutdown_connections 2023-07-21T22:34:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- 172.21.15.93:0/2905879453 wait complete. 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.707+0000 7f495e12eb80 1 -- start start 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 Processor -- start 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05692d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b05692db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05680fa60 con 0x55b056924660 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05680f760 con 0x55b056924210 2023-07-21T22:34:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0566f38c0 con 0x55b05677a6b0 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f495e12eb80 10 client.0 _ll_register_callbacks cb 0x55b05680d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05692d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05692d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36168/0 (socket says 172.21.15.93:36168) 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49092/0 (socket says 172.21.15.93:49092) 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494b7fe700 1 -- 172.21.15.93:0/2080598352 learned_addr learned my addr 172.21.15.93:0/2080598352 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.708+0000 7f494affd700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b05692db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 -- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 msgr2=0x55b05692d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05692d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 -- 172.21.15.93:0/2080598352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 msgr2=0x55b05692db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b05692db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 -- 172.21.15.93:0/2080598352 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494401c040 con 0x55b056924660 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f494bfff700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49440062a0 tx=0x7f49440099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca6a0cf6c8dda server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f492affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:34:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400bb00 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f49440074e0 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0567c9490 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4944007e50 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f495e12eb80 10 client.17556 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.709+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55b056686c00 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b0567c9490 con 0x55b056924660 2023-07-21T22:34:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4944029070 con 0x55b056924660 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 10 client.17556 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 10 client.17556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 20 client.17556 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b056937620 0x55b056939a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 4 client.17556 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b056868e50 con 0x55b056937620 2023-07-21T22:34:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.710+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b056937620 0x55b056939a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.711+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b056937620 0x55b056939a00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6bf0950751732b server_cookie=1906053f506d5e2d in_seq=0 out_seq=0 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.711+0000 7f492affd700 10 client.17556 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f492affd700 1 -- 172.21.15.93:0/2080598352 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b056868e50 con 0x55b056937620 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f492affd700 10 client.17556 handle_command_reply: tid=0 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 client.17556 shutdown 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 msgr2=0x55b0569278b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49440062a0 tx=0x7f49440099e0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 shutdown_connections 2023-07-21T22:34:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b056937620 0x55b056939a00 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569278b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05692d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2080598352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b05692db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.712+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 >> 172.21.15.93:0/2080598352 conn(0x55b056913ae0 msgr2=0x55b0569183e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.713+0000 7f492affd700 0 client.17556 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.713+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 shutdown_connections 2023-07-21T22:34:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.713+0000 7f495e12eb80 1 -- 172.21.15.93:0/2080598352 wait complete. 2023-07-21T22:34:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.713+0000 7f495e12eb80 20 client.17556 trim_cache size 0 max 16384 2023-07-21T22:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 Processor -- start 2023-07-21T22:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 -- start start 2023-07-21T22:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05691df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b056918b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b056923920 con 0x55b056924660 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b056919ef0 con 0x55b056924210 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f495e12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b056925e10 con 0x55b05677a6b0 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49100/0 (socket says 172.21.15.93:49100) 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f494bfff700 1 -- 172.21.15.93:0/201471668 learned_addr learned my addr 172.21.15.93:0/201471668 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f494b7fe700 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05691df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.730+0000 7f494affd700 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b056918b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 -- 172.21.15.93:0/201471668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 msgr2=0x55b05691df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05691df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 -- 172.21.15.93:0/201471668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 msgr2=0x55b056918b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b056918b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 -- 172.21.15.93:0/201471668 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b056927420 con 0x55b056924660 2023-07-21T22:34:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f494bfff700 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4944009850 tx=0x7f49440061a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859e1051820cc0a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f49327fc700 1 -- 172.21.15.93:0/201471668 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494400bb00 con 0x55b056924660 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f49327fc700 1 -- 172.21.15.93:0/201471668 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4944005250 con 0x55b056924660 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.731+0000 7f49327fc700 1 -- 172.21.15.93:0/201471668 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944005710 con 0x55b056924660 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 -- 172.21.15.93:0/201471668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 msgr2=0x55b0569190a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4944009850 tx=0x7f49440061a0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 -- 172.21.15.93:0/201471668 shutdown_connections 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b0569190a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b05691df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 --2- 172.21.15.93:0/201471668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b056918b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.732+0000 7f495e12eb80 1 -- 172.21.15.93:0/201471668 >> 172.21.15.93:0/201471668 conn(0x55b056913ae0 msgr2=0x55b05693e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.734+0000 7f495e12eb80 1 -- 172.21.15.93:0/201471668 shutdown_connections 2023-07-21T22:34:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.734+0000 7f495e12eb80 1 -- 172.21.15.93:0/201471668 wait complete. 2023-07-21T22:34:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 -- start start 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 Processor -- start 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b056934080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b0569206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b056921340 con 0x55b056924660 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0569214b0 con 0x55b056924210 2023-07-21T22:34:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05698a4a0 con 0x55b05677a6b0 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f495e12eb80 10 client.0 _ll_register_callbacks cb 0x7f493c095f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f494bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49106/0 (socket says 172.21.15.93:49106) 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f494bfff700 1 -- 172.21.15.93:0/2257451245 learned_addr learned my addr 172.21.15.93:0/2257451245 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f494affd700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b0569206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.735+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b056934080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 -- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 msgr2=0x55b056934080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b056934080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 -- 172.21.15.93:0/2257451245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 msgr2=0x55b0569206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b0569206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 -- 172.21.15.93:0/2257451245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b056686c00 con 0x55b056924660 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f494bfff700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f494400b520 tx=0x7f494400bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a4dfc2de2ac87b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f492affd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440075a0 con 0x55b056924660 2023-07-21T22:34:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4944017390 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f494400be90 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4944017550 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f495e12eb80 10 client.17562 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55b0566c41f0 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.736+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f494400be90 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f494400b200 con 0x55b056924660 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 10 client.17562 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 20 client.17562 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05698b660 0x55b05698da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 4 client.17562 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b05682aff0 con 0x55b05698b660 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05698b660 0x55b05698da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f494b7fe700 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05698b660 0x55b05698da70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea71c52bfccc4556 server_cookie=5549df9374c47938 in_seq=0 out_seq=0 2023-07-21T22:34:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.737+0000 7f492affd700 10 client.17562 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f492affd700 1 -- 172.21.15.93:0/2257451245 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b05682aff0 con 0x55b05698b660 2023-07-21T22:34:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f492affd700 10 client.17562 handle_command_reply: tid=0 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 client.17562 shutdown 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 msgr2=0x55b056920c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f494400b520 tx=0x7f494400bbf0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 shutdown_connections 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b05698b660 0x55b05698da70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b056924660 0x55b056920c30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b05677a6b0 0x55b056934080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 --2- 172.21.15.93:0/2257451245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b056924210 0x55b0569206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 >> 172.21.15.93:0/2257451245 conn(0x55b056913ae0 msgr2=0x55b05692d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f492affd700 0 client.17562 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 shutdown_connections 2023-07-21T22:34:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.995+0000 7f495e12eb80 1 -- 172.21.15.93:0/2257451245 wait complete. 2023-07-21T22:34:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f495e12eb80 20 client.17562 trim_cache size 0 max 16384 2023-07-21T22:34:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:34:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:34:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:34:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 msgr2=0x7f49340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 0x7f49340a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f494c194a30 tx=0x7f4940009260 comp rx=0 tx=0).stop 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 msgr2=0x7f494c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f493c00eb30 tx=0x7f493c00ee40 comp rx=0 tx=0).stop 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 shutdown_connections 2023-07-21T22:34:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f494c104db0 0x7f494c193820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f49340a12c0 0x7f49340a3770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494c0fe260 0x7f494c199950 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 --2- 172.21.15.93:0/4046443327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f494c0fec00 0x7f494c199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.996+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 >> 172.21.15.93:0/4046443327 conn(0x7f494c075f90 msgr2=0x7f494c0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.997+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 shutdown_connections 2023-07-21T22:34:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:25.997+0000 7f4951fa9700 1 -- 172.21.15.93:0/4046443327 wait complete. 2023-07-21T22:34:26.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 193667 34.18 MB/sec execute 182 sec latency 181.665 ms 2023-07-21T22:34:27.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 194374 34.25 MB/sec execute 183 sec latency 58.849 ms 2023-07-21T22:34:27.632 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:27 smithi134 ceph-mon[97873]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.1 MiB/s wr, 502 op/s 2023-07-21T22:34:27.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:27 smithi093 ceph-mon[96579]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.1 MiB/s wr, 502 op/s 2023-07-21T22:34:27.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:27 smithi194 ceph-mon[98288]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.1 MiB/s wr, 502 op/s 2023-07-21T22:34:28.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 194996 34.25 MB/sec execute 184 sec latency 117.216 ms 2023-07-21T22:34:29.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 195395 34.16 MB/sec execute 185 sec latency 422.204 ms 2023-07-21T22:34:29.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:29 smithi093 ceph-mon[96579]: pgmap v574: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 821 op/s 2023-07-21T22:34:29.845 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:29 smithi194 ceph-mon[98288]: pgmap v574: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 821 op/s 2023-07-21T22:34:29.890 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:29 smithi134 ceph-mon[97873]: pgmap v574: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 821 op/s 2023-07-21T22:34:30.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 195421 33.98 MB/sec execute 186 sec latency 1125.590 ms 2023-07-21T22:34:31.526 INFO:tasks.workunit.client.0.smithi093.stdout: 10 195789 33.89 MB/sec execute 187 sec latency 656.854 ms 2023-07-21T22:34:31.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:31 smithi134 ceph-mon[97873]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 579 op/s 2023-07-21T22:34:31.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:31 smithi093 ceph-mon[96579]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 579 op/s 2023-07-21T22:34:31.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:31 smithi194 ceph-mon[98288]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 10 MiB/s wr, 579 op/s 2023-07-21T22:34:32.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 196239 33.84 MB/sec execute 188 sec latency 111.509 ms 2023-07-21T22:34:33.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 196939 33.86 MB/sec execute 189 sec latency 70.946 ms 2023-07-21T22:34:33.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:33 smithi093 ceph-mon[96579]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-07-21T22:34:33.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:33 smithi194 ceph-mon[98288]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-07-21T22:34:33.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:33 smithi134 ceph-mon[97873]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 716 op/s 2023-07-21T22:34:34.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 197613 33.90 MB/sec execute 190 sec latency 140.823 ms 2023-07-21T22:34:35.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 198230 33.92 MB/sec execute 191 sec latency 81.643 ms 2023-07-21T22:34:35.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:35 smithi093 ceph-mon[96579]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-07-21T22:34:35.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:35 smithi194 ceph-mon[98288]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-07-21T22:34:35.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:35 smithi134 ceph-mon[97873]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.11k op/s 2023-07-21T22:34:36.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 198847 33.88 MB/sec execute 192 sec latency 49.996 ms 2023-07-21T22:34:37.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 199501 33.90 MB/sec execute 193 sec latency 90.714 ms 2023-07-21T22:34:37.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:37 smithi093 ceph-mon[96579]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 21 MiB/s wr, 914 op/s 2023-07-21T22:34:37.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:37 smithi194 ceph-mon[98288]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 21 MiB/s wr, 914 op/s 2023-07-21T22:34:37.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:37 smithi134 ceph-mon[97873]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 21 MiB/s wr, 914 op/s 2023-07-21T22:34:38.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 200152 33.92 MB/sec execute 194 sec latency 66.976 ms 2023-07-21T22:34:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.147+0000 7f9b31299700 1 -- 172.21.15.93:0/4258617089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c0fe550 msgr2=0x7f9b2c0fe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.147+0000 7f9b31299700 1 --2- 172.21.15.93:0/4258617089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c0fe920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0099c0 tx=0x7f9b1c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 -- 172.21.15.93:0/4258617089 shutdown_connections 2023-07-21T22:34:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 --2- 172.21.15.93:0/4258617089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c104e20 0x7f9b2c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 --2- 172.21.15.93:0/4258617089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c0fe920 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 --2- 172.21.15.93:0/4258617089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 -- 172.21.15.93:0/4258617089 >> 172.21.15.93:0/4258617089 conn(0x7f9b2c075060 msgr2=0x7f9b2c075460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 -- 172.21.15.93:0/4258617089 shutdown_connections 2023-07-21T22:34:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 -- 172.21.15.93:0/4258617089 wait complete. 2023-07-21T22:34:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.148+0000 7f9b31299700 1 Processor -- start 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 -- start start 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1960c0 con 0x7f9b2c104e20 2023-07-21T22:34:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c196230 con 0x7f9b2c0feef0 2023-07-21T22:34:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b31299700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1963f0 con 0x7f9b2c0fe550 2023-07-21T22:34:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49120/0 (socket says 172.21.15.93:49120) 2023-07-21T22:34:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36196/0 (socket says 172.21.15.93:36196) 2023-07-21T22:34:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.149+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3375534254 learned_addr learned my addr 172.21.15.93:0/3375534254 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3375534254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 msgr2=0x7f9b2c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3375534254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 msgr2=0x7f9b2c19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3375534254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0095f0 con 0x7f9b2c104e20 2023-07-21T22:34:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400ef30 tx=0x7f9b2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b13fff700 1 -- 172.21.15.93:0/3375534254 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24018070 con 0x7f9b2c104e20 2023-07-21T22:34:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c196670 con 0x7f9b2c104e20 2023-07-21T22:34:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b13fff700 1 -- 172.21.15.93:0/3375534254 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2400f040 con 0x7f9b2c104e20 2023-07-21T22:34:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b13fff700 1 -- 172.21.15.93:0/3375534254 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cd00 con 0x7f9b2c104e20 2023-07-21T22:34:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.150+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c1a2be0 con 0x7f9b2c104e20 2023-07-21T22:34:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.153+0000 7f9b13fff700 1 -- 172.21.15.93:0/3375534254 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9b24003a00 con 0x7f9b2c104e20 2023-07-21T22:34:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.154+0000 7f9b13fff700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 0x7f9b140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.154+0000 7f9b13fff700 1 -- 172.21.15.93:0/3375534254 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b240cc7d0 con 0x7f9b2c104e20 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.154+0000 7f9b2affd700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 0x7f9b140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.154+0000 7f9b2affd700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 0x7f9b140a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009990 tx=0x7f9b1c005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.155+0000 7f9b3d41eb80 1 Processor -- start 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 -- start start 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c57af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581660 0x560e2c57b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c480c50 con 0x560e2c3d76b0 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c4d39e0 con 0x560e2c581210 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c4cce70 con 0x560e2c581660 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49124/0 (socket says 172.21.15.93:49124) 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 -- 172.21.15.93:0/752192017 learned_addr learned my addr 172.21.15.93:0/752192017 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c57af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581660 0x560e2c57b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 -- 172.21.15.93:0/752192017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581660 msgr2=0x560e2c57b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581660 0x560e2c57b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 -- 172.21.15.93:0/752192017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 msgr2=0x560e2c57af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c57af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.156+0000 7f9b2affd700 1 -- 172.21.15.93:0/752192017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2c2fad80 con 0x560e2c3d76b0 2023-07-21T22:34:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.157+0000 7f9b2affd700 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0099c0 tx=0x7f9b1c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e96c5e89edce863 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.157+0000 7f9b117fa700 1 -- 172.21.15.93:0/752192017 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00b990 con 0x560e2c3d76b0 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.157+0000 7f9b117fa700 1 -- 172.21.15.93:0/752192017 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1c01e040 con 0x560e2c3d76b0 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.157+0000 7f9b117fa700 1 -- 172.21.15.93:0/752192017 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c004800 con 0x560e2c3d76b0 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/752192017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 msgr2=0x560e2c580cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0099c0 tx=0x7f9b1c01f000 comp rx=0 tx=0).stop 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/752192017 shutdown_connections 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581660 0x560e2c57b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c3d76b0 0x560e2c580cd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/752192017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c57af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/752192017 >> 172.21.15.93:0/752192017 conn(0x560e2c570ae0 msgr2=0x560e2c5760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/752192017 shutdown_connections 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/752192017 wait complete. 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.158+0000 7f9b3d41eb80 1 -- start start 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 Processor -- start 2023-07-21T22:34:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c58a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 0x560e2c58aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 0x560e2c584820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c46ca60 con 0x560e2c581210 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c46c760 con 0x560e2c581660 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c3508c0 con 0x560e2c3d76b0 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b3d41eb80 10 client.0 _ll_register_callbacks cb 0x560e2c46a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 0x560e2c58aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 0x560e2c58aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49132/0 (socket says 172.21.15.93:49132) 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b2a7fc700 1 -- 172.21.15.93:0/1818073113 learned_addr learned my addr 172.21.15.93:0/1818073113 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b2affd700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c58a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.159+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 0x560e2c584820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 msgr2=0x560e2c58a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c58a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/1818073113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 msgr2=0x560e2c58aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 0x560e2c58aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/1818073113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c10d270 con 0x560e2c581660 2023-07-21T22:34:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.160+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 0x560e2c584820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b240c9f20 tx=0x7f9b24099290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc7e31090fd2a51 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.161+0000 7f9b0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.161+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24084040 con 0x560e2c581660 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.161+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b24099930 con 0x560e2c581660 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.161+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e2c426490 con 0x560e2c581660 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b2409d960 con 0x560e2c581660 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 10 client.26948 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x560e2c2e3c00 con 0x560e2c581660 2023-07-21T22:34:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e2c426490 con 0x560e2c581660 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9b2409dbc0 con 0x560e2c581660 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 10 client.26948 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 20 client.26948 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5a5910 0x560e2c5a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 4 client.26948 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.162+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e2c4c5e50 con 0x560e2c5a5910 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.163+0000 7f9b2affd700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5a5910 0x560e2c5a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.163+0000 7f9b0eff5700 10 client.26948 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.163+0000 7f9b2affd700 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5a5910 0x560e2c5a7d20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed7d1823a996d0c server_cookie=52274230d6399008 in_seq=0 out_seq=0 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.165+0000 7f9b0eff5700 1 -- 172.21.15.93:0/1818073113 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560e2c4c5e50 con 0x560e2c5a5910 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.165+0000 7f9b0eff5700 10 client.26948 handle_command_reply: tid=0 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.165+0000 7f9b3d41eb80 1 client.26948 shutdown 2023-07-21T22:34:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 msgr2=0x560e2c584820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 0x560e2c584820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b240c9f20 tx=0x7f9b24099290 comp rx=0 tx=0).stop 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 shutdown_connections 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5a5910 0x560e2c5a7d20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581660 0x560e2c584820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c58a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/1818073113 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581210 0x560e2c58aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 >> 172.21.15.93:0/1818073113 conn(0x560e2c570ae0 msgr2=0x560e2c5753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.166+0000 7f9b0eff5700 0 client.26948 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.167+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 shutdown_connections 2023-07-21T22:34:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.168+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/1818073113 wait complete. 2023-07-21T22:34:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.169+0000 7f9b3d41eb80 20 client.26948 trim_cache size 0 max 16384 2023-07-21T22:34:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 Processor -- start 2023-07-21T22:34:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 -- start start 2023-07-21T22:34:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c5801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c583720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c57c160 con 0x560e2c581660 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c572c00 con 0x560e2c581210 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c585d90 con 0x560e2c3d76b0 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49140/0 (socket says 172.21.15.93:49140) 2023-07-21T22:34:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3364863645 learned_addr learned my addr 172.21.15.93:0/3364863645 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c583720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2affd700 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c5801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3364863645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 msgr2=0x560e2c5801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.193+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c5801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3364863645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 msgr2=0x560e2c583720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c583720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/3364863645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e2c584390 con 0x560e2c581660 2023-07-21T22:34:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9b240998c0 tx=0x7f9b240092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745c6ea7ef59b42a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b11ffb700 1 -- 172.21.15.93:0/3364863645 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24084040 con 0x560e2c581660 2023-07-21T22:34:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b11ffb700 1 -- 172.21.15.93:0/3364863645 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b240a1a40 con 0x560e2c581660 2023-07-21T22:34:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.194+0000 7f9b11ffb700 1 -- 172.21.15.93:0/3364863645 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2409d960 con 0x560e2c581660 2023-07-21T22:34:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/3364863645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 msgr2=0x560e2c575b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9b240998c0 tx=0x7f9b240092e0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/3364863645 shutdown_connections 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c575b60 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c3d76b0 0x560e2c5801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/3364863645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c581210 0x560e2c583720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/3364863645 >> 172.21.15.93:0/3364863645 conn(0x560e2c570ae0 msgr2=0x560e2c583010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/3364863645 shutdown_connections 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/3364863645 wait complete. 2023-07-21T22:34:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.195+0000 7f9b3d41eb80 1 -- start start 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 Processor -- start 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 0x560e2c588710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 0x560e2c588c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c5f8d10 con 0x560e2c581660 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c5f8e50 con 0x560e2c3d76b0 2023-07-21T22:34:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e2c5f8f90 con 0x560e2c581210 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b3d41eb80 10 client.0 _ll_register_callbacks cb 0x7f9b240997f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b2affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 0x560e2c588710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49146/0 (socket says 172.21.15.93:49146) 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.196+0000 7f9b2b7fe700 1 -- 172.21.15.93:0/2767030691 learned_addr learned my addr 172.21.15.93:0/2767030691 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 -- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 msgr2=0x560e2c588c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 0x560e2c588c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 0x560e2c588c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 -- 172.21.15.93:0/2767030691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 msgr2=0x560e2c5f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 -- 172.21.15.93:0/2767030691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e2c2e3c00 con 0x560e2c3d76b0 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2b7fe700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 0x560e2c588c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b2affd700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 0x560e2c588710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0058e0 tx=0x7f9b1c003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcddd2429ac0cbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c036040 con 0x560e2c3d76b0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b1c03da30 con 0x560e2c3d76b0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b1c01e040 con 0x560e2c3d76b0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b1c03dbf0 con 0x560e2c3d76b0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b3d41eb80 10 client.26954 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.197+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x560e2c3211f0 con 0x560e2c3d76b0 2023-07-21T22:34:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1c01e040 con 0x560e2c3d76b0 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9b1c00b820 con 0x560e2c3d76b0 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 10 client.26954 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 20 client.26954 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5fa1d0 0x560e2c5fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 4 client.26954 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e2c487ff0 con 0x560e2c5fa1d0 2023-07-21T22:34:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.198+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5fa1d0 0x560e2c5fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.199+0000 7f9b0eff5700 10 client.26954 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.199+0000 7f9b2a7fc700 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5fa1d0 0x560e2c5fc5e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4196f1f4b373580d server_cookie=7449b5eed569a39d in_seq=0 out_seq=0 2023-07-21T22:34:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b0eff5700 1 -- 172.21.15.93:0/2767030691 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e2c487ff0 con 0x560e2c5fa1d0 2023-07-21T22:34:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b0eff5700 10 client.26954 handle_command_reply: tid=0 2023-07-21T22:34:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 client.26954 shutdown 2023-07-21T22:34:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 msgr2=0x560e2c588710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 0x560e2c588710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0058e0 tx=0x7f9b1c003e40 comp rx=0 tx=0).stop 2023-07-21T22:34:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 shutdown_connections 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560e2c5fa1d0 0x560e2c5fc5e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560e2c581660 0x560e2c5f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e2c3d76b0 0x560e2c588710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b0eff5700 0 client.26954 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 --2- 172.21.15.93:0/2767030691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e2c581210 0x560e2c588c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 >> 172.21.15.93:0/2767030691 conn(0x560e2c570ae0 msgr2=0x560e2c5838a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 shutdown_connections 2023-07-21T22:34:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 1 -- 172.21.15.93:0/2767030691 wait complete. 2023-07-21T22:34:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.383+0000 7f9b3d41eb80 20 client.26954 trim_cache size 0 max 16384 2023-07-21T22:34:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:34:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:34:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:34:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:34:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 msgr2=0x7f9b140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 0x7f9b140a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009990 tx=0x7f9b1c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 msgr2=0x7f9b2c195a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400ef30 tx=0x7f9b2400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 shutdown_connections 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f9b140a12c0 0x7f9b140a3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b2c104e20 0x7f9b2c195a50 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b2c0fe550 0x7f9b2c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 --2- 172.21.15.93:0/3375534254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b2c0feef0 0x7f9b2c19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 >> 172.21.15.93:0/3375534254 conn(0x7f9b2c075060 msgr2=0x7f9b2c0fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 shutdown_connections 2023-07-21T22:34:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:39.384+0000 7f9b31299700 1 -- 172.21.15.93:0/3375534254 wait complete. 2023-07-21T22:34:39.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 200613 33.82 MB/sec execute 195 sec latency 164.485 ms 2023-07-21T22:34:39.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:39 smithi093 ceph-mon[96579]: pgmap v579: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.16k op/s 2023-07-21T22:34:39.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:39 smithi194 ceph-mon[98288]: pgmap v579: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.16k op/s 2023-07-21T22:34:39.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:39 smithi134 ceph-mon[97873]: pgmap v579: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.16k op/s 2023-07-21T22:34:40.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 201288 33.86 MB/sec execute 196 sec latency 93.126 ms 2023-07-21T22:34:41.523 INFO:tasks.workunit.client.0.smithi093.stdout: 10 202101 33.93 MB/sec execute 197 sec latency 49.182 ms 2023-07-21T22:34:41.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:41 smithi093 ceph-mon[96579]: pgmap v580: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 850 op/s 2023-07-21T22:34:41.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:41 smithi194 ceph-mon[98288]: pgmap v580: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 850 op/s 2023-07-21T22:34:41.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:41 smithi134 ceph-mon[97873]: pgmap v580: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 850 op/s 2023-07-21T22:34:42.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 202653 33.94 MB/sec execute 198 sec latency 122.977 ms 2023-07-21T22:34:43.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 203033 33.84 MB/sec execute 199 sec latency 174.849 ms 2023-07-21T22:34:43.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:43 smithi093 ceph-mon[96579]: pgmap v581: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 882 op/s 2023-07-21T22:34:43.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:43 smithi194 ceph-mon[98288]: pgmap v581: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 882 op/s 2023-07-21T22:34:43.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:43 smithi134 ceph-mon[97873]: pgmap v581: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 20 MiB/s wr, 882 op/s 2023-07-21T22:34:44.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 203641 33.82 MB/sec execute 200 sec latency 63.926 ms 2023-07-21T22:34:45.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 204330 33.83 MB/sec execute 201 sec latency 76.812 ms 2023-07-21T22:34:45.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:45 smithi093 ceph-mon[96579]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 943 op/s 2023-07-21T22:34:45.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:45 smithi194 ceph-mon[98288]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 943 op/s 2023-07-21T22:34:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:45 smithi134 ceph-mon[97873]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 20 MiB/s wr, 943 op/s 2023-07-21T22:34:46.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 205118 33.83 MB/sec execute 202 sec latency 126.660 ms 2023-07-21T22:34:47.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 205848 33.95 MB/sec execute 203 sec latency 53.638 ms 2023-07-21T22:34:47.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:47 smithi093 ceph-mon[96579]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 514 op/s 2023-07-21T22:34:47.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:47 smithi194 ceph-mon[98288]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 514 op/s 2023-07-21T22:34:47.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:47 smithi134 ceph-mon[97873]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 514 op/s 2023-07-21T22:34:48.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 206316 33.91 MB/sec execute 204 sec latency 422.496 ms 2023-07-21T22:34:49.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 206893 33.86 MB/sec execute 205 sec latency 56.884 ms 2023-07-21T22:34:49.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:49 smithi093 ceph-mon[96579]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 714 op/s 2023-07-21T22:34:49.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:49 smithi194 ceph-mon[98288]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 714 op/s 2023-07-21T22:34:49.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:49 smithi134 ceph-mon[97873]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 714 op/s 2023-07-21T22:34:50.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 207582 33.86 MB/sec execute 206 sec latency 87.742 ms 2023-07-21T22:34:51.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 208317 33.88 MB/sec execute 207 sec latency 75.927 ms 2023-07-21T22:34:51.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:51 smithi093 ceph-mon[96579]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 480 op/s 2023-07-21T22:34:51.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:51 smithi194 ceph-mon[98288]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 480 op/s 2023-07-21T22:34:51.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:51 smithi134 ceph-mon[97873]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 480 op/s 2023-07-21T22:34:52.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 209022 33.92 MB/sec execute 208 sec latency 69.494 ms 2023-07-21T22:34:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 -- 172.21.15.93:0/1558891846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5800ff3e0 msgr2=0x7fd58010b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 --2- 172.21.15.93:0/1558891846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5800ff3e0 0x7fd58010b860 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b0a0 tx=0x7fd57800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 -- 172.21.15.93:0/1558891846 shutdown_connections 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 --2- 172.21.15.93:0/1558891846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5800ff3e0 0x7fd58010b860 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 --2- 172.21.15.93:0/1558891846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd580104a30 0x7fd580104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 --2- 172.21.15.93:0/1558891846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd5800feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 -- 172.21.15.93:0/1558891846 >> 172.21.15.93:0/1558891846 conn(0x7fd580074760 msgr2=0x7fd580074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 -- 172.21.15.93:0/1558891846 shutdown_connections 2023-07-21T22:34:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.527+0000 7fd587192700 1 -- 172.21.15.93:0/1558891846 wait complete. 2023-07-21T22:34:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 Processor -- start 2023-07-21T22:34:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 -- start start 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd580197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5800ff3e0 0x7fd580198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 0x7fd58019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019d140 con 0x7fd580104a30 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019d2b0 con 0x7fd5800fea30 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.528+0000 7fd587192700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58019d470 con 0x7fd5800ff3e0 2023-07-21T22:34:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd580197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd580197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60320/0 (socket says 172.21.15.93:60320) 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 0x7fd58019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd57ffff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5800ff3e0 0x7fd580198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd584f2e700 1 -- 172.21.15.93:0/590134670 learned_addr learned my addr 172.21.15.93:0/590134670 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 -- 172.21.15.93:0/590134670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5800ff3e0 msgr2=0x7fd580198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5800ff3e0 0x7fd580198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 -- 172.21.15.93:0/590134670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 msgr2=0x7fd580197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd580197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 -- 172.21.15.93:0/590134670 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009d00 con 0x7fd580104a30 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd58572f700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 0x7fd58019cb00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b0a0 tx=0x7fd5780079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd57dffb700 1 -- 172.21.15.93:0/590134670 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800e040 con 0x7fd580104a30 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58019d6f0 con 0x7fd580104a30 2023-07-21T22:34:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd57dffb700 1 -- 172.21.15.93:0/590134670 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd578006080 con 0x7fd580104a30 2023-07-21T22:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.529+0000 7fd57dffb700 1 -- 172.21.15.93:0/590134670 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57801b850 con 0x7fd580104a30 2023-07-21T22:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.530+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5801a4f90 con 0x7fd580104a30 2023-07-21T22:34:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.533+0000 7fd57dffb700 1 -- 172.21.15.93:0/590134670 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd57801d480 con 0x7fd580104a30 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.533+0000 7fd57dffb700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 0x7fd5680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.533+0000 7fd57dffb700 1 -- 172.21.15.93:0/590134670 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd5780ce8b0 con 0x7fd580104a30 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.534+0000 7fd593317b80 1 Processor -- start 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.534+0000 7fd593317b80 1 -- start start 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b91d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b814c50 con 0x56352b76b6b0 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b8679e0 con 0x56352b91da20 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd584f2e700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 0x7fd5680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd593317b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b860e70 con 0x56352b91de70 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36246/0 (socket says 172.21.15.93:36246) 2023-07-21T22:34:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 -- 172.21.15.93:0/3044572727 learned_addr learned my addr 172.21.15.93:0/3044572727 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b91d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 -- 172.21.15.93:0/3044572727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 msgr2=0x56352b917750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:34:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 -- 172.21.15.93:0/3044572727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 msgr2=0x56352b91d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b91d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 -- 172.21.15.93:0/3044572727 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b68ed80 con 0x56352b91de70 2023-07-21T22:34:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.535+0000 7fd58572f700 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd578097f10 tx=0x7fd578007ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d46ebd4216181f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.536+0000 7fd566ffd700 1 -- 172.21.15.93:0/3044572727 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57801d730 con 0x56352b91de70 2023-07-21T22:34:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.536+0000 7fd566ffd700 1 -- 172.21.15.93:0/3044572727 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57809ba10 con 0x56352b91de70 2023-07-21T22:34:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.536+0000 7fd566ffd700 1 -- 172.21.15.93:0/3044572727 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57809bed0 con 0x56352b91de70 2023-07-21T22:34:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- 172.21.15.93:0/3044572727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 msgr2=0x56352b917cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd578097f10 tx=0x7fd578007ae0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd584f2e700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 0x7fd5680a3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd574015730 tx=0x7fd574009a20 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- 172.21.15.93:0/3044572727 shutdown_connections 2023-07-21T22:34:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91de70 0x56352b917cb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b91d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 --2- 172.21.15.93:0/3044572727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- 172.21.15.93:0/3044572727 >> 172.21.15.93:0/3044572727 conn(0x56352b904ae0 msgr2=0x56352b90a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- 172.21.15.93:0/3044572727 shutdown_connections 2023-07-21T22:34:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- 172.21.15.93:0/3044572727 wait complete. 2023-07-21T22:34:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.537+0000 7fd593317b80 1 -- start start 2023-07-21T22:34:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 Processor -- start 2023-07-21T22:34:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b920ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b9213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b800a60 con 0x56352b91de70 2023-07-21T22:34:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b800760 con 0x56352b91da20 2023-07-21T22:34:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b6e48c0 con 0x56352b76b6b0 2023-07-21T22:34:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd593317b80 10 client.0 _ll_register_callbacks cb 0x56352b7fe700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36250/0 (socket says 172.21.15.93:36250) 2023-07-21T22:34:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.538+0000 7fd584f2e700 1 -- 172.21.15.93:0/3859917197 learned_addr learned my addr 172.21.15.93:0/3859917197 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd58572f700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b9213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd57ffff700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b920ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 -- 172.21.15.93:0/3859917197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 msgr2=0x56352b920ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b920ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 -- 172.21.15.93:0/3859917197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 msgr2=0x56352b9213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b9213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 -- 172.21.15.93:0/3859917197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5801a4f90 con 0x56352b76b6b0 2023-07-21T22:34:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd58572f700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b9213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd574017e00 tx=0x7fd574004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122c2aefdabd8dab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574006ca0 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd574018e10 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.539+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56352b7ba490 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd574024470 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 10 client.26857 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56352b677c00 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56352b7ba490 con 0x56352b76b6b0 2023-07-21T22:34:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd574022500 con 0x56352b76b6b0 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 10 client.26857 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 10 client.26857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 20 client.26857 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b9421b0 0x56352b9445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 4 client.26857 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.540+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56352b859e50 con 0x56352b9421b0 2023-07-21T22:34:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.541+0000 7fd57ffff700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b9421b0 0x56352b9445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.541+0000 7fd55ffff700 10 client.26857 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.541+0000 7fd57ffff700 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b9421b0 0x56352b9445c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5324b82fd522b8 server_cookie=d03a228ef080906a in_seq=0 out_seq=0 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.543+0000 7fd55ffff700 1 -- 172.21.15.93:0/3859917197 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56352b859e50 con 0x56352b9421b0 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.543+0000 7fd55ffff700 10 client.26857 handle_command_reply: tid=0 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 client.26857 shutdown 2023-07-21T22:34:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 msgr2=0x56352b923c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd574017e00 tx=0x7fd574004750 comp rx=0 tx=0).stop 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 shutdown_connections 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b9421b0 0x56352b9445c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b9213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd55ffff700 0 client.26857 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b923c20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 --2- 172.21.15.93:0/3859917197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b920ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 >> 172.21.15.93:0/3859917197 conn(0x56352b904ae0 msgr2=0x56352b9093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 shutdown_connections 2023-07-21T22:34:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 1 -- 172.21.15.93:0/3859917197 wait complete. 2023-07-21T22:34:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.544+0000 7fd593317b80 20 client.26857 trim_cache size 0 max 16384 2023-07-21T22:34:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 Processor -- start 2023-07-21T22:34:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 -- start start 2023-07-21T22:34:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b91e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b91ef00 con 0x56352b91de70 2023-07-21T22:34:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b922480 con 0x56352b91da20 2023-07-21T22:34:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd593317b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b93eae0 con 0x56352b76b6b0 2023-07-21T22:34:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd58572f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd58572f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49180/0 (socket says 172.21.15.93:49180) 2023-07-21T22:34:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.565+0000 7fd58572f700 1 -- 172.21.15.93:0/486775346 learned_addr learned my addr 172.21.15.93:0/486775346 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 -- 172.21.15.93:0/486775346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 msgr2=0x56352b91e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b91e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 -- 172.21.15.93:0/486775346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 msgr2=0x56352b917750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 -- 172.21.15.93:0/486775346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56352b6b51f0 con 0x56352b91de70 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.566+0000 7fd58572f700 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd578026000 tx=0x7fd57800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28acca2c2d692bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.567+0000 7fd5677fe700 1 -- 172.21.15.93:0/486775346 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780b41a0 con 0x56352b91de70 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.567+0000 7fd5677fe700 1 -- 172.21.15.93:0/486775346 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57809fa40 con 0x56352b91de70 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.567+0000 7fd5677fe700 1 -- 172.21.15.93:0/486775346 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57809b160 con 0x56352b91de70 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- 172.21.15.93:0/486775346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 msgr2=0x56352b917c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd578026000 tx=0x7fd57800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- 172.21.15.93:0/486775346 shutdown_connections 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b91de70 0x56352b917c90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b76b6b0 0x56352b91e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 --2- 172.21.15.93:0/486775346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91da20 0x56352b917750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- 172.21.15.93:0/486775346 >> 172.21.15.93:0/486775346 conn(0x56352b904ae0 msgr2=0x56352b91e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- 172.21.15.93:0/486775346 shutdown_connections 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- 172.21.15.93:0/486775346 wait complete. 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 -- start start 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.568+0000 7fd593317b80 1 Processor -- start 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91da20 0x56352b938b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91de70 0x56352b995030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b939420 con 0x56352b76b6b0 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b995570 con 0x56352b91de70 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56352b9956b0 con 0x56352b91da20 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd593317b80 10 client.0 _ll_register_callbacks cb 0x7fd5740177d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd584f2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49184/0 (socket says 172.21.15.93:49184) 2023-07-21T22:34:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.569+0000 7fd584f2e700 1 -- 172.21.15.93:0/3376819830 learned_addr learned my addr 172.21.15.93:0/3376819830 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:34:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 -- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91da20 msgr2=0x56352b938b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:34:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91da20 0x56352b938b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 -- 172.21.15.93:0/3376819830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91de70 msgr2=0x56352b995030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91de70 0x56352b995030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 -- 172.21.15.93:0/3376819830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56352b68ed80 con 0x56352b76b6b0 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd584f2e700 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd5740092e0 tx=0x7fd574009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2affb6314860e47 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd55ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574022420 con 0x56352b76b6b0 2023-07-21T22:34:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd574022580 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.570+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd57809ba10 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.572+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd574022c80 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.572+0000 7fd593317b80 10 client.17598 fetch_fsmap learned FSMap version 80 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.572+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x56352b68d250 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.572+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57809ba10 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd574024a90 con 0x56352b76b6b0 2023-07-21T22:34:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 10 client.17598 fetch_fsmap finished waiting for FSMap version 80 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 20 client.17598 populate_metadata read hostname 'smithi093' 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b996d10 0x56352b999120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 4 client.17598 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56352b81bff0 con 0x56352b996d10 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd57ffff700 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b996d10 0x56352b999120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.573+0000 7fd55ffff700 10 client.17598 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.574+0000 7fd57ffff700 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b996d10 0x56352b999120 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebb3388ed7dcad7 server_cookie=9eeb091f359e3029 in_seq=0 out_seq=0 2023-07-21T22:34:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.874+0000 7fd55ffff700 1 -- 172.21.15.93:0/3376819830 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56352b81bff0 con 0x56352b996d10 2023-07-21T22:34:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.874+0000 7fd55ffff700 10 client.17598 handle_command_reply: tid=0 2023-07-21T22:34:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.874+0000 7fd593317b80 1 client.17598 shutdown 2023-07-21T22:34:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 msgr2=0x56352b9385f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd5740092e0 tx=0x7fd574009560 comp rx=0 tx=0).stop 2023-07-21T22:34:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 shutdown_connections 2023-07-21T22:34:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56352b996d10 0x56352b999120 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56352b91de70 0x56352b995030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56352b76b6b0 0x56352b9385f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 --2- 172.21.15.93:0/3376819830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56352b91da20 0x56352b938b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 >> 172.21.15.93:0/3376819830 conn(0x56352b904ae0 msgr2=0x56352b91bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd55ffff700 0 client.17598 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:34:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 shutdown_connections 2023-07-21T22:34:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 1 -- 172.21.15.93:0/3376819830 wait complete. 2023-07-21T22:34:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.875+0000 7fd593317b80 20 client.17598 trim_cache size 0 max 16384 2023-07-21T22:34:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:34:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:34:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:34:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:34:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 msgr2=0x7fd5680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 0x7fd5680a3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd574015730 tx=0x7fd574009a20 comp rx=0 tx=0).stop 2023-07-21T22:34:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 msgr2=0x7fd58019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 0x7fd58019cb00 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b0a0 tx=0x7fd5780079b0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 shutdown_connections 2023-07-21T22:34:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd5680a12c0 0x7fd5680a3770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd580104a30 0x7fd58019cb00 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5800fea30 0x7fd580197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 --2- 172.21.15.93:0/590134670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5800ff3e0 0x7fd580198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.876+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 >> 172.21.15.93:0/590134670 conn(0x7fd580074760 msgr2=0x7fd5801058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.877+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 shutdown_connections 2023-07-21T22:34:52.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:34:52.877+0000 7fd587192700 1 -- 172.21.15.93:0/590134670 wait complete. 2023-07-21T22:34:53.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 209476 33.89 MB/sec execute 209 sec latency 299.015 ms 2023-07-21T22:34:53.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:53 smithi093 ceph-mon[96579]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 516 op/s 2023-07-21T22:34:53.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:53 smithi194 ceph-mon[98288]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 516 op/s 2023-07-21T22:34:53.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:53 smithi134 ceph-mon[97873]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 516 op/s 2023-07-21T22:34:54.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 210196 33.90 MB/sec execute 210 sec latency 70.128 ms 2023-07-21T22:34:55.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 210976 33.94 MB/sec execute 211 sec latency 47.621 ms 2023-07-21T22:34:55.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:55 smithi093 ceph-mon[96579]: pgmap v587: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 669 op/s 2023-07-21T22:34:55.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:55 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:34:55.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:55 smithi194 ceph-mon[98288]: pgmap v587: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 669 op/s 2023-07-21T22:34:55.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:55 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:34:55.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:55 smithi134 ceph-mon[97873]: pgmap v587: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 669 op/s 2023-07-21T22:34:55.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:55 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:34:56.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 211625 33.93 MB/sec execute 212 sec latency 69.198 ms 2023-07-21T22:34:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:56.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:34:56.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:56.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:56.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:34:56.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:56.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:56.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:34:57.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 212399 33.99 MB/sec execute 213 sec latency 68.002 ms 2023-07-21T22:34:57.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:57 smithi093 ceph-mon[96579]: pgmap v588: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 462 op/s 2023-07-21T22:34:57.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:57 smithi194 ceph-mon[98288]: pgmap v588: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 462 op/s 2023-07-21T22:34:57.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:57 smithi134 ceph-mon[97873]: pgmap v588: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.0 MiB/s wr, 462 op/s 2023-07-21T22:34:58.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 213057 33.97 MB/sec execute 214 sec latency 66.485 ms 2023-07-21T22:34:59.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 213739 33.99 MB/sec execute 215 sec latency 53.947 ms 2023-07-21T22:34:59.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:34:59 smithi093 ceph-mon[96579]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 708 op/s 2023-07-21T22:34:59.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:34:59 smithi194 ceph-mon[98288]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 708 op/s 2023-07-21T22:34:59.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:34:59 smithi134 ceph-mon[97873]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.3 MiB/s wr, 708 op/s 2023-07-21T22:35:00.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214398 34.02 MB/sec execute 216 sec latency 56.089 ms 2023-07-21T22:35:01.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214398 33.87 MB/sec execute 217 sec latency 1054.854 ms 2023-07-21T22:35:01.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:01 smithi093 ceph-mon[96579]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:01.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:01 smithi194 ceph-mon[98288]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:01.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:01 smithi134 ceph-mon[97873]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:02.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214398 33.71 MB/sec execute 218 sec latency 2054.972 ms 2023-07-21T22:35:03.524 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214398 33.56 MB/sec execute 219 sec latency 3055.063 ms 2023-07-21T22:35:03.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:03 smithi093 ceph-mon[96579]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:03.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:03 smithi194 ceph-mon[98288]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:03.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:03 smithi134 ceph-mon[97873]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 513 op/s 2023-07-21T22:35:04.525 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214487 33.41 MB/sec execute 220 sec latency 3800.087 ms 2023-07-21T22:35:05.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214493 33.27 MB/sec execute 221 sec latency 1124.365 ms 2023-07-21T22:35:05.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:05 smithi194 ceph-mon[98288]: pgmap v592: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-07-21T22:35:05.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:05 smithi134 ceph-mon[97873]: pgmap v592: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-07-21T22:35:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:05 smithi093 ceph-mon[96579]: pgmap v592: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.9 MiB/s wr, 590 op/s 2023-07-21T22:35:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 -- 172.21.15.93:0/2911506626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 msgr2=0x7f7dec06cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 --2- 172.21.15.93:0/2911506626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec06cd80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b0a0 tx=0x7f7de400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 -- 172.21.15.93:0/2911506626 shutdown_connections 2023-07-21T22:35:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 --2- 172.21.15.93:0/2911506626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dec06d3a0 0x7f7dec11b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 --2- 172.21.15.93:0/2911506626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec10e2a0 0x7f7dec10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 --2- 172.21.15.93:0/2911506626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec06cd80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 -- 172.21.15.93:0/2911506626 >> 172.21.15.93:0/2911506626 conn(0x7f7dec06bd70 msgr2=0x7f7dec070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.065+0000 7f7df275e700 1 -- 172.21.15.93:0/2911506626 shutdown_connections 2023-07-21T22:35:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.066+0000 7f7df275e700 1 -- 172.21.15.93:0/2911506626 wait complete. 2023-07-21T22:35:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.066+0000 7f7df275e700 1 Processor -- start 2023-07-21T22:35:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.066+0000 7f7df275e700 1 -- start start 2023-07-21T22:35:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec1aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dec10e2a0 0x7f7dec1a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1a5070 con 0x7f7dec06d3a0 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1a51e0 con 0x7f7dec06c910 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7df275e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec1a53a0 con 0x7f7dec10e2a0 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49192/0 (socket says 172.21.15.93:49192) 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 -- 172.21.15.93:0/358332407 learned_addr learned my addr 172.21.15.93:0/358332407 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7debfff700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec1aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 -- 172.21.15.93:0/358332407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dec10e2a0 msgr2=0x7f7dec1a4a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dec10e2a0 0x7f7dec1a4a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 -- 172.21.15.93:0/358332407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 msgr2=0x7f7dec1aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec1aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.067+0000 7f7deb7fe700 1 -- 172.21.15.93:0/358332407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4009d00 con 0x7f7dec06d3a0 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7debfff700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec1aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7deb7fe700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b070 tx=0x7f7de4005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7de97fa700 1 -- 172.21.15.93:0/358332407 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de400e040 con 0x7f7dec06d3a0 2023-07-21T22:35:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec1a5620 con 0x7f7dec06d3a0 2023-07-21T22:35:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec1b1b90 con 0x7f7dec06d3a0 2023-07-21T22:35:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7de97fa700 1 -- 172.21.15.93:0/358332407 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7de4012ae0 con 0x7f7dec06d3a0 2023-07-21T22:35:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.068+0000 7f7de97fa700 1 -- 172.21.15.93:0/358332407 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4005670 con 0x7f7dec06d3a0 2023-07-21T22:35:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.076+0000 7f7de97fa700 1 -- 172.21.15.93:0/358332407 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7de4012c50 con 0x7f7dec06d3a0 2023-07-21T22:35:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.077+0000 7f7de97fa700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 0x7f7dd40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.077+0000 7f7de97fa700 1 -- 172.21.15.93:0/358332407 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7de40ce1a0 con 0x7f7dec06d3a0 2023-07-21T22:35:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.078+0000 7f7dfe8e3b80 1 Processor -- start 2023-07-21T22:35:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.078+0000 7f7dfe8e3b80 1 -- start start 2023-07-21T22:35:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea36cdc50 con 0x55aea37ce660 2023-07-21T22:35:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea37209e0 con 0x55aea37ce210 2023-07-21T22:35:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.079+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea3719e70 con 0x55aea36246b0 2023-07-21T22:35:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.081+0000 7f7deb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.081+0000 7f7df0cfb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.081+0000 7f7df0cfb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49198/0 (socket says 172.21.15.93:49198) 2023-07-21T22:35:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.081+0000 7f7df0cfb700 1 -- 172.21.15.93:0/1988957930 learned_addr learned my addr 172.21.15.93:0/1988957930 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.082+0000 7f7df0cfb700 1 -- 172.21.15.93:0/1988957930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 msgr2=0x55aea37cdcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:35:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.082+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37cdcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.082+0000 7f7df0cfb700 1 -- 172.21.15.93:0/1988957930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 msgr2=0x55aea37c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.082+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.082+0000 7f7df0cfb700 1 -- 172.21.15.93:0/1988957930 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aea3547d80 con 0x55aea37ce660 2023-07-21T22:35:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.083+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009740 tx=0x7f7de0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3dbd5721bd94f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.083+0000 7f7dd27fc700 1 -- 172.21.15.93:0/1988957930 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0011830 con 0x55aea37ce660 2023-07-21T22:35:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.083+0000 7f7dd27fc700 1 -- 172.21.15.93:0/1988957930 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7de0011e50 con 0x55aea37ce660 2023-07-21T22:35:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.083+0000 7f7dd27fc700 1 -- 172.21.15.93:0/1988957930 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de000f2e0 con 0x55aea37ce660 2023-07-21T22:35:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.083+0000 7f7debfff700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 0x7f7dd40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/1988957930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 msgr2=0x55aea37c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009740 tx=0x7f7de0009a50 comp rx=0 tx=0).stop 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/1988957930 shutdown_connections 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37c84a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/1988957930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.084+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/1988957930 >> 172.21.15.93:0/1988957930 conn(0x55aea37bdae0 msgr2=0x55aea37c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.085+0000 7f7debfff700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 0x7f7dd40a3830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006fd0 tx=0x7f7ddc008040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.089+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/1988957930 shutdown_connections 2023-07-21T22:35:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.090+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/1988957930 wait complete. 2023-07-21T22:35:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.091+0000 7f7dfe8e3b80 1 -- start start 2023-07-21T22:35:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.091+0000 7f7dfe8e3b80 1 Processor -- start 2023-07-21T22:35:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea36b9a60 con 0x55aea37ce660 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea36b9760 con 0x55aea37ce210 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea359d8c0 con 0x55aea36246b0 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7dfe8e3b80 10 client.0 _ll_register_callbacks cb 0x55aea36b7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7df0cfb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7df0cfb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49206/0 (socket says 172.21.15.93:49206) 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.095+0000 7f7df0cfb700 1 -- 172.21.15.93:0/2139719234 learned_addr learned my addr 172.21.15.93:0/2139719234 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7debfff700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 -- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 msgr2=0x55aea37d4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 -- 172.21.15.93:0/2139719234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 msgr2=0x55aea37d1830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37d1830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 -- 172.21.15.93:0/2139719234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de0009370 con 0x55aea37ce660 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7de00048e0 tx=0x7f7de00105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0414e23bddf7255 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.096+0000 7f7dcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0010eb0 con 0x55aea37ce660 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7de0011630 con 0x55aea37ce660 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aea3673490 con 0x55aea37ce660 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7de00117f0 con 0x55aea37ce660 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dfe8e3b80 10 client.17607 fetch_fsmap learned FSMap version 81 2023-07-21T22:35:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.097+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55aea3530c00 con 0x55aea37ce660 2023-07-21T22:35:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.098+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aea3673490 con 0x55aea37ce660 2023-07-21T22:35:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.098+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 81) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7de001ac80 con 0x55aea37ce660 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 10 client.17607 fetch_fsmap finished waiting for FSMap version 81 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 20 client.17607 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37e1b00 0x55aea37e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 4 client.17607 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.099+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aea3712e50 con 0x55aea37e1b00 2023-07-21T22:35:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.102+0000 7f7debfff700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37e1b00 0x55aea37e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.124+0000 7f7debfff700 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37e1b00 0x55aea37e3f10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad9d9b41d626400 server_cookie=3f77942489608619 in_seq=0 out_seq=0 2023-07-21T22:35:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.126+0000 7f7dcb7fe700 10 client.17607 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.140+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/2139719234 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aea3712e50 con 0x55aea37e1b00 2023-07-21T22:35:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.140+0000 7f7dcb7fe700 10 client.17607 handle_command_reply: tid=0 2023-07-21T22:35:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.141+0000 7f7dfe8e3b80 1 client.17607 shutdown 2023-07-21T22:35:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.144+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 msgr2=0x55aea37d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.144+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7de00048e0 tx=0x7f7de00105b0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 shutdown_connections 2023-07-21T22:35:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37e1b00 0x55aea37e3f10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37ce660 0x55aea37d1f70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/2139719234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 >> 172.21.15.93:0/2139719234 conn(0x55aea37bdae0 msgr2=0x55aea37c23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dcb7fe700 0 client.17607 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 shutdown_connections 2023-07-21T22:35:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/2139719234 wait complete. 2023-07-21T22:35:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.146+0000 7f7dfe8e3b80 20 client.17607 trim_cache size 0 max 16384 2023-07-21T22:35:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 Processor -- start 2023-07-21T22:35:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 -- start start 2023-07-21T22:35:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea37d22d0 con 0x55aea37bdae0 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea37c49d0 con 0x55aea37ce210 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea37cc220 con 0x55aea36246b0 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7debfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7debfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49210/0 (socket says 172.21.15.93:49210) 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7debfff700 1 -- 172.21.15.93:0/3607445603 learned_addr learned my addr 172.21.15.93:0/3607445603 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.172+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 -- 172.21.15.93:0/3607445603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 msgr2=0x55aea37d4150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 -- 172.21.15.93:0/3607445603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 msgr2=0x55aea37c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 -- 172.21.15.93:0/3607445603 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aea356e1f0 con 0x55aea37bdae0 2023-07-21T22:35:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7debfff700 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00c430 tx=0x7f7ddc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84c71800430acf2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7dd2ffd700 1 -- 172.21.15.93:0/3607445603 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc024210 con 0x55aea37bdae0 2023-07-21T22:35:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7dd2ffd700 1 -- 172.21.15.93:0/3607445603 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ddc00cea0 con 0x55aea37bdae0 2023-07-21T22:35:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.173+0000 7f7dd2ffd700 1 -- 172.21.15.93:0/3607445603 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc013a70 con 0x55aea37bdae0 2023-07-21T22:35:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/3607445603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 msgr2=0x55aea37cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00c430 tx=0x7f7ddc00f000 comp rx=0 tx=0).stop 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/3607445603 shutdown_connections 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea37c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea37bdae0 0x55aea37cd020 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/3607445603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea36246b0 0x55aea37d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/3607445603 >> 172.21.15.93:0/3607445603 conn(0x55aea37dd740 msgr2=0x55aea37cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/3607445603 shutdown_connections 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/3607445603 wait complete. 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.174+0000 7f7dfe8e3b80 1 -- start start 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 Processor -- start 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea37bdae0 0x55aea37d7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea382e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea382ef00 con 0x55aea36246b0 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea382f070 con 0x55aea37ce210 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea382f230 con 0x55aea37bdae0 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7dfe8e3b80 10 client.0 _ll_register_callbacks cb 0x7f7de0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49214/0 (socket says 172.21.15.93:49214) 2023-07-21T22:35:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 -- 172.21.15.93:0/286580915 learned_addr learned my addr 172.21.15.93:0/286580915 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7df0cfb700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea382e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 -- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea37bdae0 msgr2=0x55aea37d7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea37bdae0 0x55aea37d7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 -- 172.21.15.93:0/286580915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 msgr2=0x55aea382e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea382e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.175+0000 7f7debfff700 1 -- 172.21.15.93:0/286580915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc00c640 con 0x55aea36246b0 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.176+0000 7f7debfff700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007600 tx=0x7f7ddc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755d64aa3784196b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.176+0000 7f7dcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.176+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc024620 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.176+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f7ddc005a20 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.176+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7de0011e50 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ddc005be0 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 10 client.17613 fetch_fsmap learned FSMap version 81 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55aea3546250 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ddc03b030 con 0x55aea36246b0 2023-07-21T22:35:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 81) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7ddc006240 con 0x55aea36246b0 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 10 client.17613 fetch_fsmap finished waiting for FSMap version 81 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 20 client.17613 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37ce660 0x55aea38389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 4 client.17613 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.177+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aea36d4ff0 con 0x55aea37ce660 2023-07-21T22:35:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.184+0000 7f7deb7fe700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37ce660 0x55aea38389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.188+0000 7f7dcb7fe700 10 client.17613 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.188+0000 7f7deb7fe700 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37ce660 0x55aea38389d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c179e0a5cb32f749 server_cookie=7b04ddf98d009d0d in_seq=0 out_seq=0 2023-07-21T22:35:06.525 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214514 33.12 MB/sec execute 222 sec latency 2087.004 ms 2023-07-21T22:35:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.623+0000 7f7dcb7fe700 1 -- 172.21.15.93:0/286580915 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aea36d4ff0 con 0x55aea37ce660 2023-07-21T22:35:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.623+0000 7f7dcb7fe700 10 client.17613 handle_command_reply: tid=0 2023-07-21T22:35:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 client.17613 shutdown 2023-07-21T22:35:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 msgr2=0x55aea37d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007600 tx=0x7f7ddc00cb90 comp rx=0 tx=0).stop 2023-07-21T22:35:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 shutdown_connections 2023-07-21T22:35:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55aea37ce660 0x55aea38389d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55aea37ce210 0x55aea382e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aea36246b0 0x55aea37d6e10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 --2- 172.21.15.93:0/286580915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aea37bdae0 0x55aea37d7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 >> 172.21.15.93:0/286580915 conn(0x55aea37dd740 msgr2=0x55aea37d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dcb7fe700 0 client.17613 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 shutdown_connections 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.624+0000 7f7dfe8e3b80 1 -- 172.21.15.93:0/286580915 wait complete. 2023-07-21T22:35:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.625+0000 7f7dfe8e3b80 20 client.17613 trim_cache size 0 max 16384 2023-07-21T22:35:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:35:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:35:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:35:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:35:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.626+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 msgr2=0x7f7dd40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.626+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 0x7f7dd40a3830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006fd0 tx=0x7f7ddc008040 comp rx=0 tx=0).stop 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.626+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 msgr2=0x7f7dec1ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.626+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b070 tx=0x7f7de4005b30 comp rx=0 tx=0).stop 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 shutdown_connections 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dec10e2a0 0x7f7dec1a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dec06c910 0x7f7dec1aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f7dd40a1380 0x7f7dd40a3830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 --2- 172.21.15.93:0/358332407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7dec06d3a0 0x7f7dec1ab0e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 >> 172.21.15.93:0/358332407 conn(0x7f7dec06bd70 msgr2=0x7f7dec06f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 shutdown_connections 2023-07-21T22:35:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:06.627+0000 7f7df275e700 1 -- 172.21.15.93:0/358332407 wait complete. 2023-07-21T22:35:07.475 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:35:07.475 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:35:07.475 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:35:07.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: pgmap v593: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 384 op/s 2023-07-21T22:35:07.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: daemon mds.h restarted 2023-07-21T22:35:07.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:35:07.476 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/4292712299,v1:172.21.15.134:6837/4292712299] up:boot 2023-07-21T22:35:07.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:07.477 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:07 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:35:07.525 INFO:tasks.workunit.client.0.smithi093.stdout: 10 214861 33.03 MB/sec execute 223 sec latency 250.061 ms 2023-07-21T22:35:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: pgmap v593: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 384 op/s 2023-07-21T22:35:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: daemon mds.h restarted 2023-07-21T22:35:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:35:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/4292712299,v1:172.21.15.134:6837/4292712299] up:boot 2023-07-21T22:35:07.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:07.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:07 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:35:07.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: pgmap v593: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.2 MiB/s wr, 384 op/s 2023-07-21T22:35:07.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: daemon mds.h restarted 2023-07-21T22:35:07.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:35:07.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/4292712299,v1:172.21.15.134:6837/4292712299] up:boot 2023-07-21T22:35:07.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:07.947 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:07 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:35:08.526 INFO:tasks.workunit.client.0.smithi093.stdout: 10 215160 32.95 MB/sec execute 224 sec latency 125.213 ms 2023-07-21T22:35:09.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 215398 32.87 MB/sec execute 225 sec latency 169.727 ms 2023-07-21T22:35:09.635 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:09 smithi134 ceph-mon[97873]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 718 op/s 2023-07-21T22:35:09.636 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:09 smithi134 ceph-mon[97873]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:35:09.695 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:09 smithi093 ceph-mon[96579]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 718 op/s 2023-07-21T22:35:09.696 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:09 smithi093 ceph-mon[96579]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:35:09.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:09 smithi194 ceph-mon[98288]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 20 MiB/s wr, 718 op/s 2023-07-21T22:35:09.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:09 smithi194 ceph-mon[98288]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:35:10.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 215723 32.81 MB/sec execute 226 sec latency 182.096 ms 2023-07-21T22:35:10.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:10 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:10.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:10 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:10.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:10 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:11.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 216277 32.78 MB/sec execute 227 sec latency 180.637 ms 2023-07-21T22:35:11.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:11 smithi194 ceph-mon[98288]: pgmap v597: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 27 MiB/s wr, 948 op/s 2023-07-21T22:35:11.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:11 smithi134 ceph-mon[97873]: pgmap v597: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 27 MiB/s wr, 948 op/s 2023-07-21T22:35:11.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:11 smithi093 ceph-mon[96579]: pgmap v597: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 27 MiB/s wr, 948 op/s 2023-07-21T22:35:12.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 216966 32.82 MB/sec execute 228 sec latency 59.604 ms 2023-07-21T22:35:13.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 217534 32.83 MB/sec execute 229 sec latency 61.850 ms 2023-07-21T22:35:13.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:13 smithi194 ceph-mon[98288]: pgmap v598: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 25 MiB/s wr, 884 op/s 2023-07-21T22:35:13.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:13 smithi134 ceph-mon[97873]: pgmap v598: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 25 MiB/s wr, 884 op/s 2023-07-21T22:35:13.945 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:13 smithi093 ceph-mon[96579]: pgmap v598: 97 pgs: 5 active+clean+snaptrim_wait, 90 active+clean, 2 active+clean+snaptrim; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 25 MiB/s wr, 884 op/s 2023-07-21T22:35:14.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 218145 32.80 MB/sec execute 230 sec latency 68.232 ms 2023-07-21T22:35:15.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 218849 32.79 MB/sec execute 231 sec latency 104.026 ms 2023-07-21T22:35:15.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:15 smithi194 ceph-mon[98288]: pgmap v599: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 1.18k op/s 2023-07-21T22:35:15.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:15 smithi134 ceph-mon[97873]: pgmap v599: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 1.18k op/s 2023-07-21T22:35:15.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:15 smithi093 ceph-mon[96579]: pgmap v599: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 28 MiB/s wr, 1.18k op/s 2023-07-21T22:35:16.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 219572 32.86 MB/sec execute 232 sec latency 99.261 ms 2023-07-21T22:35:17.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 220217 32.88 MB/sec execute 233 sec latency 100.513 ms 2023-07-21T22:35:17.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:17 smithi194 ceph-mon[98288]: pgmap v600: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-07-21T22:35:17.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:17 smithi194 ceph-mon[98288]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T22:35:17.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:17 smithi134 ceph-mon[97873]: pgmap v600: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-07-21T22:35:17.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:17 smithi134 ceph-mon[97873]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T22:35:17.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:17 smithi093 ceph-mon[96579]: pgmap v600: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 20 MiB/s wr, 897 op/s 2023-07-21T22:35:17.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:17 smithi093 ceph-mon[96579]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T22:35:18.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 220734 32.84 MB/sec execute 234 sec latency 70.116 ms 2023-07-21T22:35:19.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 221446 32.88 MB/sec execute 235 sec latency 51.222 ms 2023-07-21T22:35:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3865663564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 msgr2=0x7fa2f010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3865663564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f010b850 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b0a0 tx=0x7fa2e800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3865663564 shutdown_connections 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3865663564 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f010b850 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3865663564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 0x7fa2f0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3865663564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f00fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3865663564 >> 172.21.15.93:0/3865663564 conn(0x7fa2f00769b0 msgr2=0x7fa2f0076db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3865663564 shutdown_connections 2023-07-21T22:35:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.768+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3865663564 wait complete. 2023-07-21T22:35:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 Processor -- start 2023-07-21T22:35:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 -- start start 2023-07-21T22:35:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 0x7fa2f0194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01950c0 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0195230 con 0x7fa2f00fea20 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2f68f8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f01953f0 con 0x7fa2f0104a20 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.769+0000 7fa2ef7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49230/0 (socket says 172.21.15.93:49230) 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/3761101061 learned_addr learned my addr 172.21.15.93:0/3761101061 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 0x7fa2f0194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/3761101061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 msgr2=0x7fa2f0194a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 0x7fa2f0194a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/3761101061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 msgr2=0x7fa2f019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2effff700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/3761101061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009d00 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2effff700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e9a0 tx=0x7fa2e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ed7fa700 1 -- 172.21.15.93:0/3761101061 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4009790 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f01956d0 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ed7fa700 1 -- 172.21.15.93:0/3761101061 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e400cbd0 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2ed7fa700 1 -- 172.21.15.93:0/3761101061 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4018680 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.770+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f01a1be0 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.773+0000 7fa2ed7fa700 1 -- 172.21.15.93:0/3761101061 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa2e400cd40 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa2ed7fa700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 0x7fa2d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa2effff700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 0x7fa2d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa2ed7fa700 1 -- 172.21.15.93:0/3761101061 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa2e4014070 con 0x7fa2f00ff3d0 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa2effff700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 0x7fa2d80a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0068100 tx=0x7fa2e0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa302a7db80 1 Processor -- start 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.774+0000 7fa302a7db80 1 -- start start 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 0x55e69d952db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 0x55e69d94d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d94d8f0 0x55e69d961da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d852c50 con 0x55e69d94cf40 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d8a59e0 con 0x55e69d94d8f0 2023-07-21T22:35:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa302a7db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d89ee70 con 0x55e69d7a96b0 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2effff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 0x55e69d952db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 0x55e69d94d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2effff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 0x55e69d952db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36316/0 (socket says 172.21.15.93:36316) 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2effff700 1 -- 172.21.15.93:0/2197283766 learned_addr learned my addr 172.21.15.93:0/2197283766 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d94d8f0 0x55e69d961da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2197283766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 msgr2=0x55e69d952db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 0x55e69d952db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2197283766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d94d8f0 msgr2=0x55e69d961da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d94d8f0 0x55e69d961da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2197283766 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e69d6ccd80 con 0x55e69d94cf40 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.775+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 0x55e69d94d3b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ca5d0 tx=0x7fa2e4099da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742a5046bb5148ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa2d67fc700 1 -- 172.21.15.93:0/2197283766 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400b2c0 con 0x55e69d94cf40 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa2d67fc700 1 -- 172.21.15.93:0/2197283766 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e4099f50 con 0x55e69d94cf40 2023-07-21T22:35:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa2d67fc700 1 -- 172.21.15.93:0/2197283766 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4099440 con 0x55e69d94cf40 2023-07-21T22:35:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 -- 172.21.15.93:0/2197283766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 msgr2=0x55e69d94d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 0x55e69d94d3b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ca5d0 tx=0x7fa2e4099da0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 -- 172.21.15.93:0/2197283766 shutdown_connections 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d94d8f0 0x55e69d961da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d7a96b0 0x55e69d952db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2197283766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94cf40 0x55e69d94d3b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 -- 172.21.15.93:0/2197283766 >> 172.21.15.93:0/2197283766 conn(0x55e69d942ae0 msgr2=0x55e69d9480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.776+0000 7fa302a7db80 1 -- 172.21.15.93:0/2197283766 shutdown_connections 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 -- 172.21.15.93:0/2197283766 wait complete. 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 -- start start 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 Processor -- start 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d95ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d95f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d83ea60 con 0x55e69d94d8f0 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d83e760 con 0x55e69d7a96b0 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.777+0000 7fa302a7db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d7228c0 con 0x55e69d94cf40 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa302a7db80 10 client.0 _ll_register_callbacks cb 0x55e69d83c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60404/0 (socket says 172.21.15.93:60404) 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 -- 172.21.15.93:0/1445490028 learned_addr learned my addr 172.21.15.93:0/1445490028 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d95f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d95ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 -- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 msgr2=0x55e69d95ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d95ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 -- 172.21.15.93:0/1445490028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 msgr2=0x55e69d95f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d95f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 -- 172.21.15.93:0/1445490028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f01a1be0 con 0x55e69d7a96b0 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2effff700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0011440 tx=0x7fa2e0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea13b155bd87c61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.778+0000 7fa2cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:35:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.779+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0011ea0 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.779+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa2e0028670 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.780+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e69d7f8490 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.780+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e0028830 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.780+0000 7fa302a7db80 10 client.26987 fetch_fsmap learned FSMap version 83 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.780+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55e69d6b5c00 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e69d7f8490 con 0x55e69d7a96b0 2023-07-21T22:35:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 83) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa2e00325e0 con 0x55e69d7a96b0 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 10 client.26987 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 10 client.26987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 20 client.26987 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d96f260 0x55e69d971670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 4 client.26987 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e69d897e50 con 0x55e69d96f260 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.781+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d96f260 0x55e69d971670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.782+0000 7fa2cf7fe700 10 client.26987 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.782+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d96f260 0x55e69d971670 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=106a092381542140 server_cookie=185da82d7822404c in_seq=0 out_seq=0 2023-07-21T22:35:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.783+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/1445490028 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e69d897e50 con 0x55e69d96f260 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.783+0000 7fa2cf7fe700 10 client.26987 handle_command_reply: tid=0 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.783+0000 7fa302a7db80 1 client.26987 shutdown 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.783+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 msgr2=0x55e69d961d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.783+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0011440 tx=0x7fa2e0004750 comp rx=0 tx=0).stop 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 shutdown_connections 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d96f260 0x55e69d971670 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d95f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa2cf7fe700 0 client.26987 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 --2- 172.21.15.93:0/1445490028 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d95ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 >> 172.21.15.93:0/1445490028 conn(0x55e69d942ae0 msgr2=0x55e69d9473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 shutdown_connections 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 1 -- 172.21.15.93:0/1445490028 wait complete. 2023-07-21T22:35:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.784+0000 7fa302a7db80 20 client.26987 trim_cache size 0 max 16384 2023-07-21T22:35:19.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:19 smithi194 ceph-mon[98288]: pgmap v602: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.803+0000 7fa302a7db80 1 Processor -- start 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.803+0000 7fa302a7db80 1 -- start start 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d951e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d953380 con 0x55e69d94d8f0 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d950990 con 0x55e69d7a96b0 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa302a7db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d949760 con 0x55e69d94cf40 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49254/0 (socket says 172.21.15.93:49254) 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 -- 172.21.15.93:0/2152997889 learned_addr learned my addr 172.21.15.93:0/2152997889 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d951e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 -- 172.21.15.93:0/2152997889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 msgr2=0x55e69d951e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d951e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2effff700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 -- 172.21.15.93:0/2152997889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 msgr2=0x55e69d961870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 -- 172.21.15.93:0/2152997889 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e69d6f31f0 con 0x55e69d94d8f0 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2effff700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009320 tx=0x7fa2e8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6b33a7de6d397f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2d6ffd700 1 -- 172.21.15.93:0/2152997889 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e800e040 con 0x55e69d94d8f0 2023-07-21T22:35:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2d6ffd700 1 -- 172.21.15.93:0/2152997889 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e8009b80 con 0x55e69d94d8f0 2023-07-21T22:35:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.804+0000 7fa2d6ffd700 1 -- 172.21.15.93:0/2152997889 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80052a0 con 0x55e69d94d8f0 2023-07-21T22:35:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 -- 172.21.15.93:0/2152997889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 msgr2=0x55e69d952350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009320 tx=0x7fa2e8009350 comp rx=0 tx=0).stop 2023-07-21T22:35:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 -- 172.21.15.93:0/2152997889 shutdown_connections 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d952350 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d961870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2152997889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d951e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 -- 172.21.15.93:0/2152997889 >> 172.21.15.93:0/2152997889 conn(0x55e69d942ae0 msgr2=0x55e69d96b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 -- 172.21.15.93:0/2152997889 shutdown_connections 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.805+0000 7fa302a7db80 1 -- 172.21.15.93:0/2152997889 wait complete. 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 -- start start 2023-07-21T22:35:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 Processor -- start 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d9660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d964920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d9669c0 con 0x55e69d94d8f0 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d964e60 con 0x55e69d7a96b0 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69d964fd0 con 0x55e69d94cf40 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.806+0000 7fa302a7db80 10 client.0 _ll_register_callbacks cb 0x7fa2e000fbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36336/0 (socket says 172.21.15.93:36336) 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2344753098 learned_addr learned my addr 172.21.15.93:0/2344753098 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2effff700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d9660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2f4e95700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d964920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2344753098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 msgr2=0x55e69d9660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d9660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2344753098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 msgr2=0x55e69d964920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d964920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 -- 172.21.15.93:0/2344753098 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f010c150 con 0x55e69d94cf40 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2ef7fe700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4099e90 tx=0x7fa2e401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365946abe4f92cc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40a23a0 con 0x55e69d94cf40 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e69d859ff0 con 0x55e69d94cf40 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.807+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa2e4099440 con 0x55e69d94cf40 2023-07-21T22:35:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.808+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e4099600 con 0x55e69d94cf40 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.808+0000 7fa302a7db80 10 client.26878 fetch_fsmap learned FSMap version 83 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.808+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55e69d6cb250 con 0x55e69d94cf40 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e69d859ff0 con 0x55e69d94cf40 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 83) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa2e40ca020 con 0x55e69d94cf40 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 10 client.26878 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 20 client.26878 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d9c41f0 0x55e69d9c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 4 client.26878 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e69d6edf10 con 0x55e69d9c41f0 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa2effff700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d9c41f0 0x55e69d9c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa2cf7fe700 10 client.26878 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:19.809+0000 7fa2effff700 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d9c41f0 0x55e69d9c6600 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438d8394713e10ab server_cookie=9fadabe9072e9d55 in_seq=0 out_seq=0 2023-07-21T22:35:19.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:19 smithi134 ceph-mon[97873]: pgmap v602: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2023-07-21T22:35:19.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:19 smithi093 ceph-mon[96579]: pgmap v602: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 639 op/s 2023-07-21T22:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa2cf7fe700 1 -- 172.21.15.93:0/2344753098 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e69d6edf10 con 0x55e69d9c41f0 2023-07-21T22:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa2cf7fe700 10 client.26878 handle_command_reply: tid=0 2023-07-21T22:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 client.26878 shutdown 2023-07-21T22:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 msgr2=0x55e69d9643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4099e90 tx=0x7fa2e401f000 comp rx=0 tx=0).stop 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 shutdown_connections 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55e69d9c41f0 0x55e69d9c6600 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e69d94d8f0 0x55e69d964920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e69d7a96b0 0x55e69d9660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 --2- 172.21.15.93:0/2344753098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e69d94cf40 0x55e69d9643e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 >> 172.21.15.93:0/2344753098 conn(0x55e69d942ae0 msgr2=0x55e69d951e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:20.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa2cf7fe700 0 client.26878 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 shutdown_connections 2023-07-21T22:35:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.093+0000 7fa302a7db80 1 -- 172.21.15.93:0/2344753098 wait complete. 2023-07-21T22:35:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:35:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:35:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:35:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:35:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.095+0000 7fa302a7db80 20 client.26878 trim_cache size 0 max 16384 2023-07-21T22:35:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.099+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 msgr2=0x7fa2d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.099+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 0x7fa2d80a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0068100 tx=0x7fa2e0009260 comp rx=0 tx=0).stop 2023-07-21T22:35:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.099+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 msgr2=0x7fa2f019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.099+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e9a0 tx=0x7fa2e400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 shutdown_connections 2023-07-21T22:35:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2f0104a20 0x7fa2f0194a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00fea20 0x7fa2f019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2d80a12c0 0x7fa2d80a3770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 --2- 172.21.15.93:0/3761101061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2f00ff3d0 0x7fa2f019b130 secure :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400e9a0 tx=0x7fa2e400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.100+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 >> 172.21.15.93:0/3761101061 conn(0x7fa2f00769b0 msgr2=0x7fa2f00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.101+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 shutdown_connections 2023-07-21T22:35:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:20.101+0000 7fa2f68f8700 1 -- 172.21.15.93:0/3761101061 wait complete. 2023-07-21T22:35:20.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 222097 32.85 MB/sec execute 236 sec latency 190.161 ms 2023-07-21T22:35:21.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 222839 32.92 MB/sec execute 237 sec latency 65.475 ms 2023-07-21T22:35:21.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:21 smithi194 ceph-mon[98288]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 591 op/s 2023-07-21T22:35:21.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:21 smithi134 ceph-mon[97873]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 591 op/s 2023-07-21T22:35:21.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:21 smithi093 ceph-mon[96579]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 591 op/s 2023-07-21T22:35:22.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 223517 32.93 MB/sec execute 238 sec latency 81.680 ms 2023-07-21T22:35:23.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 224031 32.91 MB/sec execute 239 sec latency 72.541 ms 2023-07-21T22:35:23.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:23 smithi194 ceph-mon[98288]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2023-07-21T22:35:23.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:23 smithi134 ceph-mon[97873]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2023-07-21T22:35:23.947 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:23 smithi093 ceph-mon[96579]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2023-07-21T22:35:24.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 224682 32.90 MB/sec execute 240 sec latency 88.107 ms 2023-07-21T22:35:25.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 225444 32.96 MB/sec execute 241 sec latency 98.717 ms 2023-07-21T22:35:25.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:25 smithi194 ceph-mon[98288]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:25.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:25 smithi134 ceph-mon[97873]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:25.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:25 smithi093 ceph-mon[96579]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:26.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 226000 32.92 MB/sec execute 242 sec latency 111.420 ms 2023-07-21T22:35:27.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 226500 32.90 MB/sec execute 243 sec latency 221.068 ms 2023-07-21T22:35:27.846 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:27 smithi194 ceph-mon[98288]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:27.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:27 smithi134 ceph-mon[97873]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:27.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:27 smithi093 ceph-mon[96579]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 612 op/s 2023-07-21T22:35:28.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 227152 32.92 MB/sec execute 244 sec latency 103.807 ms 2023-07-21T22:35:29.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 227853 32.95 MB/sec execute 245 sec latency 67.453 ms 2023-07-21T22:35:29.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:29 smithi194 ceph-mon[98288]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2023-07-21T22:35:29.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:29 smithi134 ceph-mon[97873]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2023-07-21T22:35:29.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:29 smithi093 ceph-mon[96579]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2023-07-21T22:35:30.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 228512 32.97 MB/sec execute 246 sec latency 73.701 ms 2023-07-21T22:35:31.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 229131 32.93 MB/sec execute 247 sec latency 53.695 ms 2023-07-21T22:35:31.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:31 smithi194 ceph-mon[98288]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-07-21T22:35:31.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:31 smithi134 ceph-mon[97873]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-07-21T22:35:31.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:31 smithi093 ceph-mon[96579]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 501 op/s 2023-07-21T22:35:32.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 229764 32.95 MB/sec execute 248 sec latency 88.581 ms 2023-07-21T22:35:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 -- 172.21.15.93:0/2803216665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 msgr2=0x7f5bc8109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/2803216665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc8109b70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 -- 172.21.15.93:0/2803216665 shutdown_connections 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/2803216665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 0x7f5bc810c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/2803216665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 0x7f5bc8100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/2803216665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc8109b70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 -- 172.21.15.93:0/2803216665 >> 172.21.15.93:0/2803216665 conn(0x7f5bc80fa500 msgr2=0x7f5bc80fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 -- 172.21.15.93:0/2803216665 shutdown_connections 2023-07-21T22:35:33.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.236+0000 7f5bd00c4700 1 -- 172.21.15.93:0/2803216665 wait complete. 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 Processor -- start 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 -- start start 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 0x7f5bc819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 0x7f5bc8195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc81960d0 con 0x7f5bc81014a0 2023-07-21T22:35:33.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8196240 con 0x7f5bc8100b90 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bd00c4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8196400 con 0x7f5bc810a0b0 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bcde60700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 0x7f5bc819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49272/0 (socket says 172.21.15.93:49272) 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bcd65f700 1 -- 172.21.15.93:0/12308723 learned_addr learned my addr 172.21.15.93:0/12308723 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.237+0000 7f5bce661700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 0x7f5bc8195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 -- 172.21.15.93:0/12308723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 msgr2=0x7f5bc8195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 0x7f5bc8195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 -- 172.21.15.93:0/12308723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 msgr2=0x7f5bc819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 0x7f5bc819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 -- 172.21.15.93:0/12308723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80095f0 con 0x7f5bc81014a0 2023-07-21T22:35:33.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bbeffd700 1 -- 172.21.15.93:0/12308723 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801d070 con 0x7f5bc81014a0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bbeffd700 1 -- 172.21.15.93:0/12308723 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb8021410 con 0x7f5bc81014a0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8196680 con 0x7f5bc81014a0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bbeffd700 1 -- 172.21.15.93:0/12308723 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017650 con 0x7f5bc81014a0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.238+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc81a2bf0 con 0x7f5bc81014a0 2023-07-21T22:35:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.240+0000 7f5bbeffd700 1 -- 172.21.15.93:0/12308723 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5bb8021580 con 0x7f5bc81014a0 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.241+0000 7f5bbeffd700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 0x7f5bb40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.241+0000 7f5bbeffd700 1 -- 172.21.15.93:0/12308723 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5bb80cdfa0 con 0x7f5bc81014a0 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.241+0000 7f5bcde60700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 0x7f5bb40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.241+0000 7f5bcde60700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 0x7f5bb40a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400f540 tx=0x7f5bc400b040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.241+0000 7f5bdc249b80 1 Processor -- start 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 -- start start 2023-07-21T22:35:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 0x562877b31350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877a32c50 con 0x562877b37a70 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877a859e0 con 0x562877b37620 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877a7ee70 con 0x5628779896b0 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 0x562877b31350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49282/0 (socket says 172.21.15.93:49282) 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 0x562877b31350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60442/0 (socket says 172.21.15.93:60442) 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 -- 172.21.15.93:0/2043593818 learned_addr learned my addr 172.21.15.93:0/2043593818 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bcde60700 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 -- 172.21.15.93:0/2043593818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 msgr2=0x562877b370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 -- 172.21.15.93:0/2043593818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 msgr2=0x562877b31350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 0x562877b31350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.242+0000 7f5bce661700 1 -- 172.21.15.93:0/2043593818 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628778acd80 con 0x562877b37a70 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.243+0000 7f5bce661700 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009c30 tx=0x7f5bc0009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7897043024bcb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.243+0000 7f5bb3fff700 1 -- 172.21.15.93:0/2043593818 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0011d10 con 0x562877b37a70 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.243+0000 7f5bb3fff700 1 -- 172.21.15.93:0/2043593818 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc0011e70 con 0x562877b37a70 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.243+0000 7f5bb3fff700 1 -- 172.21.15.93:0/2043593818 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0019430 con 0x562877b37a70 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- 172.21.15.93:0/2043593818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 msgr2=0x562877b318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009c30 tx=0x7f5bc0009f40 comp rx=0 tx=0).stop 2023-07-21T22:35:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- 172.21.15.93:0/2043593818 shutdown_connections 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37a70 0x562877b318b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/2043593818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37620 0x562877b31350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- 172.21.15.93:0/2043593818 >> 172.21.15.93:0/2043593818 conn(0x562877b22ae0 msgr2=0x562877b280e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- 172.21.15.93:0/2043593818 shutdown_connections 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- 172.21.15.93:0/2043593818 wait complete. 2023-07-21T22:35:33.296 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.244+0000 7f5bdc249b80 1 -- start start 2023-07-21T22:35:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 Processor -- start 2023-07-21T22:35:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877a1ea60 con 0x562877b37620 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877a1e760 con 0x562877b37a70 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628779028c0 con 0x5628779896b0 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bdc249b80 10 client.0 _ll_register_callbacks cb 0x562877a1c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bcde60700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bce661700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bcde60700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36364/0 (socket says 172.21.15.93:36364) 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bce661700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60448/0 (socket says 172.21.15.93:60448) 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bcde60700 1 -- 172.21.15.93:0/3714319668 learned_addr learned my addr 172.21.15.93:0/3714319668 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.245+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 -- 172.21.15.93:0/3714319668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 msgr2=0x562877b3ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 -- 172.21.15.93:0/3714319668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 msgr2=0x562877b40f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b40f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 -- 172.21.15.93:0/3714319668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc00098e0 con 0x5628779896b0 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bce661700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bcde60700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4004660 tx=0x7f5bc4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43377bd4d268d478 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400e040 con 0x5628779896b0 2023-07-21T22:35:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5bc4015cf0 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628779d8490 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc4009020 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bdc249b80 10 client.26890 fetch_fsmap learned FSMap version 83 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.246+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x562877895c00 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628779d8490 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 83) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5bc4008c20 con 0x5628779896b0 2023-07-21T22:35:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 10 client.26890 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 10 client.26890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 20 client.26890 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b4af30 0x562877b4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 4 client.26890 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562877a77e50 con 0x562877b4af30 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.247+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b4af30 0x562877b4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.248+0000 7f5bb17fa700 10 client.26890 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:33.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.248+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b4af30 0x562877b4d340 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9aa911ff3c768a server_cookie=8cc5ac31d84500b in_seq=0 out_seq=0 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3714319668 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562877a77e50 con 0x562877b4af30 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bb17fa700 10 client.26890 handle_command_reply: tid=0 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 client.26890 shutdown 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 msgr2=0x562877b40a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4004660 tx=0x7f5bc4004ad0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 shutdown_connections 2023-07-21T22:35:33.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b4af30 0x562877b4d340 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b3ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b40a20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bb17fa700 0 client.26890 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3714319668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b40f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 >> 172.21.15.93:0/3714319668 conn(0x562877b22ae0 msgr2=0x562877b2bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.249+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 shutdown_connections 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.250+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3714319668 wait complete. 2023-07-21T22:35:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.250+0000 7f5bdc249b80 20 client.26890 trim_cache size 0 max 16384 2023-07-21T22:35:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 Processor -- start 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 -- start start 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b323b0 con 0x562877b37620 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b28f80 con 0x562877b37a70 2023-07-21T22:35:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b24c50 con 0x5628779896b0 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49294/0 (socket says 172.21.15.93:49294) 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bcd65f700 1 -- 172.21.15.93:0/826359647 learned_addr learned my addr 172.21.15.93:0/826359647 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bcde60700 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.267+0000 7f5bce661700 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 -- 172.21.15.93:0/826359647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 msgr2=0x562877b3ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b3ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 -- 172.21.15.93:0/826359647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 msgr2=0x562877b27b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b27b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 -- 172.21.15.93:0/826359647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628778d31f0 con 0x562877b37620 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8097600 tx=0x7f5bb8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf06c33badf5859 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bbcff9700 1 -- 172.21.15.93:0/826359647 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80b7070 con 0x562877b37620 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bbcff9700 1 -- 172.21.15.93:0/826359647 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bb809b100 con 0x562877b37620 2023-07-21T22:35:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.268+0000 7f5bbcff9700 1 -- 172.21.15.93:0/826359647 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80995c0 con 0x562877b37620 2023-07-21T22:35:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 -- 172.21.15.93:0/826359647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 msgr2=0x562877b36430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8097600 tx=0x7f5bb8009310 comp rx=0 tx=0).stop 2023-07-21T22:35:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 -- 172.21.15.93:0/826359647 shutdown_connections 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562877b37a70 0x562877b27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5628779896b0 0x562877b3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/826359647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b36430 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 -- 172.21.15.93:0/826359647 >> 172.21.15.93:0/826359647 conn(0x562877b22ae0 msgr2=0x562877b35d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 -- 172.21.15.93:0/826359647 shutdown_connections 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.269+0000 7f5bdc249b80 1 -- 172.21.15.93:0/826359647 wait complete. 2023-07-21T22:35:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.270+0000 7f5bdc249b80 1 -- start start 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.270+0000 7f5bdc249b80 1 Processor -- start 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.271+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628779896b0 0x562877b3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.271+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.271+0000 7f5bdc249b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562877b37a70 0x562877b3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b9dfa0 con 0x562877b37620 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b9e110 con 0x5628779896b0 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bdc249b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562877b9e2d0 con 0x562877b37a70 2023-07-21T22:35:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bdc249b80 10 client.0 _ll_register_callbacks cb 0x7f5bc4004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49298/0 (socket says 172.21.15.93:49298) 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 -- 172.21.15.93:0/3807475079 learned_addr learned my addr 172.21.15.93:0/3807475079 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bce661700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562877b37a70 0x562877b3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 -- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562877b37a70 msgr2=0x562877b3f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562877b37a70 0x562877b3f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 -- 172.21.15.93:0/3807475079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628779896b0 msgr2=0x562877b3e610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:35:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628779896b0 0x562877b3e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 -- 172.21.15.93:0/3807475079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc81a2bf0 con 0x562877b37620 2023-07-21T22:35:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bcd65f700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80cb7a0 tx=0x7f5bb80a34c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45edf0d4f905eae6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.272+0000 7f5bb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:35:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80b7020 con 0x562877b37620 2023-07-21T22:35:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5bb80cbe40 con 0x562877b37620 2023-07-21T22:35:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc0011e70 con 0x562877b37620 2023-07-21T22:35:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb809f130 con 0x562877b37620 2023-07-21T22:35:33.324 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 10 client.17655 fetch_fsmap learned FSMap version 83 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x5628778ab250 con 0x562877b37620 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc0011e70 con 0x562877b37620 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 83) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5bb80a3030 con 0x562877b37620 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 10 client.17655 fetch_fsmap finished waiting for FSMap version 83 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 20 client.17655 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.273+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b9f5c0 0x562877ba19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.274+0000 7f5bdc249b80 4 client.17655 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.274+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562877a39ff0 con 0x562877b9f5c0 2023-07-21T22:35:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.274+0000 7f5bcde60700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b9f5c0 0x562877ba19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.274+0000 7f5bb17fa700 10 client.17655 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:33.326 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.274+0000 7f5bcde60700 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b9f5c0 0x562877ba19d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8c25232b866559 server_cookie=fa9aadcd97aa7abe in_seq=0 out_seq=0 2023-07-21T22:35:33.527 INFO:tasks.workunit.client.0.smithi093.stdout: 10 230370 32.95 MB/sec execute 249 sec latency 167.129 ms 2023-07-21T22:35:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.500+0000 7f5bb17fa700 1 -- 172.21.15.93:0/3807475079 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562877a39ff0 con 0x562877b9f5c0 2023-07-21T22:35:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.500+0000 7f5bb17fa700 10 client.17655 handle_command_reply: tid=0 2023-07-21T22:35:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.500+0000 7f5bdc249b80 1 client.17655 shutdown 2023-07-21T22:35:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 msgr2=0x562877b3eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80cb7a0 tx=0x7f5bb80a34c0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 shutdown_connections 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562877b9f5c0 0x562877ba19d0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562877b37a70 0x562877b3f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bb17fa700 0 client.17655 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5628779896b0 0x562877b3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 --2- 172.21.15.93:0/3807475079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562877b37620 0x562877b3eb50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 >> 172.21.15.93:0/3807475079 conn(0x562877b22ae0 msgr2=0x562877b39cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 shutdown_connections 2023-07-21T22:35:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 1 -- 172.21.15.93:0/3807475079 wait complete. 2023-07-21T22:35:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bdc249b80 20 client.17655 trim_cache size 0 max 16384 2023-07-21T22:35:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:35:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:35:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:35:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:35:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 msgr2=0x7f5bb40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 0x7f5bb40a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400f540 tx=0x7f5bc400b040 comp rx=0 tx=0).stop 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 msgr2=0x7f5bc819c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8000c00 tx=0x7f5bb8004790 comp rx=0 tx=0).stop 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 shutdown_connections 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5bc810a0b0 0x7f5bc8195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bc8100b90 0x7f5bc819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5bb40a12c0 0x7f5bb40a3770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 --2- 172.21.15.93:0/12308723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bc81014a0 0x7f5bc819c140 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.501+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 >> 172.21.15.93:0/12308723 conn(0x7f5bc80fa500 msgr2=0x7f5bc810c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.502+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 shutdown_connections 2023-07-21T22:35:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:33.502+0000 7f5bd00c4700 1 -- 172.21.15.93:0/12308723 wait complete. 2023-07-21T22:35:33.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:33 smithi194 ceph-mon[98288]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 512 op/s 2023-07-21T22:35:33.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:33 smithi134 ceph-mon[97873]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 512 op/s 2023-07-21T22:35:33.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:33 smithi093 ceph-mon[96579]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 512 op/s 2023-07-21T22:35:34.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 231028 32.94 MB/sec execute 250 sec latency 167.370 ms 2023-07-21T22:35:35.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 231649 32.97 MB/sec execute 251 sec latency 83.010 ms 2023-07-21T22:35:35.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:35 smithi194 ceph-mon[98288]: pgmap v610: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-07-21T22:35:35.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:35 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:35 smithi134 ceph-mon[97873]: pgmap v610: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-07-21T22:35:35.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:35 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:35.953 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:35 smithi093 ceph-mon[96579]: pgmap v610: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 688 op/s 2023-07-21T22:35:35.954 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:35 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:36.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 231746 32.86 MB/sec execute 252 sec latency 873.919 ms 2023-07-21T22:35:37.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 231944 32.78 MB/sec execute 253 sec latency 1309.847 ms 2023-07-21T22:35:37.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:37 smithi134 ceph-mon[97873]: pgmap v611: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 423 op/s 2023-07-21T22:35:37.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:37 smithi093 ceph-mon[96579]: pgmap v611: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 423 op/s 2023-07-21T22:35:38.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:37 smithi194 ceph-mon[98288]: pgmap v611: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.6 MiB/s wr, 423 op/s 2023-07-21T22:35:38.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 232231 32.71 MB/sec execute 254 sec latency 160.690 ms 2023-07-21T22:35:39.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 232744 32.66 MB/sec execute 255 sec latency 102.085 ms 2023-07-21T22:35:39.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:39 smithi134 ceph-mon[97873]: pgmap v612: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 853 op/s 2023-07-21T22:35:39.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:39 smithi093 ceph-mon[96579]: pgmap v612: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 853 op/s 2023-07-21T22:35:40.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:39 smithi194 ceph-mon[98288]: pgmap v612: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 853 op/s 2023-07-21T22:35:40.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 233455 32.70 MB/sec execute 256 sec latency 48.225 ms 2023-07-21T22:35:41.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 234073 32.71 MB/sec execute 257 sec latency 87.432 ms 2023-07-21T22:35:41.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:41 smithi134 ceph-mon[97873]: pgmap v613: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 734 op/s 2023-07-21T22:35:41.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:41 smithi093 ceph-mon[96579]: pgmap v613: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 734 op/s 2023-07-21T22:35:42.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:41 smithi194 ceph-mon[98288]: pgmap v613: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 20 MiB/s wr, 734 op/s 2023-07-21T22:35:42.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 234680 32.69 MB/sec execute 258 sec latency 79.339 ms 2023-07-21T22:35:43.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 235316 32.71 MB/sec execute 259 sec latency 97.416 ms 2023-07-21T22:35:43.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:43 smithi134 ceph-mon[97873]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 20 MiB/s wr, 785 op/s 2023-07-21T22:35:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:43 smithi093 ceph-mon[96579]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 20 MiB/s wr, 785 op/s 2023-07-21T22:35:44.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:43 smithi194 ceph-mon[98288]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 20 MiB/s wr, 785 op/s 2023-07-21T22:35:44.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 235989 32.73 MB/sec execute 260 sec latency 76.058 ms 2023-07-21T22:35:45.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 236617 32.74 MB/sec execute 261 sec latency 68.712 ms 2023-07-21T22:35:45.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:45 smithi134 ceph-mon[97873]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:45.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:45 smithi093 ceph-mon[96579]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:46.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:45 smithi194 ceph-mon[98288]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:46.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 237179 32.72 MB/sec execute 262 sec latency 91.272 ms 2023-07-21T22:35:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 -- 172.21.15.93:0/3910241430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 msgr2=0x7f15c810b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/3910241430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c810b870 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f15c000b0a0 tx=0x7f15c000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 -- 172.21.15.93:0/3910241430 shutdown_connections 2023-07-21T22:35:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/3910241430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c810b870 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/3910241430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/3910241430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 0x7f15c80feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 -- 172.21.15.93:0/3910241430 >> 172.21.15.93:0/3910241430 conn(0x7f15c80fa4f0 msgr2=0x7f15c80fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 -- 172.21.15.93:0/3910241430 shutdown_connections 2023-07-21T22:35:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.750+0000 7f15cf4ed700 1 -- 172.21.15.93:0/3910241430 wait complete. 2023-07-21T22:35:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 Processor -- start 2023-07-21T22:35:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 -- start start 2023-07-21T22:35:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 0x7f15c819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c8195080 con 0x7f15c80ff3e0 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81951f0 con 0x7f15c80fea30 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cf4ed700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c81953b0 con 0x7f15c8104a40 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cca88700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cca88700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49312/0 (socket says 172.21.15.93:49312) 2023-07-21T22:35:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cca88700 1 -- 172.21.15.93:0/2810415077 learned_addr learned my addr 172.21.15.93:0/2810415077 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cda8a700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.751+0000 7f15cd289700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 0x7f15c819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 -- 172.21.15.93:0/2810415077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 msgr2=0x7f15c8194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 -- 172.21.15.93:0/2810415077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 msgr2=0x7f15c819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 0x7f15c819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 -- 172.21.15.93:0/2810415077 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0009d00 con 0x7f15c80ff3e0 2023-07-21T22:35:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cda8a700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cca88700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f15b800e9a0 tx=0x7f15b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15be7fc700 1 -- 172.21.15.93:0/2810415077 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8009790 con 0x7f15c80ff3e0 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c8195690 con 0x7f15c80ff3e0 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15be7fc700 1 -- 172.21.15.93:0/2810415077 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15b8010430 con 0x7f15c80ff3e0 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15be7fc700 1 -- 172.21.15.93:0/2810415077 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8018680 con 0x7f15c80ff3e0 2023-07-21T22:35:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.752+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c806a190 con 0x7f15c80ff3e0 2023-07-21T22:35:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.755+0000 7f15be7fc700 1 -- 172.21.15.93:0/2810415077 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f15b80105a0 con 0x7f15c80ff3e0 2023-07-21T22:35:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.756+0000 7f15be7fc700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 0x7f15b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.756+0000 7f15be7fc700 1 -- 172.21.15.93:0/2810415077 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f15b8014070 con 0x7f15c80ff3e0 2023-07-21T22:35:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.756+0000 7f15cd289700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 0x7f15b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 Processor -- start 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 -- start start 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15cd289700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 0x7f15b40a3770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15c80ffb70 tx=0x7f15c4008040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f74cfc50 con 0x5592f75d3320 2023-07-21T22:35:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f75229e0 con 0x5592f75d00c0 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15db672b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f751be70 con 0x5592f74266b0 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.757+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36398/0 (socket says 172.21.15.93:36398) 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 -- 172.21.15.93:0/1881641157 learned_addr learned my addr 172.21.15.93:0/1881641157 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cda8a700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cca88700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 -- 172.21.15.93:0/1881641157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 msgr2=0x5592f75d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 -- 172.21.15.93:0/1881641157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 msgr2=0x5592f75cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 -- 172.21.15.93:0/1881641157 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f7349d80 con 0x5592f74266b0 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cda8a700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15cd289700 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15c40317e0 tx=0x7f15c4034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736f7472e6d040e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.758+0000 7f15b37fe700 1 -- 172.21.15.93:0/1881641157 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4036880 con 0x5592f74266b0 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.759+0000 7f15b37fe700 1 -- 172.21.15.93:0/1881641157 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15c403b070 con 0x5592f74266b0 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.759+0000 7f15b37fe700 1 -- 172.21.15.93:0/1881641157 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4042bd0 con 0x5592f74266b0 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 -- 172.21.15.93:0/1881641157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 msgr2=0x5592f75d2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15c40317e0 tx=0x7f15c4034000 comp rx=0 tx=0).stop 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 -- 172.21.15.93:0/1881641157 shutdown_connections 2023-07-21T22:35:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d2de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 --2- 172.21.15.93:0/1881641157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 -- 172.21.15.93:0/1881641157 >> 172.21.15.93:0/1881641157 conn(0x5592f75bfae0 msgr2=0x5592f75c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 -- 172.21.15.93:0/1881641157 shutdown_connections 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.760+0000 7f15db672b80 1 -- 172.21.15.93:0/1881641157 wait complete. 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 -- start start 2023-07-21T22:35:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 Processor -- start 2023-07-21T22:35:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f74bba60 con 0x5592f75d3320 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f74bb760 con 0x5592f75d00c0 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.761+0000 7f15db672b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f739f8c0 con 0x5592f74266b0 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15db672b80 10 client.0 _ll_register_callbacks cb 0x5592f74b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36404/0 (socket says 172.21.15.93:36404) 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 -- 172.21.15.93:0/2960867908 learned_addr learned my addr 172.21.15.93:0/2960867908 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 -- 172.21.15.93:0/2960867908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 msgr2=0x5592f75d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 -- 172.21.15.93:0/2960867908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 msgr2=0x5592f75d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 -- 172.21.15.93:0/2960867908 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c810c170 con 0x5592f74266b0 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15cd289700 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15c40048a0 tx=0x7f15c4042440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6469accf19fee59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.762+0000 7f15b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4036880 con 0x5592f74266b0 2023-07-21T22:35:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f15c40369e0 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592f7475490 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c4042550 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15db672b80 10 client.26905 fetch_fsmap learned FSMap version 84 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x7f15c402f050 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15c4061080 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15c405b030 con 0x5592f74266b0 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15db672b80 10 client.26905 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:35:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.763+0000 7f15db672b80 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.764+0000 7f15db672b80 20 client.26905 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.764+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f75e3a10 0x5592f75e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.764+0000 7f15db672b80 4 client.26905 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.764+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592f7514e50 con 0x5592f75e3a10 2023-07-21T22:35:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.765+0000 7f15cca88700 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f75e3a10 0x5592f75e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.769+0000 7f15cca88700 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f75e3a10 0x5592f75e5e20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c00790836146f76 server_cookie=a973902f9e32b73c in_seq=0 out_seq=0 2023-07-21T22:35:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.769+0000 7f15b0ff9700 10 client.26905 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15b0ff9700 1 -- 172.21.15.93:0/2960867908 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592f7514e50 con 0x5592f75e3a10 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15b0ff9700 10 client.26905 handle_command_reply: tid=0 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 client.26905 shutdown 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 msgr2=0x5592f75d84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15c40048a0 tx=0x7f15c4042440 comp rx=0 tx=0).stop 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 shutdown_connections 2023-07-21T22:35:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f75e3a10 0x5592f75e5e20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15b0ff9700 0 client.26905 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d84a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 --2- 172.21.15.93:0/2960867908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.770+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 >> 172.21.15.93:0/2960867908 conn(0x5592f75bfae0 msgr2=0x5592f75d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.771+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 shutdown_connections 2023-07-21T22:35:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.771+0000 7f15db672b80 1 -- 172.21.15.93:0/2960867908 wait complete. 2023-07-21T22:35:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.771+0000 7f15db672b80 20 client.26905 trim_cache size 0 max 16384 2023-07-21T22:35:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.789+0000 7f15db672b80 1 Processor -- start 2023-07-21T22:35:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.789+0000 7f15db672b80 1 -- start start 2023-07-21T22:35:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f75d3990 con 0x5592f75d3320 2023-07-21T22:35:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f75d4220 con 0x5592f75d00c0 2023-07-21T22:35:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15db672b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f75c5aa0 con 0x5592f74266b0 2023-07-21T22:35:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36410/0 (socket says 172.21.15.93:36410) 2023-07-21T22:35:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 -- 172.21.15.93:0/621944461 learned_addr learned my addr 172.21.15.93:0/621944461 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cca88700 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cda8a700 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 -- 172.21.15.93:0/621944461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 msgr2=0x5592f75dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 -- 172.21.15.93:0/621944461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 msgr2=0x5592f75ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 -- 172.21.15.93:0/621944461 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f7332c00 con 0x5592f74266b0 2023-07-21T22:35:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.790+0000 7f15cd289700 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15c4055b70 tx=0x7f15c4006b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32cb76d0ccc8113 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15b3fff700 1 -- 172.21.15.93:0/621944461 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4055430 con 0x5592f74266b0 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15b3fff700 1 -- 172.21.15.93:0/621944461 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592f7475490 con 0x5592f74266b0 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15b3fff700 1 -- 172.21.15.93:0/621944461 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4036cb0 con 0x5592f74266b0 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 -- 172.21.15.93:0/621944461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 msgr2=0x5592f75d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15c4055b70 tx=0x7f15c4006b40 comp rx=0 tx=0).stop 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 -- 172.21.15.93:0/621944461 shutdown_connections 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f75d3320 0x5592f75ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f74266b0 0x5592f75d3ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 --2- 172.21.15.93:0/621944461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.791+0000 7f15db672b80 1 -- 172.21.15.93:0/621944461 >> 172.21.15.93:0/621944461 conn(0x5592f75bfae0 msgr2=0x5592f760eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.792+0000 7f15db672b80 1 -- 172.21.15.93:0/621944461 shutdown_connections 2023-07-21T22:35:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.792+0000 7f15db672b80 1 -- 172.21.15.93:0/621944461 wait complete. 2023-07-21T22:35:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.792+0000 7f15db672b80 1 -- start start 2023-07-21T22:35:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.792+0000 7f15db672b80 1 Processor -- start 2023-07-21T22:35:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.792+0000 7f15db672b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 0x5592f75db710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f75dbe50 con 0x5592f74266b0 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f7658ce0 con 0x5592f75d00c0 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f7658e50 con 0x5592f75d3320 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15db672b80 10 client.0 _ll_register_callbacks cb 0x7f15b8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cda8a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 0x5592f75db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cd289700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60500/0 (socket says 172.21.15.93:60500) 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cd289700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49336/0 (socket says 172.21.15.93:49336) 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cda8a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 0x5592f75db710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36420/0 (socket says 172.21.15.93:36420) 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 -- 172.21.15.93:0/1248546907 learned_addr learned my addr 172.21.15.93:0/1248546907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 -- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 msgr2=0x5592f75db710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 0x5592f75db710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 -- 172.21.15.93:0/1248546907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 msgr2=0x5592f75dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 -- 172.21.15.93:0/1248546907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c402f050 con 0x5592f75d00c0 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cd289700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15cca88700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15b8099f40 tx=0x7f15b8096370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5d9867180e76f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.793+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b809a210 con 0x5592f75d00c0 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f15b80b8310 con 0x5592f75d00c0 2023-07-21T22:35:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15c403b070 con 0x5592f75d00c0 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15b809a370 con 0x5592f75d00c0 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 10 client.27020 fetch_fsmap learned FSMap version 84 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x5592f73701f0 con 0x5592f75d00c0 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15c403b070 con 0x5592f75d00c0 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15b80a2000 con 0x5592f75d00c0 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 10 client.27020 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:35:46.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 20 client.27020 populate_metadata read hostname 'smithi093' 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f765a520 0x5592f765c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 4 client.27020 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592f74d6ff0 con 0x5592f765a520 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.794+0000 7f15cd289700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f765a520 0x5592f765c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.795+0000 7f15b0ff9700 10 client.27020 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:46.795+0000 7f15cd289700 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f765a520 0x5592f765c900 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb399a48789d75e0 server_cookie=6cb229d696ea933e in_seq=0 out_seq=0 2023-07-21T22:35:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15b0ff9700 1 -- 172.21.15.93:0/1248546907 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592f74d6ff0 con 0x5592f765a520 2023-07-21T22:35:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15b0ff9700 10 client.27020 handle_command_reply: tid=0 2023-07-21T22:35:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 client.27020 shutdown 2023-07-21T22:35:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 msgr2=0x5592f75dafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15b8099f40 tx=0x7f15b8096370 comp rx=0 tx=0).stop 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 shutdown_connections 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5592f765a520 0x5592f765c900 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5592f75d3320 0x5592f75db710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5592f74266b0 0x5592f75dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 --2- 172.21.15.93:0/1248546907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5592f75d00c0 0x5592f75dafd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 >> 172.21.15.93:0/1248546907 conn(0x5592f75bfae0 msgr2=0x5592f75ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15b0ff9700 0 client.27020 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 shutdown_connections 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 1 -- 172.21.15.93:0/1248546907 wait complete. 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.067+0000 7f15db672b80 20 client.27020 trim_cache size 0 max 16384 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:35:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:35:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:35:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:35:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.068+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 msgr2=0x7f15b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.068+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 0x7f15b40a3770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15c80ffb70 tx=0x7f15c4008040 comp rx=0 tx=0).stop 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.068+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 msgr2=0x7f15c819b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.068+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f15b800e9a0 tx=0x7f15b800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 shutdown_connections 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15c8104a40 0x7f15c8194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c80fea30 0x7f15c819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f15b40a12c0 0x7f15b40a3770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 --2- 172.21.15.93:0/2810415077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15c80ff3e0 0x7f15c819b0f0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 >> 172.21.15.93:0/2810415077 conn(0x7f15c80fa4f0 msgr2=0x7f15c81058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 shutdown_connections 2023-07-21T22:35:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:35:47.069+0000 7f15cf4ed700 1 -- 172.21.15.93:0/2810415077 wait complete. 2023-07-21T22:35:47.528 INFO:tasks.workunit.client.0.smithi093.stdout: 10 237658 32.69 MB/sec execute 263 sec latency 174.263 ms 2023-07-21T22:35:47.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:47 smithi134 ceph-mon[97873]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 802 op/s 2023-07-21T22:35:47.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:47 smithi093 ceph-mon[96579]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 802 op/s 2023-07-21T22:35:48.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:47 smithi194 ceph-mon[98288]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 802 op/s 2023-07-21T22:35:48.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 238266 32.67 MB/sec execute 264 sec latency 98.279 ms 2023-07-21T22:35:49.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 238853 32.66 MB/sec execute 265 sec latency 174.792 ms 2023-07-21T22:35:49.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:49 smithi134 ceph-mon[97873]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:49.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:49 smithi093 ceph-mon[96579]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:50.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:49 smithi194 ceph-mon[98288]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-07-21T22:35:50.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 239564 32.72 MB/sec execute 266 sec latency 72.294 ms 2023-07-21T22:35:51.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 240111 32.70 MB/sec execute 267 sec latency 62.776 ms 2023-07-21T22:35:51.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:51 smithi134 ceph-mon[97873]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2023-07-21T22:35:51.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:51 smithi093 ceph-mon[96579]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2023-07-21T22:35:52.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:51 smithi194 ceph-mon[98288]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.6 MiB/s wr, 623 op/s 2023-07-21T22:35:52.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 240663 32.68 MB/sec execute 268 sec latency 98.773 ms 2023-07-21T22:35:53.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 241182 32.68 MB/sec execute 269 sec latency 99.021 ms 2023-07-21T22:35:53.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:53 smithi134 ceph-mon[97873]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.3 MiB/s wr, 616 op/s 2023-07-21T22:35:53.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:53 smithi093 ceph-mon[96579]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.3 MiB/s wr, 616 op/s 2023-07-21T22:35:54.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:53 smithi194 ceph-mon[98288]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.3 MiB/s wr, 616 op/s 2023-07-21T22:35:54.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 241788 32.64 MB/sec execute 270 sec latency 109.193 ms 2023-07-21T22:35:55.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 242517 32.67 MB/sec execute 271 sec latency 72.525 ms 2023-07-21T22:35:55.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:55 smithi134 ceph-mon[97873]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.4 MiB/s wr, 689 op/s 2023-07-21T22:35:55.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:55 smithi093 ceph-mon[96579]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.4 MiB/s wr, 689 op/s 2023-07-21T22:35:56.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:55 smithi194 ceph-mon[98288]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.4 MiB/s wr, 689 op/s 2023-07-21T22:35:56.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 243229 32.75 MB/sec execute 272 sec latency 57.461 ms 2023-07-21T22:35:56.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:56 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:56.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:56 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:57.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:56 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:57.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 243731 32.71 MB/sec execute 273 sec latency 144.017 ms 2023-07-21T22:35:57.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:57 smithi134 ceph-mon[97873]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:35:57.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:57 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:57.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:57 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:57.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:57 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:35:57.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:57 smithi093 ceph-mon[96579]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:35:57.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:57 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:57.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:57 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:57.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:57 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:35:58.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:57 smithi194 ceph-mon[98288]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:35:58.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:57 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:58.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:57 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:58.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:57 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:35:58.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 244330 32.69 MB/sec execute 274 sec latency 67.319 ms 2023-07-21T22:35:59.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 244973 32.69 MB/sec execute 275 sec latency 74.505 ms 2023-07-21T22:35:59.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:35:59 smithi134 ceph-mon[97873]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2023-07-21T22:35:59.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:35:59 smithi093 ceph-mon[96579]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2023-07-21T22:36:00.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:35:59 smithi194 ceph-mon[98288]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2023-07-21T22:36:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 -- 172.21.15.93:0/2569772283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a780ff450 msgr2=0x7f5a7810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/2569772283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a780ff450 0x7f5a7810b8e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b0a0 tx=0x7f5a7000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 -- 172.21.15.93:0/2569772283 shutdown_connections 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/2569772283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a780ff450 0x7f5a7810b8e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/2569772283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a78104ab0 0x7f5a78104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/2569772283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 0x7f5a780fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.405+0000 7f5a7f13c700 1 -- 172.21.15.93:0/2569772283 >> 172.21.15.93:0/2569772283 conn(0x7f5a78075ba0 msgr2=0x7f5a78075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.406+0000 7f5a7f13c700 1 -- 172.21.15.93:0/2569772283 shutdown_connections 2023-07-21T22:36:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.406+0000 7f5a7f13c700 1 -- 172.21.15.93:0/2569772283 wait complete. 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.406+0000 7f5a7f13c700 1 Processor -- start 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 -- start start 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 0x7f5a7810c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 0x7f5a7810c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 0x7f5a78070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78070e60 con 0x7f5a78104ab0 2023-07-21T22:36:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78070fa0 con 0x7f5a780feaa0 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7f13c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78071160 con 0x7f5a780ff450 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 0x7f5a7810c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7d6d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 0x7f5a78070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a7ced8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 0x7f5a7810c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 0x7f5a7810c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36432/0 (socket says 172.21.15.93:36432) 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.407+0000 7f5a77fff700 1 -- 172.21.15.93:0/613335506 learned_addr learned my addr 172.21.15.93:0/613335506 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/613335506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 msgr2=0x7f5a7810c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 0x7f5a7810c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/613335506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 msgr2=0x7f5a7810c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 0x7f5a7810c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/613335506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64009520 con 0x7f5a78104ab0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 0x7f5a78070920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b0a0 tx=0x7f5a70007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a75ffb700 1 -- 172.21.15.93:0/613335506 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000e040 con 0x7f5a78104ab0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a75ffb700 1 -- 172.21.15.93:0/613335506 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a7000be40 con 0x7f5a78104ab0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a70009d00 con 0x7f5a78104ab0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.408+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78071670 con 0x7f5a78104ab0 2023-07-21T22:36:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.409+0000 7f5a75ffb700 1 -- 172.21.15.93:0/613335506 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700052a0 con 0x7f5a78104ab0 2023-07-21T22:36:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.411+0000 7f5a75ffb700 1 -- 172.21.15.93:0/613335506 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5a70004c50 con 0x7f5a78104ab0 2023-07-21T22:36:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.412+0000 7f5a75ffb700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 0x7f5a680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.412+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 0x7f5a680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.412+0000 7f5a75ffb700 1 -- 172.21.15.93:0/613335506 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5a700ce0d0 con 0x7f5a78104ab0 2023-07-21T22:36:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 0x7f5a680a3870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a780ffbe0 tx=0x7f5a6c00a3b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 Processor -- start 2023-07-21T22:36:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 -- start start 2023-07-21T22:36:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc9352cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc934d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc9252c50 con 0x558bc9353660 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc92a59e0 con 0x558bc91a96b0 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.413+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc929ee70 con 0x558bc9353210 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc934d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36440/0 (socket says 172.21.15.93:36440) 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a77fff700 1 -- 172.21.15.93:0/3747723825 learned_addr learned my addr 172.21.15.93:0/3747723825 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc9352cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3747723825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 msgr2=0x558bc934cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3747723825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 msgr2=0x558bc9352cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc9352cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3747723825 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bc90ccd80 con 0x558bc9353660 2023-07-21T22:36:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc934d4a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cb7c0 tx=0x7f5a7009ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402cc5db76318aba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a62ffd700 1 -- 172.21.15.93:0/3747723825 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7009b1d0 con 0x558bc9353660 2023-07-21T22:36:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a62ffd700 1 -- 172.21.15.93:0/3747723825 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a7009f200 con 0x558bc9353660 2023-07-21T22:36:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.414+0000 7f5a62ffd700 1 -- 172.21.15.93:0/3747723825 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7009a6c0 con 0x558bc9353660 2023-07-21T22:36:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.415+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3747723825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 msgr2=0x558bc934d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.415+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc934d4a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5a700cb7c0 tx=0x7f5a7009ae30 comp rx=0 tx=0).stop 2023-07-21T22:36:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3747723825 shutdown_connections 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc934d4a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc9352cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3747723825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3747723825 >> 172.21.15.93:0/3747723825 conn(0x558bc9342ae0 msgr2=0x558bc93480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3747723825 shutdown_connections 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3747723825 wait complete. 2023-07-21T22:36:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 -- start start 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.416+0000 7f5a8b2c1b80 1 Processor -- start 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc935c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc935c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc923ea60 con 0x558bc9353660 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc923e760 con 0x558bc91a96b0 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc91228c0 con 0x558bc9353210 2023-07-21T22:36:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a8b2c1b80 10 client.0 _ll_register_callbacks cb 0x558bc923c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49364/0 (socket says 172.21.15.93:49364) 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3364849259 learned_addr learned my addr 172.21.15.93:0/3364849259 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a77fff700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc935c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 msgr2=0x558bc935c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc935c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3364849259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 msgr2=0x558bc935c370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc935c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.417+0000 7f5a7d6d9700 1 -- 172.21.15.93:0/3364849259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78071670 con 0x558bc9353660 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5a70099520 tx=0x7f5a7009a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b401b710eacf56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a57fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700a3590 con 0x558bc9353660 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bc91f8490 con 0x558bc9353660 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5a700aee40 con 0x558bc9353660 2023-07-21T22:36:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a7009b1d0 con 0x558bc9353660 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a8b2c1b80 10 client.17682 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x558bc90b5c00 con 0x558bc9353660 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.418+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558bc91f8490 con 0x558bc9353660 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a7009a130 con 0x558bc9353660 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 10 client.17682 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 20 client.17682 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc9366420 0x558bc9368830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 4 client.17682 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bc9297e50 con 0x558bc9366420 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc9366420 0x558bc9368830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a57fff700 10 client.17682 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:00.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.419+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc9366420 0x558bc9368830 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b22af9738235d server_cookie=3363ddd275618e5c in_seq=0 out_seq=0 2023-07-21T22:36:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a57fff700 1 -- 172.21.15.93:0/3364849259 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558bc9297e50 con 0x558bc9366420 2023-07-21T22:36:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a57fff700 10 client.17682 handle_command_reply: tid=0 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 client.17682 shutdown 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 msgr2=0x558bc93565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5a70099520 tx=0x7f5a7009a9a0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 shutdown_connections 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc9366420 0x558bc9368830 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353660 0x558bc93565c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc91a96b0 0x558bc935c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3364849259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353210 0x558bc935c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 >> 172.21.15.93:0/3364849259 conn(0x558bc9342ae0 msgr2=0x558bc93473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.421+0000 7f5a57fff700 0 client.17682 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.424+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 shutdown_connections 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.424+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3364849259 wait complete. 2023-07-21T22:36:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.424+0000 7f5a8b2c1b80 20 client.17682 trim_cache size 0 max 16384 2023-07-21T22:36:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.440+0000 7f5a8b2c1b80 1 Processor -- start 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.440+0000 7f5a8b2c1b80 1 -- start start 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353660 0x558bc934d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc9348cb0 con 0x558bc91a96b0 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc9344c00 con 0x558bc9353210 2023-07-21T22:36:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc93576d0 con 0x558bc9353660 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49372/0 (socket says 172.21.15.93:49372) 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 -- 172.21.15.93:0/4141122828 learned_addr learned my addr 172.21.15.93:0/4141122828 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353660 0x558bc934d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a77fff700 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 -- 172.21.15.93:0/4141122828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353660 msgr2=0x558bc934d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353660 0x558bc934d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 -- 172.21.15.93:0/4141122828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353210 msgr2=0x558bc934cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 -- 172.21.15.93:0/4141122828 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bc90f31f0 con 0x558bc91a96b0 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x558bc93690d0 tx=0x7f5a6c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a3e0d565700002 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.441+0000 7f5a637fe700 1 -- 172.21.15.93:0/4141122828 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c022070 con 0x558bc91a96b0 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a637fe700 1 -- 172.21.15.93:0/4141122828 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a6c014070 con 0x558bc91a96b0 2023-07-21T22:36:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a637fe700 1 -- 172.21.15.93:0/4141122828 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c01d970 con 0x558bc91a96b0 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/4141122828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 msgr2=0x558bc9354160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x558bc93690d0 tx=0x7f5a6c012000 comp rx=0 tx=0).stop 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/4141122828 shutdown_connections 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc9353660 0x558bc934d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc91a96b0 0x558bc9354160 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/4141122828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353210 0x558bc934cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/4141122828 >> 172.21.15.93:0/4141122828 conn(0x558bc9342ae0 msgr2=0x558bc9353a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/4141122828 shutdown_connections 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.442+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/4141122828 wait complete. 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 -- start start 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 Processor -- start 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc91a96b0 0x558bc93b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353660 0x558bc935f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc935f680 con 0x558bc9353210 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc935f7f0 con 0x558bc9353660 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bc935f9b0 con 0x558bc91a96b0 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a8b2c1b80 10 client.0 _ll_register_callbacks cb 0x7f5a6c00a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49380/0 (socket says 172.21.15.93:49380) 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a7ced8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc91a96b0 0x558bc93b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 -- 172.21.15.93:0/3602888895 learned_addr learned my addr 172.21.15.93:0/3602888895 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a7d6d9700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353660 0x558bc935f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 -- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc91a96b0 msgr2=0x558bc93b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc91a96b0 0x558bc93b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 -- 172.21.15.93:0/3602888895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353660 msgr2=0x558bc935f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353660 0x558bc935f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.443+0000 7f5a77fff700 1 -- 172.21.15.93:0/3602888895 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c00fd60 con 0x558bc9353210 2023-07-21T22:36:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a77fff700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013af0 tx=0x7f5a64013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc83ba4fe66ea047 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64021070 con 0x558bc9353210 2023-07-21T22:36:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5a64011c40 con 0x558bc9353210 2023-07-21T22:36:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a7009f200 con 0x558bc9353210 2023-07-21T22:36:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a64011e00 con 0x558bc9353210 2023-07-21T22:36:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 10 client.17688 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x558bc90cb250 con 0x558bc9353210 2023-07-21T22:36:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a64029020 con 0x558bc9353210 2023-07-21T22:36:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a64005450 con 0x558bc9353210 2023-07-21T22:36:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 10 client.17688 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 20 client.17688 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc93bb560 0x558bc93bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 4 client.17688 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.444+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558bc9259ff0 con 0x558bc93bb560 2023-07-21T22:36:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.445+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc93bb560 0x558bc93bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.445+0000 7f5a57fff700 10 client.17688 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.445+0000 7f5a7ced8700 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc93bb560 0x558bc93bd970 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a803f8c699763669 server_cookie=f1b646cba4ee6365 in_seq=0 out_seq=0 2023-07-21T22:36:00.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 245723 32.71 MB/sec execute 276 sec latency 85.130 ms 2023-07-21T22:36:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a57fff700 1 -- 172.21.15.93:0/3602888895 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558bc9259ff0 con 0x558bc93bb560 2023-07-21T22:36:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a57fff700 10 client.17688 handle_command_reply: tid=0 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 client.17688 shutdown 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 msgr2=0x558bc93b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5a64013af0 tx=0x7f5a64013e00 comp rx=0 tx=0).stop 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 shutdown_connections 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x558bc93bb560 0x558bc93bd970 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558bc9353660 0x558bc935f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a57fff700 0 client.17688 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.632+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558bc91a96b0 0x558bc93b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.643+0000 7f5a8b2c1b80 1 --2- 172.21.15.93:0/3602888895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558bc9353210 0x558bc93b3b20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.643+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 >> 172.21.15.93:0/3602888895 conn(0x558bc9342ae0 msgr2=0x558bc9354d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.643+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 shutdown_connections 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.643+0000 7f5a8b2c1b80 1 -- 172.21.15.93:0/3602888895 wait complete. 2023-07-21T22:36:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.643+0000 7f5a8b2c1b80 20 client.17688 trim_cache size 0 max 16384 2023-07-21T22:36:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:36:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:36:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:36:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 msgr2=0x7f5a680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 0x7f5a680a3870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a780ffbe0 tx=0x7f5a6c00a3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 msgr2=0x7f5a78070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 0x7f5a78070920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b0a0 tx=0x7f5a70007990 comp rx=0 tx=0).stop 2023-07-21T22:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 shutdown_connections 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5a680a13c0 0x7f5a680a3870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a78104ab0 0x7f5a78070920 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a780feaa0 0x7f5a7810c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 --2- 172.21.15.93:0/613335506 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a780ff450 0x7f5a7810c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 >> 172.21.15.93:0/613335506 conn(0x7f5a78075ba0 msgr2=0x7f5a78105920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 shutdown_connections 2023-07-21T22:36:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:00.644+0000 7f5a7f13c700 1 -- 172.21.15.93:0/613335506 wait complete. 2023-07-21T22:36:00.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:00 smithi093 ceph-mon[96579]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 428 op/s 2023-07-21T22:36:01.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:00 smithi194 ceph-mon[98288]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 428 op/s 2023-07-21T22:36:01.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:00 smithi134 ceph-mon[97873]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.4 MiB/s wr, 428 op/s 2023-07-21T22:36:01.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 245723 32.59 MB/sec execute 277 sec latency 1085.205 ms 2023-07-21T22:36:02.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 245723 32.47 MB/sec execute 278 sec latency 2085.303 ms 2023-07-21T22:36:03.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 245723 32.36 MB/sec execute 279 sec latency 3085.406 ms 2023-07-21T22:36:03.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:03 smithi093 ceph-mon[96579]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-07-21T22:36:03.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:03 smithi194 ceph-mon[98288]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-07-21T22:36:03.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:03 smithi134 ceph-mon[97873]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.2 MiB/s wr, 429 op/s 2023-07-21T22:36:04.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 245852 32.26 MB/sec execute 280 sec latency 3842.433 ms 2023-07-21T22:36:05.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 246149 32.19 MB/sec execute 281 sec latency 146.963 ms 2023-07-21T22:36:05.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:05 smithi093 ceph-mon[96579]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 523 op/s 2023-07-21T22:36:05.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:05 smithi194 ceph-mon[98288]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 523 op/s 2023-07-21T22:36:05.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:05 smithi134 ceph-mon[97873]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 523 op/s 2023-07-21T22:36:06.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 246332 32.13 MB/sec execute 282 sec latency 188.858 ms 2023-07-21T22:36:07.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 246842 32.15 MB/sec execute 283 sec latency 144.819 ms 2023-07-21T22:36:07.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:07 smithi093 ceph-mon[96579]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 374 op/s 2023-07-21T22:36:07.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:07 smithi194 ceph-mon[98288]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 374 op/s 2023-07-21T22:36:07.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:07 smithi134 ceph-mon[97873]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 374 op/s 2023-07-21T22:36:08.529 INFO:tasks.workunit.client.0.smithi093.stdout: 10 247490 32.15 MB/sec execute 284 sec latency 97.921 ms 2023-07-21T22:36:09.531 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248021 32.13 MB/sec execute 285 sec latency 144.466 ms 2023-07-21T22:36:09.697 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:09 smithi093 ceph-mon[96579]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2023-07-21T22:36:09.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:09 smithi194 ceph-mon[98288]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2023-07-21T22:36:09.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:09 smithi134 ceph-mon[97873]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 7.2 MiB/s wr, 641 op/s 2023-07-21T22:36:10.531 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248205 32.04 MB/sec execute 286 sec latency 256.427 ms 2023-07-21T22:36:10.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:10 smithi093 ceph-mon[96579]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T22:36:10.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:10 smithi194 ceph-mon[98288]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T22:36:10.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:10 smithi134 ceph-mon[97873]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T22:36:11.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248505 31.97 MB/sec execute 287 sec latency 203.557 ms 2023-07-21T22:36:11.849 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:11 smithi194 ceph-mon[98288]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T22:36:11.898 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:11 smithi134 ceph-mon[97873]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T22:36:11.948 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:11 smithi093 ceph-mon[96579]: pgmap v629: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T22:36:12.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248526 31.87 MB/sec execute 288 sec latency 986.163 ms 2023-07-21T22:36:13.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248721 31.80 MB/sec execute 289 sec latency 1196.286 ms 2023-07-21T22:36:13.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:13 smithi194 ceph-mon[98288]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 566 op/s 2023-07-21T22:36:13.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:13 smithi134 ceph-mon[97873]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 566 op/s 2023-07-21T22:36:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.867+0000 7f8d3b267700 1 -- 172.21.15.93:0/2397405695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3410e2a0 msgr2=0x7f8d3410e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.867+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2397405695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3410e2a0 0x7f8d3410e670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009990 tx=0x7f8d24009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 -- 172.21.15.93:0/2397405695 shutdown_connections 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2397405695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3406d3a0 0x7f8d3411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2397405695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3410e2a0 0x7f8d3410e670 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2397405695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d3406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 -- 172.21.15.93:0/2397405695 >> 172.21.15.93:0/2397405695 conn(0x7f8d3406bd70 msgr2=0x7f8d34070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 -- 172.21.15.93:0/2397405695 shutdown_connections 2023-07-21T22:36:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.870+0000 7f8d3b267700 1 -- 172.21.15.93:0/2397405695 wait complete. 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 Processor -- start 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 -- start start 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d341aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 0x7f8d341ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 0x7f8d341a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341a50c0 con 0x7f8d3406c910 2023-07-21T22:36:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341a5230 con 0x7f8d3410e2a0 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d3b267700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341a53f0 con 0x7f8d3406d3a0 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d38802700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 0x7f8d341ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d39804700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 0x7f8d341a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d38802700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 0x7f8d341ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36474/0 (socket says 172.21.15.93:36474) 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.871+0000 7f8d38802700 1 -- 172.21.15.93:0/2295768749 learned_addr learned my addr 172.21.15.93:0/2295768749 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39003700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d341aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 -- 172.21.15.93:0/2295768749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 msgr2=0x7f8d341ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 0x7f8d341ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 -- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 msgr2=0x7f8d341aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d341aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 -- 172.21.15.93:0/2295768749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240095f0 con 0x7f8d3410e2a0 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39003700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d341aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.872+0000 7f8d39804700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 0x7f8d341a48f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00be20 tx=0x7f8d2c00be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.874+0000 7f8d2a7fc700 1 -- 172.21.15.93:0/2295768749 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00cb90 con 0x7f8d3410e2a0 2023-07-21T22:36:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.874+0000 7f8d2a7fc700 1 -- 172.21.15.93:0/2295768749 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d2c00ccf0 con 0x7f8d3410e2a0 2023-07-21T22:36:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.874+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d341a56d0 con 0x7f8d3410e2a0 2023-07-21T22:36:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.874+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d341b1aa0 con 0x7f8d3410e2a0 2023-07-21T22:36:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.874+0000 7f8d2a7fc700 1 -- 172.21.15.93:0/2295768749 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c012670 con 0x7f8d3410e2a0 2023-07-21T22:36:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.878+0000 7f8d473ecb80 1 Processor -- start 2023-07-21T22:36:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 -- start start 2023-07-21T22:36:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4210 0x55f8891cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d4660 0x55f8891ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8890d3c50 con 0x55f88902a6b0 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8891269e0 con 0x55f8891d4210 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88911fe70 con 0x55f8891d4660 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d38802700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d38802700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49398/0 (socket says 172.21.15.93:49398) 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d38802700 1 -- 172.21.15.93:0/1568383906 learned_addr learned my addr 172.21.15.93:0/1568383906 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.879+0000 7f8d38802700 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d4660 0x55f8891ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 -- 172.21.15.93:0/1568383906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d4660 msgr2=0x55f8891ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d4660 0x55f8891ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 -- 172.21.15.93:0/1568383906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4210 msgr2=0x55f8891cdf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:36:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4210 0x55f8891cdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 -- 172.21.15.93:0/1568383906 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f888f4dd80 con 0x55f88902a6b0 2023-07-21T22:36:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.880+0000 7f8d38802700 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fd0 tx=0x7f8d30009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ab44bf3db358e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.881+0000 7f8d2a7fc700 1 -- 172.21.15.93:0/2295768749 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8d2c0127d0 con 0x7f8d3410e2a0 2023-07-21T22:36:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.882+0000 7f8d1f7fe700 1 -- 172.21.15.93:0/1568383906 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30013170 con 0x55f88902a6b0 2023-07-21T22:36:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.882+0000 7f8d1f7fe700 1 -- 172.21.15.93:0/1568383906 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d300132d0 con 0x55f88902a6b0 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.882+0000 7f8d1f7fe700 1 -- 172.21.15.93:0/1568383906 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30010430 con 0x55f88902a6b0 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.882+0000 7f8d2a7fc700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 0x7f8d200a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.882+0000 7f8d2a7fc700 1 -- 172.21.15.93:0/2295768749 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8d2c006e40 con 0x7f8d3410e2a0 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 -- 172.21.15.93:0/1568383906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 msgr2=0x55f8891d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009fd0 tx=0x7f8d30009620 comp rx=0 tx=0).stop 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 -- 172.21.15.93:0/1568383906 shutdown_connections 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d4660 0x55f8891ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f88902a6b0 0x55f8891d3cd0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/1568383906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4210 0x55f8891cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.883+0000 7f8d473ecb80 1 -- 172.21.15.93:0/1568383906 >> 172.21.15.93:0/1568383906 conn(0x55f8891c3ae0 msgr2=0x55f8891c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.885+0000 7f8d39003700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 0x7f8d200a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.885+0000 7f8d473ecb80 1 -- 172.21.15.93:0/1568383906 shutdown_connections 2023-07-21T22:36:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.885+0000 7f8d473ecb80 1 -- 172.21.15.93:0/1568383906 wait complete. 2023-07-21T22:36:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.886+0000 7f8d473ecb80 1 -- start start 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.886+0000 7f8d473ecb80 1 Processor -- start 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.886+0000 7f8d39003700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 0x7f8d200a38c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55f8891cab80 tx=0x7f8d2400b260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f88902a6b0 0x55f8891d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d8440 0x55f8891f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8890bfa60 con 0x55f8891d4660 2023-07-21T22:36:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8890bf760 con 0x55f88902a6b0 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f888fa38c0 con 0x55f8891d8440 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d473ecb80 10 client.0 _ll_register_callbacks cb 0x55f8890bd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d38802700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d38802700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49408/0 (socket says 172.21.15.93:49408) 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.887+0000 7f8d38802700 1 -- 172.21.15.93:0/182542067 learned_addr learned my addr 172.21.15.93:0/182542067 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d39003700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f88902a6b0 0x55f8891d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d39804700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d8440 0x55f8891f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 -- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d8440 msgr2=0x55f8891f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d8440 0x55f8891f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 -- 172.21.15.93:0/182542067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f88902a6b0 msgr2=0x55f8891d79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f88902a6b0 0x55f8891d79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 -- 172.21.15.93:0/182542067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3001b040 con 0x55f8891d4660 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d38802700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8d300096b0 tx=0x7f8d30004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95be1e8b92ca5bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:13.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30013170 con 0x55f8891d4660 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8d30004dc0 con 0x55f8891d4660 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.888+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f889079490 con 0x55f8891d4660 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d300051a0 con 0x55f8891d4660 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 10 client.17700 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55f888f36c00 con 0x55f8891d4660 2023-07-21T22:36:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f889079490 con 0x55f8891d4660 2023-07-21T22:36:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d30027020 con 0x55f8891d4660 2023-07-21T22:36:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 10 client.17700 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 10 client.17700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 20 client.17700 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891f8e40 0x55f8891fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 4 client.17700 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.889+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f889118e50 con 0x55f8891f8e40 2023-07-21T22:36:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.892+0000 7f8d39003700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891f8e40 0x55f8891fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:13.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:13 smithi093 ceph-mon[96579]: pgmap v630: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 566 op/s 2023-07-21T22:36:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.911+0000 7f8d39003700 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891f8e40 0x55f8891fb250 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8b3392b1a285fe server_cookie=5df7150e1933e8ad in_seq=0 out_seq=0 2023-07-21T22:36:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.912+0000 7f8d1cff9700 10 client.17700 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.921+0000 7f8d1cff9700 1 -- 172.21.15.93:0/182542067 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f889118e50 con 0x55f8891f8e40 2023-07-21T22:36:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.921+0000 7f8d1cff9700 10 client.17700 handle_command_reply: tid=0 2023-07-21T22:36:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.921+0000 7f8d473ecb80 1 client.17700 shutdown 2023-07-21T22:36:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 msgr2=0x55f8891d7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8d300096b0 tx=0x7f8d30004960 comp rx=0 tx=0).stop 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 shutdown_connections 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891f8e40 0x55f8891fb250 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f8891d8440 0x55f8891f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f88902a6b0 0x55f8891d79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d1cff9700 0 client.17700 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/182542067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d4660 0x55f8891d7f00 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.928+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 >> 172.21.15.93:0/182542067 conn(0x55f8891c3ae0 msgr2=0x55f8891c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.935+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 shutdown_connections 2023-07-21T22:36:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.938+0000 7f8d473ecb80 1 -- 172.21.15.93:0/182542067 wait complete. 2023-07-21T22:36:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.938+0000 7f8d473ecb80 20 client.17700 trim_cache size 0 max 16384 2023-07-21T22:36:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 Processor -- start 2023-07-21T22:36:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 -- start start 2023-07-21T22:36:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f8891efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8891d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8891d8970 con 0x55f8891d8440 2023-07-21T22:36:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8891cf160 con 0x55f8891d4660 2023-07-21T22:36:14.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.962+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8891d8f30 con 0x55f88902a6b0 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49416/0 (socket says 172.21.15.93:49416) 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 -- 172.21.15.93:0/2607753537 learned_addr learned my addr 172.21.15.93:0/2607753537 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 -- 172.21.15.93:0/2607753537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 msgr2=0x55f8891efc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f8891efc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 -- 172.21.15.93:0/2607753537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 msgr2=0x55f8891d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8891d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 -- 172.21.15.93:0/2607753537 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f888f741f0 con 0x55f8891d8440 2023-07-21T22:36:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d39804700 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0990f0 tx=0x7f8d2c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d234c28dc47348 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d1ffff700 1 -- 172.21.15.93:0/2607753537 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0a1840 con 0x55f8891d8440 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d1ffff700 1 -- 172.21.15.93:0/2607753537 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d2c097790 con 0x55f8891d8440 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.963+0000 7f8d1ffff700 1 -- 172.21.15.93:0/2607753537 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0988e0 con 0x55f8891d8440 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2607753537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 msgr2=0x55f8891d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0990f0 tx=0x7f8d2c004be0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2607753537 shutdown_connections 2023-07-21T22:36:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f8891d3010 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f8891efc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2607753537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8891d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2607753537 >> 172.21.15.93:0/2607753537 conn(0x55f8891c3ae0 msgr2=0x55f8891c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2607753537 shutdown_connections 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.964+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2607753537 wait complete. 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.965+0000 7f8d473ecb80 1 -- start start 2023-07-21T22:36:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.965+0000 7f8d473ecb80 1 Processor -- start 2023-07-21T22:36:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8892413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f889245d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f889246240 con 0x55f8891d8440 2023-07-21T22:36:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8892463b0 con 0x55f8891d4660 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f889246570 con 0x55f88902a6b0 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d473ecb80 10 client.0 _ll_register_callbacks cb 0x7f8d30013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d39003700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d39003700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36500/0 (socket says 172.21.15.93:36500) 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d39003700 1 -- 172.21.15.93:0/2043648505 learned_addr learned my addr 172.21.15.93:0/2043648505 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.970+0000 7f8d39804700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f889245d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 -- 172.21.15.93:0/2043648505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 msgr2=0x55f8892413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8892413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 -- 172.21.15.93:0/2043648505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 msgr2=0x55f889245d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f889245d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 -- 172.21.15.93:0/2043648505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d341b1aa0 con 0x55f88902a6b0 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.971+0000 7f8d39003700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d24000c00 tx=0x7f8d24009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43688aa5fb534c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.972+0000 7f8d1cff9700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.972+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2402c040 con 0x55f88902a6b0 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.972+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8d240315b0 con 0x55f88902a6b0 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.972+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d300132d0 con 0x55f88902a6b0 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.972+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d24053960 con 0x55f88902a6b0 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.975+0000 7f8d473ecb80 10 client.26941 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.975+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55f888f4c250 con 0x55f88902a6b0 2023-07-21T22:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d300132d0 con 0x55f88902a6b0 2023-07-21T22:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d24004c50 con 0x55f88902a6b0 2023-07-21T22:36:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 10 client.26941 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 10 client.26941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 20 client.26941 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891db0b0 0x55f8891dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 4 client.26941 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f8890daff0 con 0x55f8891db0b0 2023-07-21T22:36:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.981+0000 7f8d38802700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891db0b0 0x55f8891dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.982+0000 7f8d1cff9700 10 client.26941 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:13.982+0000 7f8d38802700 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891db0b0 0x55f8891dd4c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39a6f0431768cb2 server_cookie=ae6396e75f04d57d in_seq=0 out_seq=0 2023-07-21T22:36:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.202+0000 7f8d1cff9700 1 -- 172.21.15.93:0/2043648505 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f8890daff0 con 0x55f8891db0b0 2023-07-21T22:36:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.202+0000 7f8d1cff9700 10 client.26941 handle_command_reply: tid=0 2023-07-21T22:36:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.203+0000 7f8d473ecb80 1 client.26941 shutdown 2023-07-21T22:36:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.205+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 msgr2=0x55f889240ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.205+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d24000c00 tx=0x7f8d24009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 shutdown_connections 2023-07-21T22:36:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f8891db0b0 0x55f8891dd4c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f8891d8440 0x55f889245d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f88902a6b0 0x55f889240ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 --2- 172.21.15.93:0/2043648505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f8891d4660 0x55f8892413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 >> 172.21.15.93:0/2043648505 conn(0x55f8891c3ae0 msgr2=0x55f8891d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.207+0000 7f8d1cff9700 0 client.26941 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.208+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 shutdown_connections 2023-07-21T22:36:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.209+0000 7f8d473ecb80 1 -- 172.21.15.93:0/2043648505 wait complete. 2023-07-21T22:36:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.209+0000 7f8d473ecb80 20 client.26941 trim_cache size 0 max 16384 2023-07-21T22:36:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:36:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:36:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:36:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 msgr2=0x7f8d200a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 0x7f8d200a38c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55f8891cab80 tx=0x7f8d2400b260 comp rx=0 tx=0).stop 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 msgr2=0x7f8d341a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 0x7f8d341a48f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00be20 tx=0x7f8d2c00be50 comp rx=0 tx=0).stop 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 shutdown_connections 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d3410e2a0 0x7f8d341a48f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f8d200a1410 0x7f8d200a38c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8d3406c910 0x7f8d341aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 --2- 172.21.15.93:0/2295768749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d3406d3a0 0x7f8d341ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.214+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 >> 172.21.15.93:0/2295768749 conn(0x7f8d3406bd70 msgr2=0x7f8d340707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.216+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 shutdown_connections 2023-07-21T22:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:14.216+0000 7f8d3b267700 1 -- 172.21.15.93:0/2295768749 wait complete. 2023-07-21T22:36:14.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 248929 31.73 MB/sec execute 290 sec latency 183.975 ms 2023-07-21T22:36:15.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 249574 31.73 MB/sec execute 291 sec latency 136.647 ms 2023-07-21T22:36:15.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:15 smithi194 ceph-mon[98288]: pgmap v631: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:15.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:15 smithi134 ceph-mon[97873]: pgmap v631: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:15.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:15 smithi093 ceph-mon[96579]: pgmap v631: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:16.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 250228 31.79 MB/sec execute 292 sec latency 55.563 ms 2023-07-21T22:36:17.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 250874 31.79 MB/sec execute 293 sec latency 53.548 ms 2023-07-21T22:36:17.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:17 smithi194 ceph-mon[98288]: pgmap v632: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:17.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:17 smithi134 ceph-mon[97873]: pgmap v632: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:17.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:17 smithi093 ceph-mon[96579]: pgmap v632: 97 pgs: 9 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 83 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 20 MiB/s wr, 763 op/s 2023-07-21T22:36:18.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 251375 31.77 MB/sec execute 294 sec latency 82.655 ms 2023-07-21T22:36:19.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 251894 31.75 MB/sec execute 295 sec latency 86.625 ms 2023-07-21T22:36:19.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:19 smithi194 ceph-mon[98288]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 871 op/s 2023-07-21T22:36:19.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:19 smithi134 ceph-mon[97873]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 871 op/s 2023-07-21T22:36:19.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:19 smithi093 ceph-mon[96579]: pgmap v633: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 871 op/s 2023-07-21T22:36:20.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 252543 31.74 MB/sec execute 296 sec latency 1019.282 ms 2023-07-21T22:36:21.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 253287 31.80 MB/sec execute 297 sec latency 2019.310 ms 2023-07-21T22:36:21.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:21 smithi194 ceph-mon[98288]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 780 op/s 2023-07-21T22:36:21.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:21 smithi134 ceph-mon[97873]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 780 op/s 2023-07-21T22:36:21.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:21 smithi093 ceph-mon[96579]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 20 MiB/s wr, 780 op/s 2023-07-21T22:36:22.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 253850 31.79 MB/sec execute 298 sec latency 3019.346 ms 2023-07-21T22:36:23.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 254375 31.78 MB/sec execute 299 sec latency 4019.384 ms 2023-07-21T22:36:23.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:23 smithi093 ceph-mon[96579]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 720 op/s 2023-07-21T22:36:23.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:23 smithi093 ceph-mon[96579]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T22:36:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:23 smithi194 ceph-mon[98288]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 720 op/s 2023-07-21T22:36:23.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:23 smithi194 ceph-mon[98288]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T22:36:23.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:23 smithi134 ceph-mon[97873]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 18 MiB/s wr, 720 op/s 2023-07-21T22:36:23.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:23 smithi134 ceph-mon[97873]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T22:36:24.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 254836 31.75 MB/sec execute 300 sec latency 5019.413 ms 2023-07-21T22:36:25.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 255431 31.73 MB/sec execute 301 sec latency 6019.441 ms 2023-07-21T22:36:25.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:25 smithi093 ceph-mon[96579]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:25.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:25 smithi194 ceph-mon[98288]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:25.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:25 smithi134 ceph-mon[97873]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:26.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 256042 31.75 MB/sec execute 302 sec latency 7019.469 ms 2023-07-21T22:36:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 -- 172.21.15.93:0/174213087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7806ce50 msgr2=0x7f2d7806d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 --2- 172.21.15.93:0/174213087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7806ce50 0x7f2d7806d2c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009990 tx=0x7f2d60009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 -- 172.21.15.93:0/174213087 shutdown_connections 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 --2- 172.21.15.93:0/174213087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d78114c80 0x7f2d78117060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 --2- 172.21.15.93:0/174213087 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d7810a1b0 0x7f2d7806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 --2- 172.21.15.93:0/174213087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7806ce50 0x7f2d7806d2c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.406+0000 7f2d7d311700 1 -- 172.21.15.93:0/174213087 >> 172.21.15.93:0/174213087 conn(0x7f2d7806bd70 msgr2=0x7f2d78070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- 172.21.15.93:0/174213087 shutdown_connections 2023-07-21T22:36:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- 172.21.15.93:0/174213087 wait complete. 2023-07-21T22:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 Processor -- start 2023-07-21T22:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- start start 2023-07-21T22:36:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d7806ce50 0x7f2d781a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 0x7f2d781a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781a0b40 con 0x7f2d78114c80 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781a0cb0 con 0x7f2d7810a1b0 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.407+0000 7f2d7d311700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781a0e70 con 0x7f2d7806ce50 2023-07-21T22:36:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 0x7f2d781a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60598/0 (socket says 172.21.15.93:60598) 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 -- 172.21.15.93:0/35436527 learned_addr learned my addr 172.21.15.93:0/35436527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d7806ce50 0x7f2d781a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 -- 172.21.15.93:0/35436527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d7806ce50 msgr2=0x7f2d781a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d7806ce50 0x7f2d781a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 -- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 msgr2=0x7f2d781a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 0x7f2d781a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 -- 172.21.15.93:0/35436527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d600095f0 con 0x7f2d7810a1b0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d777fe700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 0x7f2d781a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005910 tx=0x7f2d60004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d74ff9700 1 -- 172.21.15.93:0/35436527 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6001d070 con 0x7f2d7810a1b0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d781a10f0 con 0x7f2d7810a1b0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d74ff9700 1 -- 172.21.15.93:0/35436527 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d60022470 con 0x7f2d7810a1b0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d74ff9700 1 -- 172.21.15.93:0/35436527 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60017650 con 0x7f2d7810a1b0 2023-07-21T22:36:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.408+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d781ad660 con 0x7f2d7810a1b0 2023-07-21T22:36:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.411+0000 7f2d74ff9700 1 -- 172.21.15.93:0/35436527 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2d6000fad0 con 0x7f2d7810a1b0 2023-07-21T22:36:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.412+0000 7f2d74ff9700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 0x7f2d640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.412+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 0x7f2d640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.412+0000 7f2d74ff9700 1 -- 172.21.15.93:0/35436527 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d600ce0e0 con 0x7f2d7810a1b0 2023-07-21T22:36:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.412+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 0x7f2d640a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009c60 tx=0x7f2d68009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.439+0000 7f2d89496b80 1 Processor -- start 2023-07-21T22:36:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.439+0000 7f2d89496b80 1 -- start start 2023-07-21T22:36:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b13bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b13c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b041c50 con 0x55ca1af986b0 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b0949e0 con 0x55ca1b142210 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d89496b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b08de70 con 0x55ca1b142660 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49442/0 (socket says 172.21.15.93:49442) 2023-07-21T22:36:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 -- 172.21.15.93:0/2585178323 learned_addr learned my addr 172.21.15.93:0/2585178323 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b13bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 -- 172.21.15.93:0/2585178323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 msgr2=0x55ca1b13c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b13c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 -- 172.21.15.93:0/2585178323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 msgr2=0x55ca1b13bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b13bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.440+0000 7f2d76ffd700 1 -- 172.21.15.93:0/2585178323 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1aebbd80 con 0x55ca1af986b0 2023-07-21T22:36:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.441+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800e4e0 tx=0x7f2d68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579bed0d1d618d38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.441+0000 7f2d6d7fa700 1 -- 172.21.15.93:0/2585178323 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800ee00 con 0x55ca1af986b0 2023-07-21T22:36:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.441+0000 7f2d6d7fa700 1 -- 172.21.15.93:0/2585178323 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d68021070 con 0x55ca1af986b0 2023-07-21T22:36:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.441+0000 7f2d6d7fa700 1 -- 172.21.15.93:0/2585178323 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6801d810 con 0x55ca1af986b0 2023-07-21T22:36:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 -- 172.21.15.93:0/2585178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 msgr2=0x55ca1b141cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800e4e0 tx=0x7f2d68014000 comp rx=0 tx=0).stop 2023-07-21T22:36:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 -- 172.21.15.93:0/2585178323 shutdown_connections 2023-07-21T22:36:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b13c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141cd0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 --2- 172.21.15.93:0/2585178323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b13bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 -- 172.21.15.93:0/2585178323 >> 172.21.15.93:0/2585178323 conn(0x55ca1b131ae0 msgr2=0x55ca1b1370e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 -- 172.21.15.93:0/2585178323 shutdown_connections 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.442+0000 7f2d89496b80 1 -- 172.21.15.93:0/2585178323 wait complete. 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 -- start start 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 Processor -- start 2023-07-21T22:36:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b148620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b145830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b145f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b02da60 con 0x55ca1af986b0 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b02d760 con 0x55ca1b142210 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1af118c0 con 0x55ca1b142660 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d89496b80 10 client.0 _ll_register_callbacks cb 0x55ca1b02b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49448/0 (socket says 172.21.15.93:49448) 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.443+0000 7f2d76ffd700 1 -- 172.21.15.93:0/843517453 learned_addr learned my addr 172.21.15.93:0/843517453 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.444+0000 7f2d777fe700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 -- 172.21.15.93:0/843517453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 msgr2=0x55ca1b145830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b145830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 -- 172.21.15.93:0/843517453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 msgr2=0x55ca1b148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 -- 172.21.15.93:0/843517453 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d68017070 con 0x55ca1b142660 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d777fe700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b145f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d781a7370 tx=0x7f2d700045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad78d540fa10a6eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d5e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007af0 con 0x55ca1b142660 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca1afe7490 con 0x55ca1b142660 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.445+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d70007c50 con 0x55ca1b142660 2023-07-21T22:36:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d70007e10 con 0x55ca1b142660 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 10 client.26950 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55ca1aea4c00 con 0x55ca1b142660 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca1afe7490 con 0x55ca1b142660 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d7001e570 con 0x55ca1b142660 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 10 client.26950 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 10 client.26950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 20 client.26950 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b155c20 0x55ca1b158090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 4 client.26950 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.446+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca1b086e50 con 0x55ca1b155c20 2023-07-21T22:36:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.448+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b155c20 0x55ca1b158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.448+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b155c20 0x55ca1b158090 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbef42edc26cf75b server_cookie=cd712061c648a3e7 in_seq=0 out_seq=0 2023-07-21T22:36:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.448+0000 7f2d5e7fc700 10 client.26950 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.451+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/843517453 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca1b086e50 con 0x55ca1b155c20 2023-07-21T22:36:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.451+0000 7f2d5e7fc700 10 client.26950 handle_command_reply: tid=0 2023-07-21T22:36:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.451+0000 7f2d89496b80 1 client.26950 shutdown 2023-07-21T22:36:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.451+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 msgr2=0x55ca1b145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.451+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b145f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d781a7370 tx=0x7f2d700045a0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 shutdown_connections 2023-07-21T22:36:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b155c20 0x55ca1b158090 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b145f70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 --2- 172.21.15.93:0/843517453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 >> 172.21.15.93:0/843517453 conn(0x55ca1b131ae0 msgr2=0x55ca1b1363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d5e7fc700 0 client.26950 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 shutdown_connections 2023-07-21T22:36:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 1 -- 172.21.15.93:0/843517453 wait complete. 2023-07-21T22:36:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.452+0000 7f2d89496b80 20 client.26950 trim_cache size 0 max 16384 2023-07-21T22:36:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 Processor -- start 2023-07-21T22:36:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 -- start start 2023-07-21T22:36:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b148150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b136b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b13cff0 con 0x55ca1af986b0 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b1462d0 con 0x55ca1b142210 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d89496b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b1334f0 con 0x55ca1b142660 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49454/0 (socket says 172.21.15.93:49454) 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d76ffd700 1 -- 172.21.15.93:0/1254952585 learned_addr learned my addr 172.21.15.93:0/1254952585 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.473+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 -- 172.21.15.93:0/1254952585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 msgr2=0x55ca1b136b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b136b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 -- 172.21.15.93:0/1254952585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 msgr2=0x55ca1b148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 -- 172.21.15.93:0/1254952585 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1aee21f0 con 0x55ca1af986b0 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011ab0 tx=0x7f2d6801cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c980bcad1fe11ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d6dffb700 1 -- 172.21.15.93:0/1254952585 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800ec00 con 0x55ca1af986b0 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d6dffb700 1 -- 172.21.15.93:0/1254952585 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d68033110 con 0x55ca1af986b0 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.474+0000 7f2d6dffb700 1 -- 172.21.15.93:0/1254952585 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680054c0 con 0x55ca1af986b0 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- 172.21.15.93:0/1254952585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 msgr2=0x55ca1b141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011ab0 tx=0x7f2d6801cef0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- 172.21.15.93:0/1254952585 shutdown_connections 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142660 0x55ca1b136b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1af986b0 0x55ca1b141020 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 --2- 172.21.15.93:0/1254952585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1b142210 0x55ca1b148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- 172.21.15.93:0/1254952585 >> 172.21.15.93:0/1254952585 conn(0x55ca1b131ae0 msgr2=0x55ca1b140910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- 172.21.15.93:0/1254952585 shutdown_connections 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- 172.21.15.93:0/1254952585 wait complete. 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.475+0000 7f2d89496b80 1 -- start start 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 Processor -- start 2023-07-21T22:36:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1af986b0 0x55ca1b14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 0x55ca1b14b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b1a8b80 con 0x55ca1b142660 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b1a8cc0 con 0x55ca1af986b0 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1b1a8e00 con 0x55ca1b142210 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d89496b80 10 client.0 _ll_register_callbacks cb 0x7f2d70003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49464/0 (socket says 172.21.15.93:49464) 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d777fe700 1 -- 172.21.15.93:0/3233134635 learned_addr learned my addr 172.21.15.93:0/3233134635 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1af986b0 0x55ca1b14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.476+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 0x55ca1b14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 -- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 msgr2=0x55ca1b14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 0x55ca1b14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 -- 172.21.15.93:0/3233134635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1af986b0 msgr2=0x55ca1b14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1af986b0 0x55ca1b14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 -- 172.21.15.93:0/3233134635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6802a040 con 0x55ca1b142660 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d6ffff700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 0x55ca1b14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d777fe700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2d70002230 tx=0x7f2d70005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e605d8a67b9191b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d5e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7001e5c0 con 0x55ca1b142660 2023-07-21T22:36:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2d700078f0 con 0x55ca1b142660 2023-07-21T22:36:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d68021070 con 0x55ca1b142660 2023-07-21T22:36:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7001e720 con 0x55ca1b142660 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d89496b80 10 client.17724 fetch_fsmap learned FSMap version 84 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.477+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) v3 -- 0x55ca1aeba250 con 0x55ca1b142660 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d68021070 con 0x55ca1b142660 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 84) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d70015b50 con 0x55ca1b142660 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 10 client.17724 fetch_fsmap finished waiting for FSMap version 84 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 10 client.17724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 20 client.17724 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b1aa4d0 0x55ca1b1ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 4 client.17724 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca1b048ff0 con 0x55ca1b1aa4d0 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.478+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b1aa4d0 0x55ca1b1ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.479+0000 7f2d5e7fc700 10 client.17724 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.479+0000 7f2d76ffd700 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b1aa4d0 0x55ca1b1ac8e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cef2633ddaf5f04 server_cookie=9b2a7a5416e304a2 in_seq=0 out_seq=0 2023-07-21T22:36:27.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 256647 31.77 MB/sec execute 303 sec latency 7467.065 ms 2023-07-21T22:36:27.694 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:27 smithi093 ceph-mon[96579]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:27.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.734+0000 7f2d5e7fc700 1 -- 172.21.15.93:0/3233134635 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca1b048ff0 con 0x55ca1b1aa4d0 2023-07-21T22:36:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.734+0000 7f2d5e7fc700 10 client.17724 handle_command_reply: tid=0 2023-07-21T22:36:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.734+0000 7f2d89496b80 1 client.17724 shutdown 2023-07-21T22:36:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 msgr2=0x55ca1b14b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2d70002230 tx=0x7f2d70005ae0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 shutdown_connections 2023-07-21T22:36:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ca1b1aa4d0 0x55ca1b1ac8e0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ca1b142660 0x55ca1b14b6a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d5e7fc700 0 client.17724 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca1af986b0 0x55ca1b14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 --2- 172.21.15.93:0/3233134635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ca1b142210 0x55ca1b14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 >> 172.21.15.93:0/3233134635 conn(0x55ca1b131ae0 msgr2=0x55ca1b1411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 shutdown_connections 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 1 -- 172.21.15.93:0/3233134635 wait complete. 2023-07-21T22:36:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.735+0000 7f2d89496b80 20 client.17724 trim_cache size 0 max 16384 2023-07-21T22:36:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:36:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:36:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:36:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:36:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 msgr2=0x7f2d640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 0x7f2d640a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009c60 tx=0x7f2d68009260 comp rx=0 tx=0).stop 2023-07-21T22:36:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 msgr2=0x7f2d781a6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d60005910 tx=0x7f2d60004730 comp rx=0 tx=0).stop 2023-07-21T22:36:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 shutdown_connections 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2d640a12c0 0x7f2d640a3770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d78114c80 0x7f2d781a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d7806ce50 0x7f2d781a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 --2- 172.21.15.93:0/35436527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7810a1b0 0x7f2d781a6bb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 >> 172.21.15.93:0/35436527 conn(0x7f2d7806bd70 msgr2=0x7f2d78115330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 shutdown_connections 2023-07-21T22:36:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:27.736+0000 7f2d7d311700 1 -- 172.21.15.93:0/35436527 wait complete. 2023-07-21T22:36:27.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:27 smithi194 ceph-mon[98288]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:27.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:27 smithi134 ceph-mon[97873]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 708 op/s 2023-07-21T22:36:28.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 257094 31.74 MB/sec execute 304 sec latency 163.361 ms 2023-07-21T22:36:29.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 257709 31.74 MB/sec execute 305 sec latency 69.093 ms 2023-07-21T22:36:29.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:29 smithi093 ceph-mon[96579]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 559 op/s 2023-07-21T22:36:29.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:29 smithi194 ceph-mon[98288]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 559 op/s 2023-07-21T22:36:29.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:29 smithi134 ceph-mon[97873]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.4 MiB/s wr, 559 op/s 2023-07-21T22:36:30.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 258290 31.72 MB/sec execute 306 sec latency 67.784 ms 2023-07-21T22:36:31.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 258931 31.73 MB/sec execute 307 sec latency 92.330 ms 2023-07-21T22:36:31.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:31 smithi093 ceph-mon[96579]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 545 op/s 2023-07-21T22:36:31.693 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:31 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:31.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:31 smithi194 ceph-mon[98288]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 545 op/s 2023-07-21T22:36:31.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:31 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:31.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:31 smithi134 ceph-mon[97873]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 545 op/s 2023-07-21T22:36:31.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:31 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:32.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 259535 31.73 MB/sec execute 308 sec latency 275.538 ms 2023-07-21T22:36:33.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 260233 31.78 MB/sec execute 309 sec latency 73.494 ms 2023-07-21T22:36:33.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:33 smithi093 ceph-mon[96579]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 571 op/s 2023-07-21T22:36:33.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:33 smithi194 ceph-mon[98288]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 571 op/s 2023-07-21T22:36:33.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:33 smithi134 ceph-mon[97873]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 571 op/s 2023-07-21T22:36:34.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 260736 31.74 MB/sec execute 310 sec latency 70.295 ms 2023-07-21T22:36:35.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 261349 31.75 MB/sec execute 311 sec latency 66.434 ms 2023-07-21T22:36:35.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:35 smithi093 ceph-mon[96579]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 647 op/s 2023-07-21T22:36:35.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:35 smithi194 ceph-mon[98288]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 647 op/s 2023-07-21T22:36:35.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:35 smithi134 ceph-mon[97873]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 647 op/s 2023-07-21T22:36:36.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 262073 31.79 MB/sec execute 312 sec latency 71.028 ms 2023-07-21T22:36:36.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:36.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:36.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:37.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 262647 31.77 MB/sec execute 313 sec latency 90.156 ms 2023-07-21T22:36:37.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:37 smithi194 ceph-mon[98288]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2023-07-21T22:36:37.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:37 smithi134 ceph-mon[97873]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2023-07-21T22:36:37.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:37 smithi093 ceph-mon[96579]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2023-07-21T22:36:38.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 263222 31.77 MB/sec execute 314 sec latency 74.804 ms 2023-07-21T22:36:39.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 263926 31.77 MB/sec execute 315 sec latency 94.484 ms 2023-07-21T22:36:39.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:39 smithi194 ceph-mon[98288]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 645 op/s 2023-07-21T22:36:39.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:39 smithi134 ceph-mon[97873]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 645 op/s 2023-07-21T22:36:39.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:39 smithi093 ceph-mon[96579]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 645 op/s 2023-07-21T22:36:40.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 264546 31.78 MB/sec execute 316 sec latency 126.616 ms 2023-07-21T22:36:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 -- 172.21.15.93:0/830584885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b81016b0 msgr2=0x7fe8b8109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 --2- 172.21.15.93:0/830584885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b81016b0 0x7fe8b8109ba0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 -- 172.21.15.93:0/830584885 shutdown_connections 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 --2- 172.21.15.93:0/830584885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b810a1c0 0x7fe8b810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 --2- 172.21.15.93:0/830584885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b8100da0 0x7fe8b8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 --2- 172.21.15.93:0/830584885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b81016b0 0x7fe8b8109ba0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 -- 172.21.15.93:0/830584885 >> 172.21.15.93:0/830584885 conn(0x7fe8b80fa530 msgr2=0x7fe8b80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 -- 172.21.15.93:0/830584885 shutdown_connections 2023-07-21T22:36:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.409+0000 7fe8c048c700 1 -- 172.21.15.93:0/830584885 wait complete. 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 Processor -- start 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 -- start start 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b8100da0 0x7fe8b80febd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b810a1c0 0x7fe8b81a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a43c0 con 0x7fe8b810a1c0 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a4500 con 0x7fe8b81016b0 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.410+0000 7fe8c048c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a4670 con 0x7fe8b8100da0 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60638/0 (socket says 172.21.15.93:60638) 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 -- 172.21.15.93:0/626770123 learned_addr learned my addr 172.21.15.93:0/626770123 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 -- 172.21.15.93:0/626770123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b8100da0 msgr2=0x7fe8b80febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b8100da0 0x7fe8b80febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 -- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b810a1c0 msgr2=0x7fe8b81a3e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:36:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b810a1c0 0x7fe8b81a3e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.411+0000 7fe8bda27700 1 -- 172.21.15.93:0/626770123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80095f0 con 0x7fe8b81016b0 2023-07-21T22:36:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8bda27700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8af7fe700 1 -- 172.21.15.93:0/626770123 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a801d070 con 0x7fe8b81016b0 2023-07-21T22:36:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8af7fe700 1 -- 172.21.15.93:0/626770123 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a80054f0 con 0x7fe8b81016b0 2023-07-21T22:36:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8af7fe700 1 -- 172.21.15.93:0/626770123 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8004c70 con 0x7fe8b81016b0 2023-07-21T22:36:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b81a48f0 con 0x7fe8b81016b0 2023-07-21T22:36:41.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.414+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81a4cf0 con 0x7fe8b81016b0 2023-07-21T22:36:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.417+0000 7fe8af7fe700 1 -- 172.21.15.93:0/626770123 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe8a800fbc0 con 0x7fe8b81016b0 2023-07-21T22:36:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.418+0000 7fe8af7fe700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 0x7fe8a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.418+0000 7fe8af7fe700 1 -- 172.21.15.93:0/626770123 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8a80cdde0 con 0x7fe8b81016b0 2023-07-21T22:36:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.418+0000 7fe8be228700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 0x7fe8a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.418+0000 7fe8be228700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 0x7fe8a40a37f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4005950 tx=0x7fe8b40058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 Processor -- start 2023-07-21T22:36:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 -- start start 2023-07-21T22:36:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d220 0x556451e46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451d3bc50 con 0x556451c926b0 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451d8e9e0 con 0x556451e4d670 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.486+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451d87e70 con 0x556451e4d220 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49480/0 (socket says 172.21.15.93:49480) 2023-07-21T22:36:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 -- 172.21.15.93:0/3000564645 learned_addr learned my addr 172.21.15.93:0/3000564645 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8bea29700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8bda27700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d220 0x556451e46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 -- 172.21.15.93:0/3000564645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d220 msgr2=0x556451e46f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d220 0x556451e46f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 -- 172.21.15.93:0/3000564645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 msgr2=0x556451e474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 -- 172.21.15.93:0/3000564645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556451bb5d80 con 0x556451c926b0 2023-07-21T22:36:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8bea29700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8be228700 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ebc0 tx=0x7fe8b4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa0df2816b4bae7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8acdac700 1 -- 172.21.15.93:0/3000564645 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4010ac0 con 0x556451c926b0 2023-07-21T22:36:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8acdac700 1 -- 172.21.15.93:0/3000564645 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b4009820 con 0x556451c926b0 2023-07-21T22:36:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.487+0000 7fe8acdac700 1 -- 172.21.15.93:0/3000564645 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b403d400 con 0x556451c926b0 2023-07-21T22:36:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.488+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3000564645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 msgr2=0x556451e4cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.488+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ebc0 tx=0x7fe8b4011000 comp rx=0 tx=0).stop 2023-07-21T22:36:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.488+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3000564645 shutdown_connections 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e4cce0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3000564645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d220 0x556451e46f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3000564645 >> 172.21.15.93:0/3000564645 conn(0x556451e2bae0 msgr2=0x556451e310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3000564645 shutdown_connections 2023-07-21T22:36:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.489+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3000564645 wait complete. 2023-07-21T22:36:41.539 INFO:tasks.workunit.client.0.smithi093.stdout: 10 265228 31.81 MB/sec execute 317 sec latency 58.585 ms 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 -- start start 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 Processor -- start 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451c926b0 0x556451e468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d670 0x556451e41b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451d27a60 con 0x556451e4d220 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451d27760 con 0x556451c926b0 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451c0b8c0 con 0x556451e4d670 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8cc611b80 10 client.0 _ll_register_callbacks cb 0x556451d25700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8bda27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8bda27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49488/0 (socket says 172.21.15.93:49488) 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8bda27700 1 -- 172.21.15.93:0/3809154937 learned_addr learned my addr 172.21.15.93:0/3809154937 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8bea29700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d670 0x556451e41b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.490+0000 7fe8be228700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451c926b0 0x556451e468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 -- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d670 msgr2=0x556451e41b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d670 0x556451e41b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 -- 172.21.15.93:0/3809154937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451c926b0 msgr2=0x556451e468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451c926b0 0x556451e468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 -- 172.21.15.93:0/3809154937 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b000e0c0 con 0x556451e4d220 2023-07-21T22:36:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8bda27700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80979b0 tx=0x7fe8a809afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9390f2dbd45b87b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b2070 con 0x556451e4d220 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8a80a3300 con 0x556451e4d220 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.491+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556451d80e50 con 0x556451e4d220 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a80b9170 con 0x556451e4d220 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 10 client.17733 fetch_fsmap learned FSMap version 86 2023-07-21T22:36:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fe8b400e870 con 0x556451e4d220 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a80c9080 con 0x556451e4d220 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8a80a2ee0 con 0x556451e4d220 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 10 client.17733 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 10 client.17733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 20 client.17733 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451e38390 0x556451e3a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 4 client.17733 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.492+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556451d42ff0 con 0x556451e38390 2023-07-21T22:36:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.493+0000 7fe8be228700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451e38390 0x556451e3a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.493+0000 7fe8be228700 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451e38390 0x556451e3a7a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0c531d001930f6 server_cookie=3b5b589c66c072ec in_seq=0 out_seq=0 2023-07-21T22:36:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.493+0000 7fe8a1ffb700 10 client.17733 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.505+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/3809154937 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556451d42ff0 con 0x556451e38390 2023-07-21T22:36:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8a1ffb700 10 client.17733 handle_command_reply: tid=0 2023-07-21T22:36:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 client.17733 shutdown 2023-07-21T22:36:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 msgr2=0x556451e46e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80979b0 tx=0x7fe8a809afc0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 shutdown_connections 2023-07-21T22:36:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451e38390 0x556451e3a7a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e4d670 0x556451e41b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451c926b0 0x556451e468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/3809154937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e4d220 0x556451e46e20 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 >> 172.21.15.93:0/3809154937 conn(0x556451e2bae0 msgr2=0x556451e303e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8a1ffb700 0 client.17733 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 shutdown_connections 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 1 -- 172.21.15.93:0/3809154937 wait complete. 2023-07-21T22:36:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.506+0000 7fe8cc611b80 20 client.17733 trim_cache size 0 max 16384 2023-07-21T22:36:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 Processor -- start 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 -- start start 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e3f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e3f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e32760 con 0x556451c926b0 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e47490 con 0x556451e4d670 2023-07-21T22:36:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e4c930 con 0x556451e2bae0 2023-07-21T22:36:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8be228700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8be228700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36574/0 (socket says 172.21.15.93:36574) 2023-07-21T22:36:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8be228700 1 -- 172.21.15.93:0/1436466695 learned_addr learned my addr 172.21.15.93:0/1436466695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.523+0000 7fe8bea29700 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e3f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 -- 172.21.15.93:0/1436466695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 msgr2=0x556451e3f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e3f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 -- 172.21.15.93:0/1436466695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 msgr2=0x556451e3f110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e3f110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 -- 172.21.15.93:0/1436466695 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556451b9ec00 con 0x556451e2bae0 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8bea29700 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e3f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8be228700 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009fd0 tx=0x7fe8b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd42182e2781817 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8ad5ad700 1 -- 172.21.15.93:0/1436466695 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b404b040 con 0x556451e2bae0 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8ad5ad700 1 -- 172.21.15.93:0/1436466695 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b404c040 con 0x556451e2bae0 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.524+0000 7fe8ad5ad700 1 -- 172.21.15.93:0/1436466695 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b403e8d0 con 0x556451e2bae0 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.525+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1436466695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 msgr2=0x556451e35640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.525+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009fd0 tx=0x7fe8b4004730 comp rx=0 tx=0).stop 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1436466695 shutdown_connections 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451e3f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e2bae0 0x556451e35640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1436466695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451c926b0 0x556451e3f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1436466695 >> 172.21.15.93:0/1436466695 conn(0x556451e4c200 msgr2=0x556451e34f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.526+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1436466695 shutdown_connections 2023-07-21T22:36:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.530+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1436466695 wait complete. 2023-07-21T22:36:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.530+0000 7fe8cc611b80 1 -- start start 2023-07-21T22:36:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 Processor -- start 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 0x556451ebf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451ebf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 0x556451e3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e3ee10 con 0x556451e2bae0 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e3ef80 con 0x556451e4d670 2023-07-21T22:36:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556451e3f140 con 0x556451e3e460 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8cc611b80 10 client.0 _ll_register_callbacks cb 0x7fe8a809ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8bea29700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 0x556451e3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8bea29700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 0x556451e3e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36578/0 (socket says 172.21.15.93:36578) 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8bea29700 1 -- 172.21.15.93:0/1355263107 learned_addr learned my addr 172.21.15.93:0/1355263107 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8be228700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 0x556451ebf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.531+0000 7fe8bda27700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451ebf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 -- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 msgr2=0x556451e3e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 0x556451e3e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 -- 172.21.15.93:0/1355263107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 msgr2=0x556451ebf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451ebf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 -- 172.21.15.93:0/1355263107 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b0003af0 con 0x556451e2bae0 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8bda27700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451ebf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8be228700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 0x556451ebf240 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4013040 tx=0x7fe8b4057b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba91c42085b62e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.532+0000 7fe8a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.533+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4063070 con 0x556451e2bae0 2023-07-21T22:36:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.533+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe8b403e550 con 0x556451e2bae0 2023-07-21T22:36:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.533+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556451ce1490 con 0x556451e2bae0 2023-07-21T22:36:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b404b040 con 0x556451e2bae0 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 10 client.17736 fetch_fsmap learned FSMap version 86 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x556451bb5d80 con 0x556451e2bae0 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556451ce1490 con 0x556451e2bae0 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8b4066020 con 0x556451e2bae0 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 10 client.17736 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:36:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 20 client.17736 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451ec7060 0x556451ec9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 4 client.17736 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.536+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556451bd6f10 con 0x556451ec7060 2023-07-21T22:36:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.537+0000 7fe8bda27700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451ec7060 0x556451ec9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.540+0000 7fe8a1ffb700 10 client.17736 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.540+0000 7fe8bda27700 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451ec7060 0x556451ec9470 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18ebd79fe3b8a4c server_cookie=5681eb7d7d10ebd in_seq=0 out_seq=0 2023-07-21T22:36:41.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:41 smithi194 ceph-mon[98288]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 433 op/s 2023-07-21T22:36:41.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:41 smithi134 ceph-mon[97873]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 433 op/s 2023-07-21T22:36:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8a1ffb700 1 -- 172.21.15.93:0/1355263107 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556451bd6f10 con 0x556451ec7060 2023-07-21T22:36:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8a1ffb700 10 client.17736 handle_command_reply: tid=0 2023-07-21T22:36:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 client.17736 shutdown 2023-07-21T22:36:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:36:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:36:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:36:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 msgr2=0x556451ebf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 0x556451ebf240 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4013040 tx=0x7fe8b4057b60 comp rx=0 tx=0).stop 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 shutdown_connections 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556451ec7060 0x556451ec9470 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556451e3e460 0x556451e3e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556451e2bae0 0x556451ebf240 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 --2- 172.21.15.93:0/1355263107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556451e4d670 0x556451ebf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 >> 172.21.15.93:0/1355263107 conn(0x556451e4c200 msgr2=0x556451e40030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8a1ffb700 0 client.17736 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 shutdown_connections 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.879+0000 7fe8cc611b80 1 -- 172.21.15.93:0/1355263107 wait complete. 2023-07-21T22:36:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8cc611b80 20 client.17736 trim_cache size 0 max 16384 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 msgr2=0x7fe8a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 0x7fe8a40a37f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4005950 tx=0x7fe8b40058e0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 msgr2=0x7fe8b80ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8000c00 tx=0x7fe8a8005ef0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 shutdown_connections 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe8a40a1340 0x7fe8a40a37f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8b810a1c0 0x7fe8b81a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8b8100da0 0x7fe8b80febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 --2- 172.21.15.93:0/626770123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8b81016b0 0x7fe8b80ff110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 >> 172.21.15.93:0/626770123 conn(0x7fe8b80fa530 msgr2=0x7fe8b8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 shutdown_connections 2023-07-21T22:36:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:41.880+0000 7fe8c048c700 1 -- 172.21.15.93:0/626770123 wait complete. 2023-07-21T22:36:41.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:41 smithi093 ceph-mon[96579]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 433 op/s 2023-07-21T22:36:42.534 INFO:tasks.workunit.client.0.smithi093.stdout: 10 265507 31.77 MB/sec execute 318 sec latency 312.329 ms 2023-07-21T22:36:43.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 265547 31.68 MB/sec execute 319 sec latency 1034.736 ms 2023-07-21T22:36:43.841 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:43 smithi194 ceph-mon[98288]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 434 op/s 2023-07-21T22:36:43.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:43 smithi134 ceph-mon[97873]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 434 op/s 2023-07-21T22:36:43.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:43 smithi093 ceph-mon[96579]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 434 op/s 2023-07-21T22:36:44.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 265814 31.64 MB/sec execute 320 sec latency 494.874 ms 2023-07-21T22:36:45.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 266017 31.57 MB/sec execute 321 sec latency 150.953 ms 2023-07-21T22:36:45.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:45 smithi194 ceph-mon[98288]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 756 op/s 2023-07-21T22:36:45.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:45 smithi134 ceph-mon[97873]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 756 op/s 2023-07-21T22:36:45.945 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:45 smithi093 ceph-mon[96579]: pgmap v647: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 756 op/s 2023-07-21T22:36:46.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 266478 31.53 MB/sec execute 322 sec latency 114.447 ms 2023-07-21T22:36:47.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 267138 31.53 MB/sec execute 323 sec latency 126.774 ms 2023-07-21T22:36:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:47 smithi194 ceph-mon[98288]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 581 op/s 2023-07-21T22:36:47.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:47 smithi134 ceph-mon[97873]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 581 op/s 2023-07-21T22:36:47.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:47 smithi093 ceph-mon[96579]: pgmap v648: 97 pgs: 97 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 581 op/s 2023-07-21T22:36:48.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 267831 31.55 MB/sec execute 324 sec latency 118.312 ms 2023-07-21T22:36:49.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 268491 31.56 MB/sec execute 325 sec latency 65.671 ms 2023-07-21T22:36:49.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:49 smithi194 ceph-mon[98288]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 973 op/s 2023-07-21T22:36:49.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:49 smithi134 ceph-mon[97873]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 973 op/s 2023-07-21T22:36:49.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:49 smithi093 ceph-mon[96579]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 973 op/s 2023-07-21T22:36:50.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 269117 31.60 MB/sec execute 326 sec latency 78.064 ms 2023-07-21T22:36:51.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 269660 31.58 MB/sec execute 327 sec latency 84.531 ms 2023-07-21T22:36:51.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:51 smithi194 ceph-mon[98288]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 810 op/s 2023-07-21T22:36:51.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:51 smithi134 ceph-mon[97873]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 810 op/s 2023-07-21T22:36:51.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:51 smithi093 ceph-mon[96579]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 810 op/s 2023-07-21T22:36:52.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 270190 31.56 MB/sec execute 328 sec latency 114.027 ms 2023-07-21T22:36:53.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 270817 31.56 MB/sec execute 329 sec latency 87.623 ms 2023-07-21T22:36:53.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:53 smithi194 ceph-mon[98288]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 835 op/s 2023-07-21T22:36:53.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:53 smithi134 ceph-mon[97873]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 835 op/s 2023-07-21T22:36:53.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:53 smithi093 ceph-mon[96579]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 835 op/s 2023-07-21T22:36:54.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 271413 31.56 MB/sec execute 330 sec latency 126.023 ms 2023-07-21T22:36:55.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 272073 31.59 MB/sec execute 331 sec latency 65.368 ms 2023-07-21T22:36:55.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:55 smithi194 ceph-mon[98288]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 980 op/s 2023-07-21T22:36:55.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:55 smithi134 ceph-mon[97873]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 980 op/s 2023-07-21T22:36:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 -- 172.21.15.93:0/3953183427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00ff470 msgr2=0x7ff0a010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 --2- 172.21.15.93:0/3953183427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00ff470 0x7ff0a010b900 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff09800b0a0 tx=0x7ff09800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 -- 172.21.15.93:0/3953183427 shutdown_connections 2023-07-21T22:36:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 --2- 172.21.15.93:0/3953183427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00ff470 0x7ff0a010b900 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 --2- 172.21.15.93:0/3953183427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 0x7ff0a0104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 --2- 172.21.15.93:0/3953183427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00feac0 0x7ff0a00fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 -- 172.21.15.93:0/3953183427 >> 172.21.15.93:0/3953183427 conn(0x7ff0a0075bd0 msgr2=0x7ff0a0075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 -- 172.21.15.93:0/3953183427 shutdown_connections 2023-07-21T22:36:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 -- 172.21.15.93:0/3953183427 wait complete. 2023-07-21T22:36:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.895+0000 7ff0a7468700 1 Processor -- start 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 -- start start 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00ff470 0x7ff0a01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 0x7ff0a01a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01a2150 con 0x7ff0a00feac0 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01a22c0 con 0x7ff0a0104ad0 2023-07-21T22:36:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a7468700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01a2480 con 0x7ff0a00ff470 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49512/0 (socket says 172.21.15.93:49512) 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 -- 172.21.15.93:0/2216158742 learned_addr learned my addr 172.21.15.93:0/2216158742 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00ff470 0x7ff0a01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 0x7ff0a01a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 -- 172.21.15.93:0/2216158742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00ff470 msgr2=0x7ff0a01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00ff470 0x7ff0a01971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 -- 172.21.15.93:0/2216158742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 msgr2=0x7ff0a01a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 0x7ff0a01a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.896+0000 7ff0a5204700 1 -- 172.21.15.93:0/2216158742 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009d00 con 0x7ff0a00feac0 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0a5204700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e970 tx=0x7ff09c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0967fc700 1 -- 172.21.15.93:0/2216158742 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c009790 con 0x7ff0a00feac0 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a01949e0 con 0x7ff0a00feac0 2023-07-21T22:36:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0967fc700 1 -- 172.21.15.93:0/2216158742 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff09c00c7e0 con 0x7ff0a00feac0 2023-07-21T22:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0967fc700 1 -- 172.21.15.93:0/2216158742 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c003c70 con 0x7ff0a00feac0 2023-07-21T22:36:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.897+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a0194ef0 con 0x7ff0a00feac0 2023-07-21T22:36:55.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:55 smithi093 ceph-mon[96579]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 980 op/s 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.900+0000 7ff0967fc700 1 -- 172.21.15.93:0/2216158742 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff09c00cc50 con 0x7ff0a00feac0 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.900+0000 7ff0967fc700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 0x7ff08c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.901+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 0x7ff08c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.901+0000 7ff0967fc700 1 -- 172.21.15.93:0/2216158742 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff09c014070 con 0x7ff0a00feac0 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.901+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 0x7ff08c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00692f0 tx=0x7ff0900073d0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 Processor -- start 2023-07-21T22:36:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 -- start start 2023-07-21T22:36:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdaacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bcaac50 con 0x55a89bdab660 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bcfd9e0 con 0x55a89bc016b0 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bcf6e70 con 0x55a89bdab210 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49524/0 (socket says 172.21.15.93:49524) 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3279700072 learned_addr learned my addr 172.21.15.93:0/3279700072 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5204700 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdaacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3279700072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 msgr2=0x55a89bda4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3279700072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 msgr2=0x55a89bdaacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdaacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.902+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3279700072 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a89bb24d80 con 0x55a89bdab660 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.903+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff098009310 tx=0x7ff098009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1dc0334df646b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.903+0000 7ff08b7fe700 1 -- 172.21.15.93:0/3279700072 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800e040 con 0x55a89bdab660 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.903+0000 7ff08b7fe700 1 -- 172.21.15.93:0/3279700072 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff098003a00 con 0x55a89bdab660 2023-07-21T22:36:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.903+0000 7ff08b7fe700 1 -- 172.21.15.93:0/3279700072 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09801c6a0 con 0x55a89bdab660 2023-07-21T22:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3279700072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 msgr2=0x55a89bda54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff098009310 tx=0x7ff098009340 comp rx=0 tx=0).stop 2023-07-21T22:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3279700072 shutdown_connections 2023-07-21T22:36:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bda54a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdaacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3279700072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3279700072 >> 172.21.15.93:0/3279700072 conn(0x55a89bd9aae0 msgr2=0x55a89bda00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3279700072 shutdown_connections 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3279700072 wait complete. 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 -- start start 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.904+0000 7ff0b35edb80 1 Processor -- start 2023-07-21T22:36:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bc96a60 con 0x55a89bdab660 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bc96760 con 0x55a89bc016b0 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bb7a8c0 con 0x55a89bdab210 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0b35edb80 10 client.0 _ll_register_callbacks cb 0x55a89bc94700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a5204700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36608/0 (socket says 172.21.15.93:36608) 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a5204700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60688/0 (socket says 172.21.15.93:60688) 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 -- 172.21.15.93:0/681603631 learned_addr learned my addr 172.21.15.93:0/681603631 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 -- 172.21.15.93:0/681603631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 msgr2=0x55a89bdb4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 -- 172.21.15.93:0/681603631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 msgr2=0x55a89bdae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.905+0000 7ff0a4a03700 1 -- 172.21.15.93:0/681603631 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a010c200 con 0x55a89bdab210 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff090007fd0 tx=0x7ff090004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1882501a44a911f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff088ff9700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090010ee0 con 0x55a89bdab210 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff090017070 con 0x55a89bdab210 2023-07-21T22:36:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a89bc50490 con 0x55a89bdab210 2023-07-21T22:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.906+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff090003880 con 0x55a89bdab210 2023-07-21T22:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.907+0000 7ff0b35edb80 10 client.26977 fetch_fsmap learned FSMap version 86 2023-07-21T22:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.907+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55a89bb0dc00 con 0x55a89bdab210 2023-07-21T22:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.907+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a89bc50490 con 0x55a89bdab210 2023-07-21T22:36:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff090011c10 con 0x55a89bdab210 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 10 client.26977 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 10 client.26977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 20 client.26977 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89bdbeb20 0x55a89bdc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 4 client.26977 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a89bcefe50 con 0x55a89bdbeb20 2023-07-21T22:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.908+0000 7ff0a5204700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89bdbeb20 0x55a89bdc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.909+0000 7ff088ff9700 10 client.26977 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.909+0000 7ff0a5204700 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89bdbeb20 0x55a89bdc0f30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb33fda637ead6c server_cookie=83838326d022ef12 in_seq=0 out_seq=0 2023-07-21T22:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.911+0000 7ff088ff9700 1 -- 172.21.15.93:0/681603631 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a89bcefe50 con 0x55a89bdbeb20 2023-07-21T22:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.911+0000 7ff088ff9700 10 client.26977 handle_command_reply: tid=0 2023-07-21T22:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.911+0000 7ff0b35edb80 1 client.26977 shutdown 2023-07-21T22:36:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 msgr2=0x55a89bdb4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff090007fd0 tx=0x7ff090004750 comp rx=0 tx=0).stop 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 shutdown_connections 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89bdbeb20 0x55a89bdc0f30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/681603631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab210 0x55a89bdb4b90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 >> 172.21.15.93:0/681603631 conn(0x55a89bd9aae0 msgr2=0x55a89bd9f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff088ff9700 0 client.26977 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 shutdown_connections 2023-07-21T22:36:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.912+0000 7ff0b35edb80 1 -- 172.21.15.93:0/681603631 wait complete. 2023-07-21T22:36:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.913+0000 7ff0b35edb80 20 client.26977 trim_cache size 0 max 16384 2023-07-21T22:36:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 Processor -- start 2023-07-21T22:36:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 -- start start 2023-07-21T22:36:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bc016b0 0x55a89bdaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bdaa900 con 0x55a89bdab660 2023-07-21T22:36:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bd9c4f0 con 0x55a89bdab210 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.929+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bda9520 con 0x55a89bc016b0 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49534/0 (socket says 172.21.15.93:49534) 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60696/0 (socket says 172.21.15.93:60696) 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3505949658 learned_addr learned my addr 172.21.15.93:0/3505949658 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5204700 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bc016b0 0x55a89bdaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3505949658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bc016b0 msgr2=0x55a89bdaea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bc016b0 0x55a89bdaea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3505949658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 msgr2=0x55a89bda4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 -- 172.21.15.93:0/3505949658 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a89bdc5b60 con 0x55a89bdab660 2023-07-21T22:36:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff098009310 tx=0x7ff0980099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c4f9944ea9316f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.930+0000 7ff08bfff700 1 -- 172.21.15.93:0/3505949658 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800e040 con 0x55a89bdab660 2023-07-21T22:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff08bfff700 1 -- 172.21.15.93:0/3505949658 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff098007770 con 0x55a89bdab660 2023-07-21T22:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff08bfff700 1 -- 172.21.15.93:0/3505949658 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980054c0 con 0x55a89bdab660 2023-07-21T22:36:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3505949658 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 msgr2=0x55a89bdaa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff098009310 tx=0x7ff0980099e0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3505949658 shutdown_connections 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab660 0x55a89bdaa040 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bc016b0 0x55a89bdaea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/3505949658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bdab210 0x55a89bda4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3505949658 >> 172.21.15.93:0/3505949658 conn(0x55a89bd9aae0 msgr2=0x55a89bda9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.931+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3505949658 shutdown_connections 2023-07-21T22:36:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 -- 172.21.15.93:0/3505949658 wait complete. 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 -- start start 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 Processor -- start 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab660 0x55a89bdb2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.932+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89bdb3050 con 0x55a89bdab210 2023-07-21T22:36:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89be119e0 con 0x55a89bc016b0 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0b35edb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89be11b20 con 0x55a89bdab660 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0b35edb80 10 client.0 _ll_register_callbacks cb 0x7ff0900041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49542/0 (socket says 172.21.15.93:49542) 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 -- 172.21.15.93:0/2956233155 learned_addr learned my addr 172.21.15.93:0/2956233155 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:36:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a5a05700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab660 0x55a89bdb2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 -- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab660 msgr2=0x55a89bdb2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab660 0x55a89bdb2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 -- 172.21.15.93:0/2956233155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 msgr2=0x55a89bdb4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.933+0000 7ff0a4a03700 1 -- 172.21.15.93:0/2956233155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a89bb0dc00 con 0x55a89bdab210 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.934+0000 7ff0a4a03700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x55a89bdaff40 tx=0x7ff0900107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3170a4daef1fb81 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.934+0000 7ff088ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.934+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090011d70 con 0x55a89bdab210 2023-07-21T22:36:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.934+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff090017070 con 0x55a89bdab210 2023-07-21T22:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.934+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff098003a00 con 0x55a89bdab210 2023-07-21T22:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.935+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff090003680 con 0x55a89bdab210 2023-07-21T22:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.936+0000 7ff0b35edb80 10 client.17754 fetch_fsmap learned FSMap version 86 2023-07-21T22:36:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.936+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55a89bb24d80 con 0x55a89bdab210 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff090041030 con 0x55a89bdab210 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff09001f880 con 0x55a89bdab210 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 10 client.17754 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 20 client.17754 populate_metadata read hostname 'smithi093' 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89be131d0 0x55a89be155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 4 client.17754 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a89bcb1ff0 con 0x55a89be131d0 2023-07-21T22:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.939+0000 7ff0a5204700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89be131d0 0x55a89be155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.940+0000 7ff088ff9700 10 client.17754 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:55.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:55.940+0000 7ff0a5204700 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89be131d0 0x55a89be155e0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54e4a52f8acbc28 server_cookie=fcffc090017b2f46 in_seq=0 out_seq=0 2023-07-21T22:36:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff088ff9700 1 -- 172.21.15.93:0/2956233155 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a89bcb1ff0 con 0x55a89be131d0 2023-07-21T22:36:56.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff088ff9700 10 client.17754 handle_command_reply: tid=0 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 client.17754 shutdown 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 msgr2=0x55a89bdb2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x55a89bdaff40 tx=0x7ff0900107d0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 shutdown_connections 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55a89be131d0 0x55a89be155e0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a89bdab660 0x55a89bdb2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a89bc016b0 0x55a89bdb4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 --2- 172.21.15.93:0/2956233155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a89bdab210 0x55a89bdb2200 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 >> 172.21.15.93:0/2956233155 conn(0x55a89bd9aae0 msgr2=0x55a89bdae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff088ff9700 0 client.17754 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:36:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 shutdown_connections 2023-07-21T22:36:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.185+0000 7ff0b35edb80 1 -- 172.21.15.93:0/2956233155 wait complete. 2023-07-21T22:36:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0b35edb80 20 client.17754 trim_cache size 0 max 16384 2023-07-21T22:36:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:36:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:36:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:36:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:36:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 msgr2=0x7ff08c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 0x7ff08c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00692f0 tx=0x7ff0900073d0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 msgr2=0x7ff0a0196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e970 tx=0x7ff09c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 shutdown_connections 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a0104ad0 0x7ff0a01a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff08c0a12c0 0x7ff08c0a3770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a00feac0 0x7ff0a0196cb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 --2- 172.21.15.93:0/2216158742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0a00ff470 0x7ff0a01971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 >> 172.21.15.93:0/2216158742 conn(0x7ff0a0075bd0 msgr2=0x7ff0a0102680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 shutdown_connections 2023-07-21T22:36:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:36:56.186+0000 7ff0a7468700 1 -- 172.21.15.93:0/2216158742 wait complete. 2023-07-21T22:36:56.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 272627 31.58 MB/sec execute 332 sec latency 201.860 ms 2023-07-21T22:36:57.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 273262 31.58 MB/sec execute 333 sec latency 97.746 ms 2023-07-21T22:36:57.857 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:57 smithi194 ceph-mon[98288]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 628 op/s 2023-07-21T22:36:57.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:57 smithi134 ceph-mon[97873]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 628 op/s 2023-07-21T22:36:57.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:57 smithi093 ceph-mon[96579]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 628 op/s 2023-07-21T22:36:58.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 273372 31.50 MB/sec execute 334 sec latency 725.764 ms 2023-07-21T22:36:58.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:58 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:58.961 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:58 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:59.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:58 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:59.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 273795 31.49 MB/sec execute 335 sec latency 163.007 ms 2023-07-21T22:36:59.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:36:59 smithi134 ceph-mon[97873]: pgmap v654: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 944 op/s 2023-07-21T22:36:59.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:36:59 smithi093 ceph-mon[96579]: pgmap v654: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 944 op/s 2023-07-21T22:37:00.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:36:59 smithi194 ceph-mon[98288]: pgmap v654: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 944 op/s 2023-07-21T22:37:00.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 274285 31.48 MB/sec execute 336 sec latency 142.743 ms 2023-07-21T22:37:01.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 274382 31.40 MB/sec execute 337 sec latency 817.734 ms 2023-07-21T22:37:01.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:01 smithi134 ceph-mon[97873]: pgmap v655: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 578 op/s 2023-07-21T22:37:01.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:01 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:01.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:01 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:01.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:01 smithi093 ceph-mon[96579]: pgmap v655: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 578 op/s 2023-07-21T22:37:01.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:01 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:01.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:01 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:01 smithi194 ceph-mon[98288]: pgmap v655: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 578 op/s 2023-07-21T22:37:02.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:01 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:01 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 274382 31.31 MB/sec execute 338 sec latency 1817.783 ms 2023-07-21T22:37:02.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:02 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:02.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:02.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:02 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:03.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:03.092 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:02 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:03.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 274720 31.25 MB/sec execute 339 sec latency 2213.343 ms 2023-07-21T22:37:03.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:37:03.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:03.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 601 op/s 2023-07-21T22:37:03.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.884 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:03 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:37:03.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:03.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 601 op/s 2023-07-21T22:37:03.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:03.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:03 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:04.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-21T22:37:04.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:04.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: pgmap v656: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 12 MiB/s wr, 601 op/s 2023-07-21T22:37:04.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:04.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:04.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:04.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:03 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:04.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 275034 31.20 MB/sec execute 340 sec latency 176.429 ms 2023-07-21T22:37:04.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:04 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:37:04.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:04 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:37:04.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:04 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:04.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:04 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:05.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:04 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:37:05.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:04 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:37:05.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:04 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:05.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:04 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:05.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:04 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi093 to 196.1M 2023-07-21T22:37:05.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:04 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi093 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-21T22:37:05.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:04 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:05.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:04 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:05.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 275080 31.13 MB/sec execute 341 sec latency 725.589 ms 2023-07-21T22:37:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-07-21T22:37:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T22:37:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:05.943 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:05.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:06.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:06.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:06.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:06.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 14 MiB/s wr, 756 op/s 2023-07-21T22:37:06.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T22:37:06.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:06.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:06.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:06.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:37:06.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 275101 31.04 MB/sec execute 342 sec latency 663.052 ms 2023-07-21T22:37:06.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:06 smithi093 ceph-mon[96579]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:37:06.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:06 smithi093 ceph-mon[96579]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:07.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:06 smithi194 ceph-mon[98288]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:37:07.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:06 smithi194 ceph-mon[98288]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:07.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:06 smithi134 ceph-mon[97873]: Adjusting osd_memory_target on smithi194 to 196.2M 2023-07-21T22:37:07.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:06 smithi134 ceph-mon[97873]: Unable to set osd_memory_target on smithi194 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:07.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 275288 30.98 MB/sec execute 343 sec latency 450.853 ms 2023-07-21T22:37:07.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:07 smithi093 ceph-mon[96579]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 713 op/s 2023-07-21T22:37:08.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:07 smithi194 ceph-mon[98288]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 713 op/s 2023-07-21T22:37:08.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:07 smithi134 ceph-mon[97873]: pgmap v659: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 713 op/s 2023-07-21T22:37:08.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 275914 30.99 MB/sec execute 344 sec latency 173.008 ms 2023-07-21T22:37:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.346+0000 7f609f0e6700 1 -- 172.21.15.93:0/802279514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6098103430 msgr2=0x7f609810b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.346+0000 7f609f0e6700 1 --2- 172.21.15.93:0/802279514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6098103430 0x7f609810b000 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f609000b0d0 tx=0x7f609000b3e0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 -- 172.21.15.93:0/802279514 shutdown_connections 2023-07-21T22:37:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 --2- 172.21.15.93:0/802279514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6098103430 0x7f609810b000 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 --2- 172.21.15.93:0/802279514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 0x7f60980fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 --2- 172.21.15.93:0/802279514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f60980fed90 0x7f6098102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 -- 172.21.15.93:0/802279514 >> 172.21.15.93:0/802279514 conn(0x7f6098075f90 msgr2=0x7f6098076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 -- 172.21.15.93:0/802279514 shutdown_connections 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.347+0000 7f609f0e6700 1 -- 172.21.15.93:0/802279514 wait complete. 2023-07-21T22:37:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 Processor -- start 2023-07-21T22:37:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 -- start start 2023-07-21T22:37:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 0x7f6098071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098103430 0x7f609806aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609806b100 con 0x7f60980fed90 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609806b270 con 0x7f60980fe3f0 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f609f0e6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609806b430 con 0x7f6098103430 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49554/0 (socket says 172.21.15.93:49554) 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.348+0000 7f6097fff700 1 -- 172.21.15.93:0/2336398141 learned_addr learned my addr 172.21.15.93:0/2336398141 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f609ce82700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 0x7f6098071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 -- 172.21.15.93:0/2336398141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098103430 msgr2=0x7f609806aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098103430 0x7f609806aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 -- 172.21.15.93:0/2336398141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 msgr2=0x7f6098071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 0x7f6098071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 -- 172.21.15.93:0/2336398141 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090009d00 con 0x7f60980fed90 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6097fff700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f608400e940 tx=0x7f608400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6095ffb700 1 -- 172.21.15.93:0/2336398141 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084009730 con 0x7f60980fed90 2023-07-21T22:37:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6095ffb700 1 -- 172.21.15.93:0/2336398141 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f608400ee90 con 0x7f60980fed90 2023-07-21T22:37:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609806b710 con 0x7f60980fed90 2023-07-21T22:37:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.349+0000 7f6095ffb700 1 -- 172.21.15.93:0/2336398141 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084010ea0 con 0x7f60980fed90 2023-07-21T22:37:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.350+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a53a0 con 0x7f60980fed90 2023-07-21T22:37:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.357+0000 7f6095ffb700 1 -- 172.21.15.93:0/2336398141 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f608400cbd0 con 0x7f60980fed90 2023-07-21T22:37:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 Processor -- start 2023-07-21T22:37:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f6095ffb700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 0x7f60880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 -- start start 2023-07-21T22:37:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f6095ffb700 1 -- 172.21.15.93:0/2336398141 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6084014070 con 0x7f60980fed90 2023-07-21T22:37:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 0x55cb7d5cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 0x55cb7d5ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d4d3c50 con 0x55cb7d42a6b0 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d5269e0 con 0x55cb7d5d4660 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d51fe70 con 0x55cb7d5d4210 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f609ce82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f609ce82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49562/0 (socket says 172.21.15.93:49562) 2023-07-21T22:37:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f609ce82700 1 -- 172.21.15.93:0/3353282744 learned_addr learned my addr 172.21.15.93:0/3353282744 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.358+0000 7f6097fff700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 0x55cb7d5cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609d683700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 0x55cb7d5ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 -- 172.21.15.93:0/3353282744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 msgr2=0x55cb7d5cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 0x55cb7d5cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 -- 172.21.15.93:0/3353282744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 msgr2=0x55cb7d5ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 0x55cb7d5ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 -- 172.21.15.93:0/3353282744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb7d34dd80 con 0x55cb7d42a6b0 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f6097fff700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 0x55cb7d5cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609d683700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 0x55cb7d5ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609d683700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 0x7f60880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f609ce82700 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f608c00bd70 tx=0x7f608c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b702fe4c6f017087 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.359+0000 7f6082ffd700 1 -- 172.21.15.93:0/3353282744 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c010e10 con 0x55cb7d42a6b0 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.360+0000 7f6082ffd700 1 -- 172.21.15.93:0/3353282744 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f608c015070 con 0x55cb7d42a6b0 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.360+0000 7f6082ffd700 1 -- 172.21.15.93:0/3353282744 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011680 con 0x55cb7d42a6b0 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.360+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3353282744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 msgr2=0x55cb7d5d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.360+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f608c00bd70 tx=0x7f608c0106b0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3353282744 shutdown_connections 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d5d4660 0x55cb7d5ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5d3cd0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3353282744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4210 0x55cb7d5cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3353282744 >> 172.21.15.93:0/3353282744 conn(0x55cb7d5c3ae0 msgr2=0x55cb7d5c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3353282744 shutdown_connections 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3353282744 wait complete. 2023-07-21T22:37:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 -- start start 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 Processor -- start 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.361+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d4bfa60 con 0x55cb7d5d4210 2023-07-21T22:37:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d4bf760 con 0x55cb7d42a6b0 2023-07-21T22:37:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d3a38c0 con 0x55cb7d5d4660 2023-07-21T22:37:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f60ab26bb80 10 client.0 _ll_register_callbacks cb 0x55cb7d4bd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36650/0 (socket says 172.21.15.93:36650) 2023-07-21T22:37:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 -- 172.21.15.93:0/1460191135 learned_addr learned my addr 172.21.15.93:0/1460191135 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f6097fff700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 0x7f60880a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55cb7d5d4a70 tx=0x7f6090009a70 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 -- 172.21.15.93:0/1460191135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 msgr2=0x55cb7d5da640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 -- 172.21.15.93:0/1460191135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 msgr2=0x55cb7d5d78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.362+0000 7f609d683700 1 -- 172.21.15.93:0/1460191135 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981a53a0 con 0x55cb7d5d4660 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.366+0000 7f609d683700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6090015000 tx=0x7f6090006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6547c340ffacb2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.366+0000 7f6077fff700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:37:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.367+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090005450 con 0x55cb7d5d4660 2023-07-21T22:37:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.367+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f609002b070 con 0x55cb7d5d4660 2023-07-21T22:37:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.367+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb7d479490 con 0x55cb7d5d4660 2023-07-21T22:37:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.371+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6090004e30 con 0x55cb7d5d4660 2023-07-21T22:37:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.372+0000 7f60ab26bb80 10 client.26992 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.372+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f609810b840 con 0x55cb7d5d4660 2023-07-21T22:37:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.373+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb7d479490 con 0x55cb7d5d4660 2023-07-21T22:37:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f609002fa50 con 0x55cb7d5d4660 2023-07-21T22:37:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 10 client.26992 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 10 client.26992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 20 client.26992 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d5e7b30 0x55cb7d5e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 4 client.26992 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.374+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb7d518e50 con 0x55cb7d5e7b30 2023-07-21T22:37:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.377+0000 7f609ce82700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d5e7b30 0x55cb7d5e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.378+0000 7f609ce82700 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d5e7b30 0x55cb7d5e9f10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3c045c6b985aa4 server_cookie=44380c72e6937d2b in_seq=0 out_seq=0 2023-07-21T22:37:09.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.378+0000 7f6077fff700 10 client.26992 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f6077fff700 1 -- 172.21.15.93:0/1460191135 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb7d518e50 con 0x55cb7d5e7b30 2023-07-21T22:37:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f6077fff700 10 client.26992 handle_command_reply: tid=0 2023-07-21T22:37:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 client.26992 shutdown 2023-07-21T22:37:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 msgr2=0x55cb7d5d8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6090015000 tx=0x7f6090006000 comp rx=0 tx=0).stop 2023-07-21T22:37:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 shutdown_connections 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d5e7b30 0x55cb7d5e9f10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5d8000 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5da640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/1460191135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 >> 172.21.15.93:0/1460191135 conn(0x55cb7d5c3ae0 msgr2=0x55cb7d5c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.385+0000 7f6077fff700 0 client.26992 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.387+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 shutdown_connections 2023-07-21T22:37:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.387+0000 7f60ab26bb80 1 -- 172.21.15.93:0/1460191135 wait complete. 2023-07-21T22:37:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.387+0000 7f60ab26bb80 20 client.26992 trim_cache size 0 max 16384 2023-07-21T22:37:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.410+0000 7f60ab26bb80 1 Processor -- start 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 -- start start 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d5c0b20 con 0x55cb7d5d4210 2023-07-21T22:37:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d5d38d0 con 0x55cb7d42a6b0 2023-07-21T22:37:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d5e44a0 con 0x55cb7d5d4660 2023-07-21T22:37:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49578/0 (socket says 172.21.15.93:49578) 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f6097fff700 1 -- 172.21.15.93:0/3685828655 learned_addr learned my addr 172.21.15.93:0/3685828655 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f609ce82700 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.411+0000 7f609d683700 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 -- 172.21.15.93:0/3685828655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 msgr2=0x55cb7d5cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 -- 172.21.15.93:0/3685828655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 msgr2=0x55cb7d5dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 -- 172.21.15.93:0/3685828655 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c00b9a0 con 0x55cb7d5d4210 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.412+0000 7f6097fff700 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6084099e70 tx=0x7f608401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2decaf91646cb3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.413+0000 7f60837fe700 1 -- 172.21.15.93:0/3685828655 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608409df10 con 0x55cb7d5d4210 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.413+0000 7f60837fe700 1 -- 172.21.15.93:0/3685828655 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60840a2330 con 0x55cb7d5d4210 2023-07-21T22:37:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.413+0000 7f60837fe700 1 -- 172.21.15.93:0/3685828655 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608409a3a0 con 0x55cb7d5d4210 2023-07-21T22:37:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3685828655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 msgr2=0x55cb7d5d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6084099e70 tx=0x7f608401a040 comp rx=0 tx=0).stop 2023-07-21T22:37:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3685828655 shutdown_connections 2023-07-21T22:37:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d5d4660 0x55cb7d5cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d5dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/3685828655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4210 0x55cb7d5d30d0 secure :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6084099e70 tx=0x7f608401a040 comp rx=0 tx=0).stop 2023-07-21T22:37:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.417+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3685828655 >> 172.21.15.93:0/3685828655 conn(0x55cb7d5c3ae0 msgr2=0x55cb7d5d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.424+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3685828655 shutdown_connections 2023-07-21T22:37:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.424+0000 7f60ab26bb80 1 -- 172.21.15.93:0/3685828655 wait complete. 2023-07-21T22:37:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.424+0000 7f60ab26bb80 1 -- start start 2023-07-21T22:37:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.424+0000 7f60ab26bb80 1 Processor -- start 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d635310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d62f950 0x55cb7d62fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d630330 con 0x55cb7d5d4660 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d6304a0 con 0x55cb7d42a6b0 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb7d630660 con 0x55cb7d62f950 2023-07-21T22:37:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f60ab26bb80 10 client.0 _ll_register_callbacks cb 0x7f609000bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f6097fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49586/0 (socket says 172.21.15.93:49586) 2023-07-21T22:37:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f6097fff700 1 -- 172.21.15.93:0/2994751158 learned_addr learned my addr 172.21.15.93:0/2994751158 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.425+0000 7f6097fff700 1 -- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d62f950 msgr2=0x55cb7d62fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6097fff700 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d62f950 0x55cb7d62fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6097fff700 1 -- 172.21.15.93:0/2994751158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 msgr2=0x55cb7d635310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6097fff700 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d635310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6097fff700 1 -- 172.21.15.93:0/2994751158 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609806b710 con 0x55cb7d5d4660 2023-07-21T22:37:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6097fff700 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f60840999b0 tx=0x7f60840ca500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88817cdeb8564b61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6077fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608409df10 con 0x55cb7d5d4660 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f60840993d0 con 0x55cb7d5d4660 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f608c015070 con 0x55cb7d5d4660 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60840995b0 con 0x55cb7d5d4660 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f60ab26bb80 10 client.17772 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.426+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55cb7d336c00 con 0x55cb7d5d4660 2023-07-21T22:37:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60840a6020 con 0x55cb7d5d4660 2023-07-21T22:37:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60840a1ea0 con 0x55cb7d5d4660 2023-07-21T22:37:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 10 client.17772 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 20 client.17772 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d63d090 0x55cb7d63f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 4 client.17772 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.427+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb7d4daff0 con 0x55cb7d63d090 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.428+0000 7f609ce82700 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d63d090 0x55cb7d63f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.429+0000 7f6077fff700 10 client.17772 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.429+0000 7f609ce82700 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d63d090 0x55cb7d63f540 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32dcb077a218e083 server_cookie=86b44254ae0ebfa4 in_seq=0 out_seq=0 2023-07-21T22:37:09.538 INFO:tasks.workunit.client.0.smithi093.stdout: 10 276537 31.00 MB/sec execute 345 sec latency 91.785 ms 2023-07-21T22:37:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f6077fff700 1 -- 172.21.15.93:0/2994751158 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb7d4daff0 con 0x55cb7d63d090 2023-07-21T22:37:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f6077fff700 10 client.17772 handle_command_reply: tid=0 2023-07-21T22:37:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 client.17772 shutdown 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 msgr2=0x55cb7d635850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f60840999b0 tx=0x7f60840ca500 comp rx=0 tx=0).stop 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 shutdown_connections 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55cb7d63d090 0x55cb7d63f540 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb7d62f950 0x55cb7d62fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb7d42a6b0 0x55cb7d635310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 --2- 172.21.15.93:0/2994751158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cb7d5d4660 0x55cb7d635850 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 >> 172.21.15.93:0/2994751158 conn(0x55cb7d5c3ae0 msgr2=0x55cb7d5d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f6077fff700 0 client.17772 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 shutdown_connections 2023-07-21T22:37:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.648+0000 7f60ab26bb80 1 -- 172.21.15.93:0/2994751158 wait complete. 2023-07-21T22:37:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.649+0000 7f60ab26bb80 20 client.17772 trim_cache size 0 max 16384 2023-07-21T22:37:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:37:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:37:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:37:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:37:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.650+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 msgr2=0x7f60880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.650+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 0x7f60880a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55cb7d5d4a70 tx=0x7f6090009a70 comp rx=0 tx=0).stop 2023-07-21T22:37:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.650+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 msgr2=0x7f6098071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.650+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f608400e940 tx=0x7f608400ec50 comp rx=0 tx=0).stop 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 shutdown_connections 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6098103430 0x7f609806aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60980fe3f0 0x7f6098071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f60880a1490 0x7f60880a3940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 --2- 172.21.15.93:0/2336398141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60980fed90 0x7f6098071e80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 >> 172.21.15.93:0/2336398141 conn(0x7f6098075f90 msgr2=0x7f60980fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 shutdown_connections 2023-07-21T22:37:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:09.651+0000 7f609f0e6700 1 -- 172.21.15.93:0/2336398141 wait complete. 2023-07-21T22:37:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:09 smithi093 ceph-mon[96579]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-07-21T22:37:10.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:09 smithi194 ceph-mon[98288]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-07-21T22:37:10.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:09 smithi134 ceph-mon[97873]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.0 MiB/s wr, 591 op/s 2023-07-21T22:37:10.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 277060 30.99 MB/sec execute 346 sec latency 183.675 ms 2023-07-21T22:37:11.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 277540 30.96 MB/sec execute 347 sec latency 86.042 ms 2023-07-21T22:37:11.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:11 smithi093 ceph-mon[96579]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 601 op/s 2023-07-21T22:37:12.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:11 smithi194 ceph-mon[98288]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 601 op/s 2023-07-21T22:37:12.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:11 smithi134 ceph-mon[97873]: pgmap v661: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 601 op/s 2023-07-21T22:37:12.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 278226 30.97 MB/sec execute 348 sec latency 145.437 ms 2023-07-21T22:37:13.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 278914 31.02 MB/sec execute 349 sec latency 65.152 ms 2023-07-21T22:37:13.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:13 smithi093 ceph-mon[96579]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 571 op/s 2023-07-21T22:37:13.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:13 smithi093 ceph-mon[96579]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T22:37:13.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:13 smithi194 ceph-mon[98288]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 571 op/s 2023-07-21T22:37:13.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:13 smithi194 ceph-mon[98288]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T22:37:13.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:13 smithi134 ceph-mon[97873]: pgmap v662: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.6 MiB/s wr, 571 op/s 2023-07-21T22:37:13.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:13 smithi134 ceph-mon[97873]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T22:37:14.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 279423 31.03 MB/sec execute 350 sec latency 76.593 ms 2023-07-21T22:37:15.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 279882 30.98 MB/sec execute 351 sec latency 261.193 ms 2023-07-21T22:37:15.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:15 smithi093 ceph-mon[96579]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 604 op/s 2023-07-21T22:37:15.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:15 smithi194 ceph-mon[98288]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 604 op/s 2023-07-21T22:37:15.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:15 smithi134 ceph-mon[97873]: pgmap v664: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.1 MiB/s wr, 604 op/s 2023-07-21T22:37:16.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 280340 30.98 MB/sec execute 352 sec latency 1261.223 ms 2023-07-21T22:37:17.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 280896 30.94 MB/sec execute 353 sec latency 2261.250 ms 2023-07-21T22:37:17.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:17 smithi093 ceph-mon[96579]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 588 op/s 2023-07-21T22:37:17.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:17 smithi194 ceph-mon[98288]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 588 op/s 2023-07-21T22:37:17.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:17 smithi134 ceph-mon[97873]: pgmap v665: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 588 op/s 2023-07-21T22:37:18.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 281429 30.96 MB/sec execute 354 sec latency 2784.553 ms 2023-07-21T22:37:19.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 281991 30.98 MB/sec execute 355 sec latency 99.313 ms 2023-07-21T22:37:19.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:19 smithi093 ceph-mon[96579]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 550 op/s 2023-07-21T22:37:19.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:19 smithi194 ceph-mon[98288]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 550 op/s 2023-07-21T22:37:19.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:19 smithi134 ceph-mon[97873]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 550 op/s 2023-07-21T22:37:20.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 282451 30.94 MB/sec execute 356 sec latency 88.045 ms 2023-07-21T22:37:21.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 282930 30.93 MB/sec execute 357 sec latency 135.997 ms 2023-07-21T22:37:21.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:21 smithi093 ceph-mon[96579]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.1 MiB/s wr, 520 op/s 2023-07-21T22:37:21.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:21 smithi194 ceph-mon[98288]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.1 MiB/s wr, 520 op/s 2023-07-21T22:37:21.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:21 smithi134 ceph-mon[97873]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.1 MiB/s wr, 520 op/s 2023-07-21T22:37:22.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 283438 30.92 MB/sec execute 358 sec latency 83.724 ms 2023-07-21T22:37:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 -- 172.21.15.93:0/175779652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8540fea30 msgr2=0x7fc8540feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 --2- 172.21.15.93:0/175779652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8540fea30 0x7fc8540feea0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc844009990 tx=0x7fc844009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 -- 172.21.15.93:0/175779652 shutdown_connections 2023-07-21T22:37:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 --2- 172.21.15.93:0/175779652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8540ff3e0 0x7fc85410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 --2- 172.21.15.93:0/175779652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc854104a30 0x7fc854104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 --2- 172.21.15.93:0/175779652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8540fea30 0x7fc8540feea0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.786+0000 7fc85b771700 1 -- 172.21.15.93:0/175779652 >> 172.21.15.93:0/175779652 conn(0x7fc854074760 msgr2=0x7fc854074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.787+0000 7fc85b771700 1 -- 172.21.15.93:0/175779652 shutdown_connections 2023-07-21T22:37:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.787+0000 7fc85b771700 1 -- 172.21.15.93:0/175779652 wait complete. 2023-07-21T22:37:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.787+0000 7fc85b771700 1 Processor -- start 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.787+0000 7fc85b771700 1 -- start start 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8540fea30 0x7fc8541941a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8540ff3e0 0x7fc8541946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854199ca0 con 0x7fc854104a30 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854199e10 con 0x7fc8540ff3e0 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85b771700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854199fd0 con 0x7fc8540fea30 2023-07-21T22:37:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49598/0 (socket says 172.21.15.93:49598) 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 -- 172.21.15.93:0/3783697899 learned_addr learned my addr 172.21.15.93:0/3783697899 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc85950d700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8540fea30 0x7fc8541941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 -- 172.21.15.93:0/3783697899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8540fea30 msgr2=0x7fc8541941a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8540fea30 0x7fc8541941a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 -- 172.21.15.93:0/3783697899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8540ff3e0 msgr2=0x7fc8541946e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:37:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8540ff3e0 0x7fc8541946e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 -- 172.21.15.93:0/3783697899 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440095f0 con 0x7fc854104a30 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.788+0000 7fc859d0e700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00ef30 tx=0x7fc84c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.789+0000 7fc84a7fc700 1 -- 172.21.15.93:0/3783697899 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c018070 con 0x7fc854104a30 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.789+0000 7fc84a7fc700 1 -- 172.21.15.93:0/3783697899 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c00f040 con 0x7fc854104a30 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.789+0000 7fc84a7fc700 1 -- 172.21.15.93:0/3783697899 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00cd60 con 0x7fc854104a30 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.789+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85419a250 con 0x7fc854104a30 2023-07-21T22:37:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.789+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541b19e0 con 0x7fc854104a30 2023-07-21T22:37:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.793+0000 7fc84a7fc700 1 -- 172.21.15.93:0/3783697899 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc84c003a00 con 0x7fc854104a30 2023-07-21T22:37:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.793+0000 7fc84a7fc700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 0x7fc8400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.793+0000 7fc8678f6b80 1 Processor -- start 2023-07-21T22:37:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 -- start start 2023-07-21T22:37:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc84a7fc700 1 -- 172.21.15.93:0/3783697899 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc84c0cbf90 con 0x7fc854104a30 2023-07-21T22:37:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dbdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952db7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952cbdc50 con 0x556952c146b0 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952d109e0 con 0x556952dbe660 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952d09e70 con 0x556952dbe210 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952db7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60762/0 (socket says 172.21.15.93:60762) 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952db7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36682/0 (socket says 172.21.15.93:36682) 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 -- 172.21.15.93:0/1876510154 learned_addr learned my addr 172.21.15.93:0/1876510154 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 -- 172.21.15.93:0/1876510154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 msgr2=0x556952db7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952db7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 -- 172.21.15.93:0/1876510154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 msgr2=0x556952dbdcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dbdcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.794+0000 7fc859d0e700 1 -- 172.21.15.93:0/1876510154 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556952b37d80 con 0x556952dbe660 2023-07-21T22:37:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.795+0000 7fc859d0e700 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc84c095560 tx=0x7fc84c098f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9204cca52a1df804 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.795+0000 7fc85950d700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 0x7fc8400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.795+0000 7fc83f7fe700 1 -- 172.21.15.93:0/1876510154 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c06b570 con 0x556952dbe660 2023-07-21T22:37:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.795+0000 7fc83f7fe700 1 -- 172.21.15.93:0/1876510154 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c06b7b0 con 0x556952dbe660 2023-07-21T22:37:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.795+0000 7fc83f7fe700 1 -- 172.21.15.93:0/1876510154 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c095770 con 0x556952dbe660 2023-07-21T22:37:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- 172.21.15.93:0/1876510154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 msgr2=0x556952db84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc85950d700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 0x7fc8400a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ffba0 tx=0x7fc850007400 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc84c095560 tx=0x7fc84c098f40 comp rx=0 tx=0).stop 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- 172.21.15.93:0/1876510154 shutdown_connections 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952db84a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dbdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/1876510154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952db7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- 172.21.15.93:0/1876510154 >> 172.21.15.93:0/1876510154 conn(0x556952dadae0 msgr2=0x556952db30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- 172.21.15.93:0/1876510154 shutdown_connections 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- 172.21.15.93:0/1876510154 wait complete. 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 -- start start 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.796+0000 7fc8678f6b80 1 Processor -- start 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dc4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952ca9a60 con 0x556952c146b0 2023-07-21T22:37:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952ca9760 con 0x556952dbe660 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952b8d8c0 con 0x556952dbe210 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc8678f6b80 10 client.0 _ll_register_callbacks cb 0x556952ca7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36688/0 (socket says 172.21.15.93:36688) 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc858d0c700 1 -- 172.21.15.93:0/4207711177 learned_addr learned my addr 172.21.15.93:0/4207711177 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60772/0 (socket says 172.21.15.93:60772) 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 -- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 msgr2=0x556952dc1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 -- 172.21.15.93:0/4207711177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 msgr2=0x556952dc4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dc4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 -- 172.21.15.93:0/4207711177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541b19e0 con 0x556952dbe660 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc859d0e700 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc84c099220 tx=0x7fc84c00cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ccffd2a42ad5396 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.797+0000 7fc83cff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:37:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0a14b0 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc84c0992c0 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556952c63490 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84c099480 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 10 client.27107 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x556952b20c00 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556952c63490 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc84c06a320 con 0x556952dbe660 2023-07-21T22:37:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 10 client.27107 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 20 client.27107 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952dd3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 4 client.27107 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.798+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556952d02e50 con 0x556952dd1780 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.799+0000 7fc85950d700 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952dd3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.799+0000 7fc83cff9700 10 client.27107 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.799+0000 7fc85950d700 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952dd3b90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c862a60a6928af server_cookie=de0fd9f14d1f0894 in_seq=0 out_seq=0 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.800+0000 7fc83cff9700 1 -- 172.21.15.93:0/4207711177 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556952d02e50 con 0x556952dd1780 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.800+0000 7fc83cff9700 10 client.27107 handle_command_reply: tid=0 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.800+0000 7fc8678f6b80 1 client.27107 shutdown 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.800+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 msgr2=0x556952dc1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.800+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc84c099220 tx=0x7fc84c00cf40 comp rx=0 tx=0).stop 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 shutdown_connections 2023-07-21T22:37:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952dd3b90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc83cff9700 0 client.27107 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dc1db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952c146b0 0x556952dc4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/4207711177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 >> 172.21.15.93:0/4207711177 conn(0x556952dadae0 msgr2=0x556952db23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 shutdown_connections 2023-07-21T22:37:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 1 -- 172.21.15.93:0/4207711177 wait complete. 2023-07-21T22:37:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.801+0000 7fc8678f6b80 20 client.27107 trim_cache size 0 max 16384 2023-07-21T22:37:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 Processor -- start 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 -- start start 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952c146b0 0x556952db7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952d02e50 con 0x556952dbe660 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952c63490 con 0x556952c146b0 2023-07-21T22:37:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952dbc410 con 0x556952dbe210 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc85950d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952c146b0 0x556952db7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49618/0 (socket says 172.21.15.93:49618) 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.820+0000 7fc859d0e700 1 -- 172.21.15.93:0/2720105120 learned_addr learned my addr 172.21.15.93:0/2720105120 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 -- 172.21.15.93:0/2720105120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 msgr2=0x556952dc8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 -- 172.21.15.93:0/2720105120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952c146b0 msgr2=0x556952db7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952c146b0 0x556952db7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 -- 172.21.15.93:0/2720105120 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556952b5e1f0 con 0x556952dbe660 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc859d0e700 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc84c098ff0 tx=0x7fc84c0c97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b28d18d3d0858ea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc83ffff700 1 -- 172.21.15.93:0/2720105120 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0a1020 con 0x556952dbe660 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc83ffff700 1 -- 172.21.15.93:0/2720105120 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c0989c0 con 0x556952dbe660 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.821+0000 7fc83ffff700 1 -- 172.21.15.93:0/2720105120 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c098550 con 0x556952dbe660 2023-07-21T22:37:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- 172.21.15.93:0/2720105120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 msgr2=0x556952db2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc84c098ff0 tx=0x7fc84c0c97b0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- 172.21.15.93:0/2720105120 shutdown_connections 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe660 0x556952db2b60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952c146b0 0x556952db7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/2720105120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952dbe210 0x556952dc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- 172.21.15.93:0/2720105120 >> 172.21.15.93:0/2720105120 conn(0x556952dadae0 msgr2=0x556952dbb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- 172.21.15.93:0/2720105120 shutdown_connections 2023-07-21T22:37:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- 172.21.15.93:0/2720105120 wait complete. 2023-07-21T22:37:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.822+0000 7fc8678f6b80 1 -- start start 2023-07-21T22:37:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 Processor -- start 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952c146b0 0x556952db9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952dba850 con 0x556952dbe210 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952dba9c0 con 0x556952dbe660 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952dbab80 con 0x556952c146b0 2023-07-21T22:37:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc8678f6b80 10 client.0 _ll_register_callbacks cb 0x7fc844009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc858d0c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49622/0 (socket says 172.21.15.93:49622) 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc858d0c700 1 -- 172.21.15.93:0/719715310 learned_addr learned my addr 172.21.15.93:0/719715310 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc85950d700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952c146b0 0x556952db9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 -- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952c146b0 msgr2=0x556952db9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952c146b0 0x556952db9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 -- 172.21.15.93:0/719715310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 msgr2=0x556952db9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc859d0e700 1 -- 172.21.15.93:0/719715310 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556952b20c00 con 0x556952dbe660 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.823+0000 7fc858d0c700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc859d0e700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dba1b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc84c099220 tx=0x7fc84c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19d0517b10142ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c06a530 con 0x556952dbe660 2023-07-21T22:37:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc84c099060 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc84c06b7b0 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84c098550 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 10 client.27113 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x556952b36250 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c06b7b0 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc84c097fc0 con 0x556952dbe660 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 10 client.27113 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 10 client.27113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 20 client.27113 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.824+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952e27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.825+0000 7fc8678f6b80 4 client.27113 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.825+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556952cc4ff0 con 0x556952dd1780 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.825+0000 7fc85950d700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952e27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.825+0000 7fc83cff9700 10 client.27113 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:22.825+0000 7fc85950d700 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952e27f30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0010989686ead4 server_cookie=88e169ef6a6085ae in_seq=0 out_seq=0 2023-07-21T22:37:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.079+0000 7fc83cff9700 1 -- 172.21.15.93:0/719715310 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556952cc4ff0 con 0x556952dd1780 2023-07-21T22:37:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.079+0000 7fc83cff9700 10 client.27113 handle_command_reply: tid=0 2023-07-21T22:37:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.079+0000 7fc8678f6b80 1 client.27113 shutdown 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 msgr2=0x556952dba1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dba1b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc84c099220 tx=0x7fc84c009ed0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 shutdown_connections 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x556952dd1780 0x556952e27f30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556952dbe660 0x556952dba1b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556952c146b0 0x556952db9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 --2- 172.21.15.93:0/719715310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556952dbe210 0x556952db9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 >> 172.21.15.93:0/719715310 conn(0x556952dadae0 msgr2=0x556952dbeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc83cff9700 0 client.27113 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 shutdown_connections 2023-07-21T22:37:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.080+0000 7fc8678f6b80 1 -- 172.21.15.93:0/719715310 wait complete. 2023-07-21T22:37:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.083+0000 7fc8678f6b80 20 client.27113 trim_cache size 0 max 16384 2023-07-21T22:37:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:37:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:37:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:37:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:37:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.083+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 msgr2=0x7fc8400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.083+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 0x7fc8400a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ffba0 tx=0x7fc850007400 comp rx=0 tx=0).stop 2023-07-21T22:37:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.083+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 msgr2=0x7fc8541996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.083+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00ef30 tx=0x7fc84c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 shutdown_connections 2023-07-21T22:37:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8400a1340 0x7fc8400a37f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc854104a30 0x7fc8541996a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8540fea30 0x7fc8541941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 --2- 172.21.15.93:0/3783697899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8540ff3e0 0x7fc8541946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 >> 172.21.15.93:0/3783697899 conn(0x7fc854074760 msgr2=0x7fc854067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 shutdown_connections 2023-07-21T22:37:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:23.085+0000 7fc85b771700 1 -- 172.21.15.93:0/3783697899 wait complete. 2023-07-21T22:37:23.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 283891 30.88 MB/sec execute 359 sec latency 199.908 ms 2023-07-21T22:37:23.692 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:23 smithi093 ceph-mon[96579]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 496 op/s 2023-07-21T22:37:23.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:23 smithi194 ceph-mon[98288]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 496 op/s 2023-07-21T22:37:23.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:23 smithi134 ceph-mon[97873]: pgmap v668: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 496 op/s 2023-07-21T22:37:24.535 INFO:tasks.workunit.client.0.smithi093.stdout: 10 284487 30.89 MB/sec execute 360 sec latency 119.292 ms 2023-07-21T22:37:25.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 285080 30.92 MB/sec execute 361 sec latency 84.099 ms 2023-07-21T22:37:25.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:25 smithi093 ceph-mon[96579]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-07-21T22:37:25.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:25 smithi194 ceph-mon[98288]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-07-21T22:37:25.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:25 smithi134 ceph-mon[97873]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-07-21T22:37:26.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 285643 30.91 MB/sec execute 362 sec latency 59.854 ms 2023-07-21T22:37:27.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 286073 30.89 MB/sec execute 363 sec latency 114.776 ms 2023-07-21T22:37:27.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:27 smithi093 ceph-mon[96579]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.3 MiB/s wr, 416 op/s 2023-07-21T22:37:27.842 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:27 smithi194 ceph-mon[98288]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.3 MiB/s wr, 416 op/s 2023-07-21T22:37:27.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:27 smithi134 ceph-mon[97873]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.3 MiB/s wr, 416 op/s 2023-07-21T22:37:28.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 286607 30.88 MB/sec execute 364 sec latency 99.485 ms 2023-07-21T22:37:29.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 287192 30.86 MB/sec execute 365 sec latency 104.122 ms 2023-07-21T22:37:29.691 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:29 smithi093 ceph-mon[96579]: pgmap v671: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 594 op/s 2023-07-21T22:37:29.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:29 smithi194 ceph-mon[98288]: pgmap v671: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 594 op/s 2023-07-21T22:37:29.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:29 smithi134 ceph-mon[97873]: pgmap v671: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 594 op/s 2023-07-21T22:37:30.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 287838 30.89 MB/sec execute 366 sec latency 70.790 ms 2023-07-21T22:37:31.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 288461 30.90 MB/sec execute 367 sec latency 81.537 ms 2023-07-21T22:37:31.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:31 smithi194 ceph-mon[98288]: pgmap v672: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 416 op/s 2023-07-21T22:37:31.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:31 smithi134 ceph-mon[97873]: pgmap v672: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 416 op/s 2023-07-21T22:37:31.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:31 smithi093 ceph-mon[96579]: pgmap v672: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.7 MiB/s wr, 416 op/s 2023-07-21T22:37:32.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 288964 30.89 MB/sec execute 368 sec latency 98.728 ms 2023-07-21T22:37:33.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 289325 30.85 MB/sec execute 369 sec latency 126.776 ms 2023-07-21T22:37:33.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:33 smithi194 ceph-mon[98288]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.7 MiB/s wr, 417 op/s 2023-07-21T22:37:33.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:33 smithi134 ceph-mon[97873]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.7 MiB/s wr, 417 op/s 2023-07-21T22:37:33.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:33 smithi093 ceph-mon[96579]: pgmap v673: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.7 MiB/s wr, 417 op/s 2023-07-21T22:37:34.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 289878 30.86 MB/sec execute 370 sec latency 228.648 ms 2023-07-21T22:37:35.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 290411 30.85 MB/sec execute 371 sec latency 70.983 ms 2023-07-21T22:37:35.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:35 smithi194 ceph-mon[98288]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 586 op/s 2023-07-21T22:37:35.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:35 smithi134 ceph-mon[97873]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 586 op/s 2023-07-21T22:37:35.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:35 smithi093 ceph-mon[96579]: pgmap v674: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.9 MiB/s wr, 586 op/s 2023-07-21T22:37:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 -- 172.21.15.93:0/4292234810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab080fea30 msgr2=0x7fab080feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/4292234810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab080fea30 0x7fab080feea0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fab04009990 tx=0x7fab04009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 -- 172.21.15.93:0/4292234810 shutdown_connections 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/4292234810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080ff3e0 0x7fab0810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/4292234810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab08104a30 0x7fab08104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/4292234810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab080fea30 0x7fab080feea0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.397+0000 7fab0f8bc700 1 -- 172.21.15.93:0/4292234810 >> 172.21.15.93:0/4292234810 conn(0x7fab08074760 msgr2=0x7fab08074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.398+0000 7fab0f8bc700 1 -- 172.21.15.93:0/4292234810 shutdown_connections 2023-07-21T22:37:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.398+0000 7fab0f8bc700 1 -- 172.21.15.93:0/4292234810 wait complete. 2023-07-21T22:37:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.398+0000 7fab0f8bc700 1 Processor -- start 2023-07-21T22:37:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.398+0000 7fab0f8bc700 1 -- start start 2023-07-21T22:37:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab080fea30 0x7fab08196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080ff3e0 0x7fab08197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081a2140 con 0x7fab08104a30 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081a22b0 con 0x7fab080fea30 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0f8bc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab081948e0 con 0x7fab080ff3e0 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49634/0 (socket says 172.21.15.93:49634) 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0d658700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab080fea30 0x7fab08196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 -- 172.21.15.93:0/2748717356 learned_addr learned my addr 172.21.15.93:0/2748717356 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 -- 172.21.15.93:0/2748717356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080ff3e0 msgr2=0x7fab08197210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080ff3e0 0x7fab08197210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 -- 172.21.15.93:0/2748717356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab080fea30 msgr2=0x7fab08196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab080fea30 0x7fab08196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.399+0000 7fab0de59700 1 -- 172.21.15.93:0/2748717356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040095f0 con 0x7fab08104a30 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7fab0de59700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ef30 tx=0x7fab0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7faafe7fc700 1 -- 172.21.15.93:0/2748717356 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00018070 con 0x7fab08104a30 2023-07-21T22:37:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08194b60 con 0x7fab08104a30 2023-07-21T22:37:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7faafe7fc700 1 -- 172.21.15.93:0/2748717356 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0000f040 con 0x7fab08104a30 2023-07-21T22:37:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7faafe7fc700 1 -- 172.21.15.93:0/2748717356 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000cd80 con 0x7fab08104a30 2023-07-21T22:37:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.400+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08195020 con 0x7fab08104a30 2023-07-21T22:37:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.403+0000 7faafe7fc700 1 -- 172.21.15.93:0/2748717356 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fab00003a00 con 0x7fab08104a30 2023-07-21T22:37:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.404+0000 7faafe7fc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 0x7faaf40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.404+0000 7faafe7fc700 1 -- 172.21.15.93:0/2748717356 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab000cc950 con 0x7fab08104a30 2023-07-21T22:37:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 Processor -- start 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 -- start start 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f2166294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629a20 0x55f216623750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216520c50 con 0x55f216629a20 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2165739e0 con 0x55f2164776b0 2023-07-21T22:37:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.405+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21656ce70 con 0x55f216629e70 2023-07-21T22:37:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36722/0 (socket says 172.21.15.93:36722) 2023-07-21T22:37:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 -- 172.21.15.93:0/2678527968 learned_addr learned my addr 172.21.15.93:0/2678527968 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 -- 172.21.15.93:0/2678527968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 msgr2=0x55f2166294e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:37:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f2166294e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 -- 172.21.15.93:0/2678527968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629a20 msgr2=0x55f216623750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629a20 0x55f216623750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.406+0000 7fab0de59700 1 -- 172.21.15.93:0/2678527968 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f21639ad80 con 0x55f216629e70 2023-07-21T22:37:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.407+0000 7fab0d658700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 0x7faaf40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.408+0000 7fab0d658700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 0x7faaf40a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab080ffba0 tx=0x7faaf8009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.411+0000 7fab0de59700 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab000090a0 tx=0x7fab00099670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2344caf665bfb7f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.411+0000 7faaf37fe700 1 -- 172.21.15.93:0/2678527968 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000a1e70 con 0x55f216629e70 2023-07-21T22:37:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.411+0000 7faaf37fe700 1 -- 172.21.15.93:0/2678527968 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab00097e50 con 0x55f216629e70 2023-07-21T22:37:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.411+0000 7faaf37fe700 1 -- 172.21.15.93:0/2678527968 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00099f70 con 0x55f216629e70 2023-07-21T22:37:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2678527968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 msgr2=0x55f216623cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab000090a0 tx=0x7fab00099670 comp rx=0 tx=0).stop 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2678527968 shutdown_connections 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629e70 0x55f216623cb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f2166294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2678527968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629a20 0x55f216623750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2678527968 >> 172.21.15.93:0/2678527968 conn(0x55f216610ae0 msgr2=0x55f2166160e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2678527968 shutdown_connections 2023-07-21T22:37:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2678527968 wait complete. 2023-07-21T22:37:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 -- start start 2023-07-21T22:37:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.412+0000 7fab1ba41b80 1 Processor -- start 2023-07-21T22:37:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f21662fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629a20 0x55f21662cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21650ca60 con 0x55f216629e70 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21650c760 con 0x55f2164776b0 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2163f08c0 con 0x55f216629a20 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab1ba41b80 10 client.0 _ll_register_callbacks cb 0x55f21650a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0ce57700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629a20 0x55f21662cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0d658700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f21662fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49650/0 (socket says 172.21.15.93:49650) 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0de59700 1 -- 172.21.15.93:0/2007828103 learned_addr learned my addr 172.21.15.93:0/2007828103 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0d658700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f21662fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60808/0 (socket says 172.21.15.93:60808) 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0de59700 1 -- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629a20 msgr2=0x55f21662cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.413+0000 7fab0de59700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629a20 0x55f21662cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.414+0000 7fab0de59700 1 -- 172.21.15.93:0/2007828103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 msgr2=0x55f21662fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.414+0000 7fab0de59700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f21662fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.414+0000 7fab0de59700 1 -- 172.21.15.93:0/2007828103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08195020 con 0x55f216629e70 2023-07-21T22:37:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.414+0000 7fab0de59700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fab00009390 tx=0x7fab000093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1eb4122505945c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.414+0000 7faaf0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.416+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00098fa0 con 0x55f216629e70 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.416+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab00099100 con 0x55f216629e70 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.416+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2164c6490 con 0x55f216629e70 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab00099d10 con 0x55f216629e70 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 10 client.17793 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55f216383c00 con 0x55f216629e70 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2164c6490 con 0x55f216629e70 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab000958a0 con 0x55f216629e70 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 10 client.17793 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 10 client.17793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 20 client.17793 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f21664dda0 0x55f216650180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 4 client.17793 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f216565e50 con 0x55f21664dda0 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.417+0000 7fab0d658700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f21664dda0 0x55f216650180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.418+0000 7faaf0ff9700 10 client.17793 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.418+0000 7fab0d658700 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f21664dda0 0x55f216650180 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc67cd7ad47191 server_cookie=3e648d8cfd80d50e in_seq=0 out_seq=0 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7faaf0ff9700 1 -- 172.21.15.93:0/2007828103 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f216565e50 con 0x55f21664dda0 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7faaf0ff9700 10 client.17793 handle_command_reply: tid=0 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 client.17793 shutdown 2023-07-21T22:37:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 msgr2=0x55f21662d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fab00009390 tx=0x7fab000093e0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 shutdown_connections 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f21664dda0 0x55f216650180 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216629e70 0x55f21662d5e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f2164776b0 0x55f21662fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/2007828103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216629a20 0x55f21662cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 >> 172.21.15.93:0/2007828103 conn(0x55f216610ae0 msgr2=0x55f2166153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7faaf0ff9700 0 client.17793 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 shutdown_connections 2023-07-21T22:37:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 1 -- 172.21.15.93:0/2007828103 wait complete. 2023-07-21T22:37:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.419+0000 7fab1ba41b80 20 client.17793 trim_cache size 0 max 16384 2023-07-21T22:37:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.439+0000 7fab1ba41b80 1 Processor -- start 2023-07-21T22:37:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.439+0000 7fab1ba41b80 1 -- start start 2023-07-21T22:37:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216627870 0x55f216623750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 0x55f2166453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216616cb0 con 0x55f216610ae0 2023-07-21T22:37:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f21660db20 con 0x55f216649ed0 2023-07-21T22:37:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216612c00 con 0x55f216627870 2023-07-21T22:37:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0de59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49658/0 (socket says 172.21.15.93:49658) 2023-07-21T22:37:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0de59700 1 -- 172.21.15.93:0/313372482 learned_addr learned my addr 172.21.15.93:0/313372482 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 0x55f2166453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0d658700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216627870 0x55f216623750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 -- 172.21.15.93:0/313372482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216627870 msgr2=0x55f216623750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216627870 0x55f216623750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 -- 172.21.15.93:0/313372482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 msgr2=0x55f21662a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 -- 172.21.15.93:0/313372482 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2163c11f0 con 0x55f216649ed0 2023-07-21T22:37:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0de59700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.440+0000 7fab0ce57700 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 0x55f2166453c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab040098e0 tx=0x7fab04005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f08b620edf2e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.441+0000 7faaf3fff700 1 -- 172.21.15.93:0/313372482 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0401d030 con 0x55f216649ed0 2023-07-21T22:37:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.441+0000 7faaf3fff700 1 -- 172.21.15.93:0/313372482 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0401e040 con 0x55f216649ed0 2023-07-21T22:37:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.441+0000 7faaf3fff700 1 -- 172.21.15.93:0/313372482 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04004830 con 0x55f216649ed0 2023-07-21T22:37:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- 172.21.15.93:0/313372482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 msgr2=0x55f2166453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 0x55f2166453c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab040098e0 tx=0x7fab04005e80 comp rx=0 tx=0).stop 2023-07-21T22:37:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- 172.21.15.93:0/313372482 shutdown_connections 2023-07-21T22:37:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f21662a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216627870 0x55f216623750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/313372482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216649ed0 0x55f2166453c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- 172.21.15.93:0/313372482 >> 172.21.15.93:0/313372482 conn(0x55f2166274a0 msgr2=0x55f21662c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- 172.21.15.93:0/313372482 shutdown_connections 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- 172.21.15.93:0/313372482 wait complete. 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 -- start start 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.442+0000 7fab1ba41b80 1 Processor -- start 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f2166453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 0x55f216624fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 0x55f216625520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216625a60 con 0x55f216610ae0 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216625bd0 con 0x55f216627870 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f216625d90 con 0x55f216649ed0 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab1ba41b80 10 client.0 _ll_register_callbacks cb 0x7fab000998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab0de59700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 0x55f216625520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab0de59700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 0x55f216625520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36740/0 (socket says 172.21.15.93:36740) 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab0de59700 1 -- 172.21.15.93:0/871444704 learned_addr learned my addr 172.21.15.93:0/871444704 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab0d658700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f2166453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.443+0000 7fab0ce57700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 0x55f216624fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 -- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 msgr2=0x55f216625520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 0x55f216625520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 -- 172.21.15.93:0/871444704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 msgr2=0x55f2166453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f2166453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 -- 172.21.15.93:0/871444704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f216383c00 con 0x55f216627870 2023-07-21T22:37:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab0ce57700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 0x55f216624fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab04005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e3f83bfdefcdbe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7faaf0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04031070 con 0x55f216627870 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab00097e50 con 0x55f216627870 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fab04005530 con 0x55f216627870 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0401d030 con 0x55f216627870 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab1ba41b80 10 client.27128 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.444+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fab0810c160 con 0x55f216627870 2023-07-21T22:37:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab00097e50 con 0x55f216627870 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab0402b040 con 0x55f216627870 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 10 client.27128 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 20 client.27128 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f2164776b0 0x55f2166a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 4 client.27128 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f216527ff0 con 0x55f2164776b0 2023-07-21T22:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.445+0000 7fab0d658700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f2164776b0 0x55f2166a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.446+0000 7faaf0ff9700 10 client.27128 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.446+0000 7fab0d658700 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f2164776b0 0x55f2166a4be0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8043353b77ace329 server_cookie=11fe578b5a5204ef in_seq=0 out_seq=0 2023-07-21T22:37:36.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 290927 30.82 MB/sec execute 372 sec latency 104.683 ms 2023-07-21T22:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7faaf0ff9700 1 -- 172.21.15.93:0/871444704 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f216527ff0 con 0x55f2164776b0 2023-07-21T22:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7faaf0ff9700 10 client.27128 handle_command_reply: tid=0 2023-07-21T22:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 client.27128 shutdown 2023-07-21T22:37:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 msgr2=0x55f216624fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 0x55f216624fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab04005300 comp rx=0 tx=0).stop 2023-07-21T22:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 shutdown_connections 2023-07-21T22:37:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55f2164776b0 0x55f2166a4be0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f216649ed0 0x55f216625520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f216610ae0 0x55f2166453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 --2- 172.21.15.93:0/871444704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f216627870 0x55f216624fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 >> 172.21.15.93:0/871444704 conn(0x55f2166274a0 msgr2=0x55f21662c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7faaf0ff9700 0 client.27128 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 shutdown_connections 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.762+0000 7fab1ba41b80 1 -- 172.21.15.93:0/871444704 wait complete. 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.763+0000 7fab1ba41b80 20 client.27128 trim_cache size 0 max 16384 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:37:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:37:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 msgr2=0x7faaf40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 0x7faaf40a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab080ffba0 tx=0x7faaf8009260 comp rx=0 tx=0).stop 2023-07-21T22:37:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 msgr2=0x7fab081a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ef30 tx=0x7fab0000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 shutdown_connections 2023-07-21T22:37:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7faaf40a12c0 0x7faaf40a3770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab08104a30 0x7fab081a1b70 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab080fea30 0x7fab08196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 --2- 172.21.15.93:0/2748717356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab080ff3e0 0x7fab08197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.767+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 >> 172.21.15.93:0/2748717356 conn(0x7fab08074760 msgr2=0x7fab081025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.769+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 shutdown_connections 2023-07-21T22:37:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:36.771+0000 7fab0f8bc700 1 -- 172.21.15.93:0/2748717356 wait complete. 2023-07-21T22:37:37.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 291346 30.81 MB/sec execute 373 sec latency 239.771 ms 2023-07-21T22:37:37.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:37 smithi194 ceph-mon[98288]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 371 op/s 2023-07-21T22:37:37.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:37 smithi134 ceph-mon[97873]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 371 op/s 2023-07-21T22:37:37.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:37 smithi093 ceph-mon[96579]: pgmap v675: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 371 op/s 2023-07-21T22:37:38.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 291909 30.82 MB/sec execute 374 sec latency 109.604 ms 2023-07-21T22:37:39.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 291936 30.74 MB/sec execute 375 sec latency 962.513 ms 2023-07-21T22:37:39.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:39 smithi194 ceph-mon[98288]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 616 op/s 2023-07-21T22:37:39.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:39 smithi134 ceph-mon[97873]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 616 op/s 2023-07-21T22:37:39.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:39 smithi093 ceph-mon[96579]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 616 op/s 2023-07-21T22:37:40.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 292064 30.67 MB/sec execute 376 sec latency 1359.851 ms 2023-07-21T22:37:41.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 292291 30.61 MB/sec execute 377 sec latency 244.727 ms 2023-07-21T22:37:41.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:41 smithi194 ceph-mon[98288]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2023-07-21T22:37:41.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:41 smithi134 ceph-mon[97873]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2023-07-21T22:37:41.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:41 smithi093 ceph-mon[96579]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 8.3 MiB/s wr, 468 op/s 2023-07-21T22:37:42.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 292764 30.61 MB/sec execute 378 sec latency 136.980 ms 2023-07-21T22:37:43.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 293270 30.61 MB/sec execute 379 sec latency 77.448 ms 2023-07-21T22:37:43.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:43 smithi194 ceph-mon[98288]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 537 op/s 2023-07-21T22:37:43.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:43 smithi134 ceph-mon[97873]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 537 op/s 2023-07-21T22:37:43.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:43 smithi093 ceph-mon[96579]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 537 op/s 2023-07-21T22:37:44.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 293718 30.59 MB/sec execute 380 sec latency 113.743 ms 2023-07-21T22:37:45.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 294216 30.57 MB/sec execute 381 sec latency 103.751 ms 2023-07-21T22:37:45.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:45 smithi194 ceph-mon[98288]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 912 op/s 2023-07-21T22:37:45.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:45 smithi134 ceph-mon[97873]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 912 op/s 2023-07-21T22:37:45.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:45 smithi093 ceph-mon[96579]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 912 op/s 2023-07-21T22:37:46.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 294755 30.58 MB/sec execute 382 sec latency 89.634 ms 2023-07-21T22:37:47.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 295462 30.59 MB/sec execute 383 sec latency 73.335 ms 2023-07-21T22:37:47.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:47 smithi194 ceph-mon[98288]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-07-21T22:37:47.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:47 smithi134 ceph-mon[97873]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-07-21T22:37:47.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:47 smithi093 ceph-mon[96579]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 776 op/s 2023-07-21T22:37:48.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 296055 30.60 MB/sec execute 384 sec latency 76.415 ms 2023-07-21T22:37:49.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 296518 30.59 MB/sec execute 385 sec latency 118.570 ms 2023-07-21T22:37:49.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:49 smithi194 ceph-mon[98288]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 23 MiB/s wr, 974 op/s 2023-07-21T22:37:49.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:49 smithi134 ceph-mon[97873]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 23 MiB/s wr, 974 op/s 2023-07-21T22:37:49.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:49 smithi093 ceph-mon[96579]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 23 MiB/s wr, 974 op/s 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- 172.21.15.93:0/1063597092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2840ff3c0 msgr2=0x7fd28410b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1063597092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2840ff3c0 0x7fd28410b850 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00b0a0 tx=0x7fd27c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- 172.21.15.93:0/1063597092 shutdown_connections 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1063597092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2840ff3c0 0x7fd28410b850 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1063597092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd284104a20 0x7fd284104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1063597092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840fea10 0x7fd2840fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- 172.21.15.93:0/1063597092 >> 172.21.15.93:0/1063597092 conn(0x7fd2840fa4d0 msgr2=0x7fd2840fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- 172.21.15.93:0/1063597092 shutdown_connections 2023-07-21T22:37:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- 172.21.15.93:0/1063597092 wait complete. 2023-07-21T22:37:49.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 Processor -- start 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 -- start start 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.937+0000 7fd2897cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840ff3c0 0x7fd28419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2897cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd284104a20 0x7fd284194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2897cc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284195000 con 0x7fd284104a20 2023-07-21T22:37:49.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2897cc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284195170 con 0x7fd2840fea10 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2897cc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284195330 con 0x7fd2840ff3c0 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60836/0 (socket says 172.21.15.93:60836) 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 -- 172.21.15.93:0/1306386865 learned_addr learned my addr 172.21.15.93:0/1306386865 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840ff3c0 0x7fd28419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd2837fe700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd284104a20 0x7fd284194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 -- 172.21.15.93:0/1306386865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840ff3c0 msgr2=0x7fd28419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:49.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840ff3c0 0x7fd28419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 -- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd284104a20 msgr2=0x7fd284194990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd284104a20 0x7fd284194990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 -- 172.21.15.93:0/1306386865 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c009d00 con 0x7fd2840fea10 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d980 tx=0x7fd27400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.938+0000 7fd26bfff700 1 -- 172.21.15.93:0/1306386865 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400f890 con 0x7fd2840fea10 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.939+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284195610 con 0x7fd2840fea10 2023-07-21T22:37:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.939+0000 7fd26bfff700 1 -- 172.21.15.93:0/1306386865 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd274010460 con 0x7fd2840fea10 2023-07-21T22:37:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.939+0000 7fd26bfff700 1 -- 172.21.15.93:0/1306386865 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2740195c0 con 0x7fd2840fea10 2023-07-21T22:37:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.939+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841a1b20 con 0x7fd2840fea10 2023-07-21T22:37:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.941+0000 7fd26bfff700 1 -- 172.21.15.93:0/1306386865 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd2740105d0 con 0x7fd2840fea10 2023-07-21T22:37:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.942+0000 7fd26bfff700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 0x7fd26c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.942+0000 7fd26bfff700 1 -- 172.21.15.93:0/1306386865 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd2740cc9e0 con 0x7fd2840fea10 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.942+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 0x7fd26c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.943+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 0x7fd26c0a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2840fdd30 tx=0x7fd278009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.943+0000 7fd295951b80 1 Processor -- start 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.943+0000 7fd295951b80 1 -- start start 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbae3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb9e9c50 con 0x563cbbaea660 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbba3c9e0 con 0x563cbbaea210 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd295951b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbba35e70 con 0x563cbb9406b0 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49684/0 (socket says 172.21.15.93:49684) 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 -- 172.21.15.93:0/2511544136 learned_addr learned my addr 172.21.15.93:0/2511544136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd282ffd700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36762/0 (socket says 172.21.15.93:36762) 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2827fc700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbae3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 -- 172.21.15.93:0/2511544136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 msgr2=0x563cbbae9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 -- 172.21.15.93:0/2511544136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 msgr2=0x563cbbae3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbae3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 -- 172.21.15.93:0/2511544136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbb863d80 con 0x563cbbaea660 2023-07-21T22:37:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd282ffd700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.944+0000 7fd2837fe700 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009360 tx=0x7fd27c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397c078695b4bea4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.945+0000 7fd2697fa700 1 -- 172.21.15.93:0/2511544136 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00e040 con 0x563cbbaea660 2023-07-21T22:37:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.945+0000 7fd2697fa700 1 -- 172.21.15.93:0/2511544136 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd27c006000 con 0x563cbbaea660 2023-07-21T22:37:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.945+0000 7fd2697fa700 1 -- 172.21.15.93:0/2511544136 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c01c660 con 0x563cbbaea660 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.946+0000 7fd295951b80 1 -- 172.21.15.93:0/2511544136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 msgr2=0x563cbbae44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.946+0000 7fd295951b80 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009360 tx=0x7fd27c009c90 comp rx=0 tx=0).stop 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 -- 172.21.15.93:0/2511544136 shutdown_connections 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbae44a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbb9406b0 0x563cbbae9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 --2- 172.21.15.93:0/2511544136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbae3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 -- 172.21.15.93:0/2511544136 >> 172.21.15.93:0/2511544136 conn(0x563cbbad9ae0 msgr2=0x563cbbadf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 -- 172.21.15.93:0/2511544136 shutdown_connections 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 -- 172.21.15.93:0/2511544136 wait complete. 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 -- start start 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.947+0000 7fd295951b80 1 Processor -- start 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaf04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea660 0x563cbbaedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb9d5a60 con 0x563cbb9406b0 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb9d5760 con 0x563cbbaea210 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbb8b98c0 con 0x563cbbaea660 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd295951b80 10 client.0 _ll_register_callbacks cb 0x563cbb9d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea660 0x563cbbaedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60852/0 (socket says 172.21.15.93:60852) 2023-07-21T22:37:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 -- 172.21.15.93:0/1718178261 learned_addr learned my addr 172.21.15.93:0/1718178261 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaf04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 -- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea660 msgr2=0x563cbbaedd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea660 0x563cbbaedd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 -- 172.21.15.93:0/1718178261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 msgr2=0x563cbbaf04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaf04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 -- 172.21.15.93:0/1718178261 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841a1b20 con 0x563cbbaea210 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaf04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:37:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd2827fc700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2780044a0 tx=0x7fd2780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5f31a847eefca2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.948+0000 7fd266ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278003880 con 0x563cbbaea210 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd2780039e0 con 0x563cbbaea210 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cbb98f490 con 0x563cbbaea210 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd27801ce50 con 0x563cbbaea210 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 10 client.27140 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fd28410c150 con 0x563cbbaea210 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cbb98f490 con 0x563cbbaea210 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2780313b0 con 0x563cbbaea210 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 10 client.27140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 20 client.27140 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbafdb00 0x563cbbafff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 4 client.27140 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.949+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cbba2ee50 con 0x563cbbafdb00 2023-07-21T22:37:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.950+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbafdb00 0x563cbbafff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.950+0000 7fd266ff5700 10 client.27140 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.950+0000 7fd282ffd700 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbafdb00 0x563cbbafff10 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe9f1247007d1a0 server_cookie=616d941f3c7dafa in_seq=0 out_seq=0 2023-07-21T22:37:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd266ff5700 1 -- 172.21.15.93:0/1718178261 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563cbba2ee50 con 0x563cbbafdb00 2023-07-21T22:37:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd266ff5700 10 client.27140 handle_command_reply: tid=0 2023-07-21T22:37:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 client.27140 shutdown 2023-07-21T22:37:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 msgr2=0x563cbbaed740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2780044a0 tx=0x7fd2780047c0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 shutdown_connections 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbafdb00 0x563cbbafff10 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea660 0x563cbbaedd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd266ff5700 0 client.27140 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaf04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 --2- 172.21.15.93:0/1718178261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea210 0x563cbbaed740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 >> 172.21.15.93:0/1718178261 conn(0x563cbbad9ae0 msgr2=0x563cbbade3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.951+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 shutdown_connections 2023-07-21T22:37:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.952+0000 7fd295951b80 1 -- 172.21.15.93:0/1718178261 wait complete. 2023-07-21T22:37:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.952+0000 7fd295951b80 20 client.27140 trim_cache size 0 max 16384 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 Processor -- start 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 -- start start 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaf4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 0x563cbbaf0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbae4f20 con 0x563cbb9406b0 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbae4cd0 con 0x563cbbaea660 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd295951b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbaeb6f0 con 0x563cbbaea210 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.970+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49696/0 (socket says 172.21.15.93:49696) 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 -- 172.21.15.93:0/4107991012 learned_addr learned my addr 172.21.15.93:0/4107991012 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd2837fe700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 0x563cbbaf0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd2827fc700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaf4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 -- 172.21.15.93:0/4107991012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 msgr2=0x563cbbaf4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaf4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 -- 172.21.15.93:0/4107991012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 msgr2=0x563cbbaf0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 0x563cbbaf0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 -- 172.21.15.93:0/4107991012 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cbb84cc00 con 0x563cbb9406b0 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd2837fe700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 0x563cbbaf0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd2740ca190 tx=0x7fd274095e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ef00b74276e66e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd269ffb700 1 -- 172.21.15.93:0/4107991012 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274097bf0 con 0x563cbb9406b0 2023-07-21T22:37:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd269ffb700 1 -- 172.21.15.93:0/4107991012 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd274097e50 con 0x563cbb9406b0 2023-07-21T22:37:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.971+0000 7fd269ffb700 1 -- 172.21.15.93:0/4107991012 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274099f70 con 0x563cbb9406b0 2023-07-21T22:37:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 -- 172.21.15.93:0/4107991012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 msgr2=0x563cbbaeb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd2740ca190 tx=0x7fd274095e70 comp rx=0 tx=0).stop 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 -- 172.21.15.93:0/4107991012 shutdown_connections 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbbaea660 0x563cbbaf0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbb9406b0 0x563cbbaeb1b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 --2- 172.21.15.93:0/4107991012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaf4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 -- 172.21.15.93:0/4107991012 >> 172.21.15.93:0/4107991012 conn(0x563cbbad9ae0 msgr2=0x563cbbaeaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 -- 172.21.15.93:0/4107991012 shutdown_connections 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.972+0000 7fd295951b80 1 -- 172.21.15.93:0/4107991012 wait complete. 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 -- start start 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 Processor -- start 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbaec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbaeda80 con 0x563cbbaea660 2023-07-21T22:37:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbaec970 con 0x563cbb9406b0 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cbbaecb30 con 0x563cbbaea210 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd295951b80 10 client.0 _ll_register_callbacks cb 0x563cbb9d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd2837fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbaec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60864/0 (socket says 172.21.15.93:60864) 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.973+0000 7fd282ffd700 1 -- 172.21.15.93:0/4144095073 learned_addr learned my addr 172.21.15.93:0/4144095073 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:37:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 -- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 msgr2=0x563cbbaebef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaebef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 -- 172.21.15.93:0/4144095073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 msgr2=0x563cbbaec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbaec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 -- 172.21.15.93:0/4144095073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cbb863d80 con 0x563cbb9406b0 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd2837fe700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbaec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd282ffd700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd274000c00 tx=0x7fd274098030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=703b6cedd9ed022a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.974+0000 7fd266ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:37:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274098fa0 con 0x563cbb9406b0 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd274099100 con 0x563cbb9406b0 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd27c006000 con 0x563cbb9406b0 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd274099d10 con 0x563cbb9406b0 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 10 client.27146 fetch_fsmap learned FSMap version 86 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7fd284195610 con 0x563cbb9406b0 2023-07-21T22:37:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd27c006000 con 0x563cbb9406b0 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd2740c9e10 con 0x563cbb9406b0 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 20 client.27146 populate_metadata read hostname 'smithi093' 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbb525e0 0x563cbbb549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 4 client.27146 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.975+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563cbb9f0ff0 con 0x563cbbb525e0 2023-07-21T22:37:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.977+0000 7fd2827fc700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbb525e0 0x563cbbb549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.977+0000 7fd2827fc700 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbb525e0 0x563cbbb549c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7562c97f7c5db3 server_cookie=c67a372d699397bb in_seq=0 out_seq=0 2023-07-21T22:37:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:49.977+0000 7fd266ff5700 10 client.27146 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd266ff5700 1 -- 172.21.15.93:0/4144095073 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563cbb9f0ff0 con 0x563cbbb525e0 2023-07-21T22:37:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd266ff5700 10 client.27146 handle_command_reply: tid=0 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 client.27146 shutdown 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 msgr2=0x563cbbaed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd274000c00 tx=0x7fd274098030 comp rx=0 tx=0).stop 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 shutdown_connections 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563cbbb525e0 0x563cbbb549c0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563cbbaea660 0x563cbbaec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563cbb9406b0 0x563cbbaed460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd266ff5700 0 client.27146 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 --2- 172.21.15.93:0/4144095073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563cbbaea210 0x563cbbaebef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 >> 172.21.15.93:0/4144095073 conn(0x563cbbad9ae0 msgr2=0x563cbbae8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 shutdown_connections 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 1 -- 172.21.15.93:0/4144095073 wait complete. 2023-07-21T22:37:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.235+0000 7fd295951b80 20 client.27146 trim_cache size 0 max 16384 2023-07-21T22:37:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:37:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:37:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:37:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 msgr2=0x7fd26c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 0x7fd26c0a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd2840fdd30 tx=0x7fd278009260 comp rx=0 tx=0).stop 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 msgr2=0x7fd28419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d980 tx=0x7fd27400dc90 comp rx=0 tx=0).stop 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 shutdown_connections 2023-07-21T22:37:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fd26c0a12c0 0x7fd26c0a3770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd284104a20 0x7fd284194990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2840fea10 0x7fd28419ab30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 --2- 172.21.15.93:0/1306386865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2840ff3c0 0x7fd28419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 >> 172.21.15.93:0/1306386865 conn(0x7fd2840fa4d0 msgr2=0x7fd284105890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 shutdown_connections 2023-07-21T22:37:50.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:37:50.236+0000 7fd2897cc700 1 -- 172.21.15.93:0/1306386865 wait complete. 2023-07-21T22:37:50.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 296837 30.55 MB/sec execute 386 sec latency 213.726 ms 2023-07-21T22:37:51.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 297402 30.53 MB/sec execute 387 sec latency 99.766 ms 2023-07-21T22:37:51.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:51 smithi194 ceph-mon[98288]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:51.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:51 smithi134 ceph-mon[97873]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:51.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:51 smithi093 ceph-mon[96579]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:52.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 297977 30.55 MB/sec execute 388 sec latency 101.129 ms 2023-07-21T22:37:53.536 INFO:tasks.workunit.client.0.smithi093.stdout: 10 298590 30.56 MB/sec execute 389 sec latency 80.634 ms 2023-07-21T22:37:53.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:53 smithi194 ceph-mon[98288]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:53.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:53 smithi134 ceph-mon[97873]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:53.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:53 smithi093 ceph-mon[96579]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 735 op/s 2023-07-21T22:37:54.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 299128 30.55 MB/sec execute 390 sec latency 167.420 ms 2023-07-21T22:37:55.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 299585 30.54 MB/sec execute 391 sec latency 205.120 ms 2023-07-21T22:37:55.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:55 smithi194 ceph-mon[98288]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 845 op/s 2023-07-21T22:37:55.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:55 smithi134 ceph-mon[97873]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 845 op/s 2023-07-21T22:37:55.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:55 smithi093 ceph-mon[96579]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 19 MiB/s wr, 845 op/s 2023-07-21T22:37:56.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 300059 30.52 MB/sec execute 392 sec latency 83.680 ms 2023-07-21T22:37:57.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 300650 30.51 MB/sec execute 393 sec latency 87.452 ms 2023-07-21T22:37:57.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:57 smithi194 ceph-mon[98288]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2023-07-21T22:37:57.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:57 smithi134 ceph-mon[97873]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2023-07-21T22:37:57.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:57 smithi093 ceph-mon[96579]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.9 MiB/s wr, 471 op/s 2023-07-21T22:37:58.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 301186 30.51 MB/sec execute 394 sec latency 83.657 ms 2023-07-21T22:37:59.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 301870 30.55 MB/sec execute 395 sec latency 72.427 ms 2023-07-21T22:37:59.843 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:37:59 smithi194 ceph-mon[98288]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 600 op/s 2023-07-21T22:37:59.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:37:59 smithi134 ceph-mon[97873]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 600 op/s 2023-07-21T22:37:59.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:37:59 smithi093 ceph-mon[96579]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 600 op/s 2023-07-21T22:38:00.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302341 30.53 MB/sec execute 396 sec latency 150.961 ms 2023-07-21T22:38:01.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302436 30.47 MB/sec execute 397 sec latency 807.397 ms 2023-07-21T22:38:01.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:01 smithi194 ceph-mon[98288]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 422 op/s 2023-07-21T22:38:01.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:01 smithi134 ceph-mon[97873]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 422 op/s 2023-07-21T22:38:01.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:01 smithi093 ceph-mon[96579]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.1 MiB/s wr, 422 op/s 2023-07-21T22:38:02.537 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302436 30.39 MB/sec execute 398 sec latency 1807.511 ms 2023-07-21T22:38:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 -- 172.21.15.93:0/4020368776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 msgr2=0x7fce78103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 --2- 172.21.15.93:0/4020368776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce78103d80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099c0 tx=0x7fce6c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 -- 172.21.15.93:0/4020368776 shutdown_connections 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 --2- 172.21.15.93:0/4020368776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ffab0 0x7fce7810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 --2- 172.21.15.93:0/4020368776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce78103d80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 --2- 172.21.15.93:0/4020368776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ff100 0x7fce780ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 -- 172.21.15.93:0/4020368776 >> 172.21.15.93:0/4020368776 conn(0x7fce78075f90 msgr2=0x7fce78076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.349+0000 7fce7efda700 1 -- 172.21.15.93:0/4020368776 shutdown_connections 2023-07-21T22:38:03.395 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 -- 172.21.15.93:0/4020368776 wait complete. 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 Processor -- start 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 -- start start 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ff100 0x7fce78196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ffab0 0x7fce78197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781a2040 con 0x7fce781039b0 2023-07-21T22:38:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781a21b0 con 0x7fce780ffab0 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7efda700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781a2370 con 0x7fce780ff100 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7d577700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7d577700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49718/0 (socket says 172.21.15.93:49718) 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce7d577700 1 -- 172.21.15.93:0/617377492 learned_addr learned my addr 172.21.15.93:0/617377492 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.350+0000 7fce77fff700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ffab0 0x7fce78197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7cd76700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ff100 0x7fce78196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 -- 172.21.15.93:0/617377492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ff100 msgr2=0x7fce78196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ff100 0x7fce78196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 -- 172.21.15.93:0/617377492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ffab0 msgr2=0x7fce78197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ffab0 0x7fce78197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 -- 172.21.15.93:0/617377492 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0095f0 con 0x7fce781039b0 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7d577700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fce700094f0 tx=0x7fce7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce75ffb700 1 -- 172.21.15.93:0/617377492 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70004db0 con 0x7fce781039b0 2023-07-21T22:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce75ffb700 1 -- 172.21.15.93:0/617377492 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce7000f040 con 0x7fce781039b0 2023-07-21T22:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce781a2650 con 0x7fce781039b0 2023-07-21T22:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce75ffb700 1 -- 172.21.15.93:0/617377492 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70005980 con 0x7fce781039b0 2023-07-21T22:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.351+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce781a2b60 con 0x7fce781039b0 2023-07-21T22:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.353+0000 7fce75ffb700 1 -- 172.21.15.93:0/617377492 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fce70005020 con 0x7fce781039b0 2023-07-21T22:38:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.354+0000 7fce75ffb700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 0x7fce680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.354+0000 7fce7cd76700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 0x7fce680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.354+0000 7fce75ffb700 1 -- 172.21.15.93:0/617377492 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fce700cc670 con 0x7fce781039b0 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.354+0000 7fce7cd76700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 0x7fce680a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006010 tx=0x7fce6c009f10 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 Processor -- start 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 -- start start 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db45cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46210 0x55784db3ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784da45c50 con 0x55784db46660 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784da989e0 con 0x55784db46210 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784da91e70 con 0x55784d99c6b0 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49726/0 (socket says 172.21.15.93:49726) 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 -- 172.21.15.93:0/574742997 learned_addr learned my addr 172.21.15.93:0/574742997 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:03.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce77fff700 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46210 0x55784db3ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7cd76700 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db45cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 -- 172.21.15.93:0/574742997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 msgr2=0x55784db45cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db45cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 -- 172.21.15.93:0/574742997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46210 msgr2=0x55784db3ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46210 0x55784db3ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 -- 172.21.15.93:0/574742997 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55784d8bfd80 con 0x55784db46660 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.355+0000 7fce7d577700 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fce70095be0 tx=0x7fce70099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a005203ee0fdfee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce62ffd700 1 -- 172.21.15.93:0/574742997 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7009d710 con 0x55784db46660 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce62ffd700 1 -- 172.21.15.93:0/574742997 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce700a1b30 con 0x55784db46660 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce62ffd700 1 -- 172.21.15.93:0/574742997 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70098e30 con 0x55784db46660 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 -- 172.21.15.93:0/574742997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 msgr2=0x55784db404a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fce70095be0 tx=0x7fce70099580 comp rx=0 tx=0).stop 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 -- 172.21.15.93:0/574742997 shutdown_connections 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46660 0x55784db404a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db45cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/574742997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46210 0x55784db3ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 -- 172.21.15.93:0/574742997 >> 172.21.15.93:0/574742997 conn(0x55784db35ae0 msgr2=0x55784db3b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 -- 172.21.15.93:0/574742997 shutdown_connections 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.356+0000 7fce8b15fb80 1 -- 172.21.15.93:0/574742997 wait complete. 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 -- start start 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 Processor -- start 2023-07-21T22:38:03.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db49eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784da31a60 con 0x55784db46210 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784da31760 con 0x55784db46660 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784d9158c0 con 0x55784d99c6b0 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce8b15fb80 10 client.0 _ll_register_callbacks cb 0x55784da2f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:03.410 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49728/0 (socket says 172.21.15.93:49728) 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 -- 172.21.15.93:0/4115251643 learned_addr learned my addr 172.21.15.93:0/4115251643 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce7cd76700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 -- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 msgr2=0x55784db4c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db4c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 -- 172.21.15.93:0/4115251643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 msgr2=0x55784db49eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.411 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db49eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.357+0000 7fce77fff700 1 -- 172.21.15.93:0/4115251643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce781a2b60 con 0x55784db46210 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce77fff700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fce6400bac0 tx=0x7fce6400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01ef7c441bfee26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64004830 con 0x55784db46210 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fce64004990 con 0x55784db46210 2023-07-21T22:38:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55784d9eb490 con 0x55784db46210 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce64004b50 con 0x55784db46210 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 10 client.17826 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55784d8a8c00 con 0x55784db46210 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55784d9eb490 con 0x55784db46210 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce64011560 con 0x55784db46210 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 10 client.17826 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 20 client.17826 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784db59bf0 0x55784db64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 4 client.17826 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55784da8ae50 con 0x55784db59bf0 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.358+0000 7fce7cd76700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784db59bf0 0x55784db64890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.359+0000 7fce57fff700 10 client.17826 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.359+0000 7fce7cd76700 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784db59bf0 0x55784db64890 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fc8331302495b server_cookie=d5c0afee63bd5230 in_seq=0 out_seq=0 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.359+0000 7fce57fff700 1 -- 172.21.15.93:0/4115251643 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55784da8ae50 con 0x55784db59bf0 2023-07-21T22:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.359+0000 7fce57fff700 10 client.17826 handle_command_reply: tid=0 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.359+0000 7fce8b15fb80 1 client.17826 shutdown 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 msgr2=0x55784db49810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fce6400bac0 tx=0x7fce6400be80 comp rx=0 tx=0).stop 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 shutdown_connections 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784db59bf0 0x55784db64890 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db49eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784d99c6b0 0x55784db4c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/4115251643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784db46210 0x55784db49810 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce57fff700 0 client.17826 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 >> 172.21.15.93:0/4115251643 conn(0x55784db35ae0 msgr2=0x55784db3a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 shutdown_connections 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 1 -- 172.21.15.93:0/4115251643 wait complete. 2023-07-21T22:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.360+0000 7fce8b15fb80 20 client.17826 trim_cache size 0 max 16384 2023-07-21T22:38:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 Processor -- start 2023-07-21T22:38:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 -- start start 2023-07-21T22:38:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db3ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db3afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db40f20 con 0x55784d99c6b0 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db45920 con 0x55784db46660 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db44520 con 0x55784db3ab60 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce7cd76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce7cd76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49736/0 (socket says 172.21.15.93:49736) 2023-07-21T22:38:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce7cd76700 1 -- 172.21.15.93:0/1144543724 learned_addr learned my addr 172.21.15.93:0/1144543724 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce77fff700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db3ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.375+0000 7fce7d577700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db3afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 -- 172.21.15.93:0/1144543724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 msgr2=0x55784db3afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db3afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 -- 172.21.15.93:0/1144543724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 msgr2=0x55784db3ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db3ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 -- 172.21.15.93:0/1144543724 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55784db3c7e0 con 0x55784d99c6b0 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce77fff700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db3ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce7cd76700 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c54658d805d742c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce637fe700 1 -- 172.21.15.93:0/1144543724 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0052b0 con 0x55784d99c6b0 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce637fe700 1 -- 172.21.15.93:0/1144543724 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c01e040 con 0x55784d99c6b0 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce637fe700 1 -- 172.21.15.93:0/1144543724 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0345f0 con 0x55784d99c6b0 2023-07-21T22:38:03.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.376+0000 7fce8b15fb80 1 -- 172.21.15.93:0/1144543724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 msgr2=0x55784db50ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c020000 comp rx=0 tx=0).stop 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- 172.21.15.93:0/1144543724 shutdown_connections 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db3afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50ba0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/1144543724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db3ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- 172.21.15.93:0/1144543724 >> 172.21.15.93:0/1144543724 conn(0x55784db35ae0 msgr2=0x55784db43a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- 172.21.15.93:0/1144543724 shutdown_connections 2023-07-21T22:38:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- 172.21.15.93:0/1144543724 wait complete. 2023-07-21T22:38:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- start start 2023-07-21T22:38:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 Processor -- start 2023-07-21T22:38:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db41d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db42250 con 0x55784d99c6b0 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db423c0 con 0x55784db46660 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55784db42580 con 0x55784db3ab60 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce8b15fb80 10 client.0 _ll_register_callbacks cb 0x7fce64004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce7cd76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce7cd76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49742/0 (socket says 172.21.15.93:49742) 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce7cd76700 1 -- 172.21.15.93:0/3829543086 learned_addr learned my addr 172.21.15.93:0/3829543086 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce7d577700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db41d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.377+0000 7fce77fff700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 -- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 msgr2=0x55784db417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 -- 172.21.15.93:0/3829543086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 msgr2=0x55784db41d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db41d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 -- 172.21.15.93:0/3829543086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c02c040 con 0x55784d99c6b0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7d577700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db41d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce7cd76700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009350 tx=0x7fce6c0344f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df4c9254bb22f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c005220 con 0x55784d99c6b0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fce6c005380 con 0x55784d99c6b0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce700a1b30 con 0x55784d99c6b0 2023-07-21T22:38:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c034a70 con 0x55784d99c6b0 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 10 client.17832 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55784d8e61f0 con 0x55784d99c6b0 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce700a1b30 con 0x55784d99c6b0 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce6c004500 con 0x55784d99c6b0 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 10 client.17832 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.378+0000 7fce8b15fb80 20 client.17832 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784dbb71f0 0x55784dbb95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce8b15fb80 4 client.17832 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55784da4cff0 con 0x55784dbb71f0 2023-07-21T22:38:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce77fff700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784dbb71f0 0x55784dbb95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce57fff700 10 client.17832 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.379+0000 7fce77fff700 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784dbb71f0 0x55784dbb95d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a75864ec9b5a9d1 server_cookie=1f207d7f1f9ce6f0 in_seq=0 out_seq=0 2023-07-21T22:38:03.540 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302525 30.33 MB/sec execute 399 sec latency 2210.059 ms 2023-07-21T22:38:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce57fff700 1 -- 172.21.15.93:0/3829543086 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55784da4cff0 con 0x55784dbb71f0 2023-07-21T22:38:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce57fff700 10 client.17832 handle_command_reply: tid=0 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 client.17832 shutdown 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 msgr2=0x55784db50bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009350 tx=0x7fce6c0344f0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 shutdown_connections 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55784dbb71f0 0x55784dbb95d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55784db46660 0x55784db41d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55784d99c6b0 0x55784db50bb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 --2- 172.21.15.93:0/3829543086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55784db3ab60 0x55784db417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 >> 172.21.15.93:0/3829543086 conn(0x55784db35ae0 msgr2=0x55784db44930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.628+0000 7fce57fff700 0 client.17832 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 shutdown_connections 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce8b15fb80 1 -- 172.21.15.93:0/3829543086 wait complete. 2023-07-21T22:38:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce8b15fb80 20 client.17832 trim_cache size 0 max 16384 2023-07-21T22:38:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:38:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:38:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:38:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:38:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 msgr2=0x7fce680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 0x7fce680a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce6c006010 tx=0x7fce6c009f10 comp rx=0 tx=0).stop 2023-07-21T22:38:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 msgr2=0x7fce781a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fce700094f0 tx=0x7fce7000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 shutdown_connections 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fce680a12c0 0x7fce680a3770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce781039b0 0x7fce781a1a70 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fce780ff100 0x7fce78196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 --2- 172.21.15.93:0/617377492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce780ffab0 0x7fce78197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 >> 172.21.15.93:0/617377492 conn(0x7fce78075f90 msgr2=0x7fce780fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 shutdown_connections 2023-07-21T22:38:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:03.629+0000 7fce7efda700 1 -- 172.21.15.93:0/617377492 wait complete. 2023-07-21T22:38:03.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:03 smithi134 ceph-mon[97873]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 425 op/s 2023-07-21T22:38:03.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:03 smithi093 ceph-mon[96579]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 425 op/s 2023-07-21T22:38:04.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:03 smithi194 ceph-mon[98288]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 425 op/s 2023-07-21T22:38:04.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302525 30.25 MB/sec execute 400 sec latency 1434.048 ms 2023-07-21T22:38:05.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302525 30.17 MB/sec execute 401 sec latency 2434.835 ms 2023-07-21T22:38:05.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:05 smithi194 ceph-mon[98288]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 495 op/s 2023-07-21T22:38:05.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:05 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:05.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:05 smithi134 ceph-mon[97873]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 495 op/s 2023-07-21T22:38:05.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:05 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:05.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:05 smithi093 ceph-mon[96579]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.6 MiB/s wr, 495 op/s 2023-07-21T22:38:05.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:05 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:06.542 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302525 30.10 MB/sec execute 402 sec latency 3434.903 ms 2023-07-21T22:38:06.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:06 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:06.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:06 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:06.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:06 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:38:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:06 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:06 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:06.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:06 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:38:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:06 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:06 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:06.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:06 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:38:07.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302727 30.04 MB/sec execute 403 sec latency 3880.448 ms 2023-07-21T22:38:07.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:07 smithi194 ceph-mon[98288]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.9 MiB/s wr, 317 op/s 2023-07-21T22:38:07.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:07 smithi134 ceph-mon[97873]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.9 MiB/s wr, 317 op/s 2023-07-21T22:38:07.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:07 smithi093 ceph-mon[96579]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.9 MiB/s wr, 317 op/s 2023-07-21T22:38:08.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 302957 30.00 MB/sec execute 404 sec latency 155.531 ms 2023-07-21T22:38:09.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 303080 29.94 MB/sec execute 405 sec latency 186.584 ms 2023-07-21T22:38:09.881 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:09 smithi134 ceph-mon[97873]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:38:09.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:09 smithi093 ceph-mon[96579]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:38:10.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:09 smithi194 ceph-mon[98288]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2023-07-21T22:38:10.542 INFO:tasks.workunit.client.0.smithi093.stdout: 10 303175 29.89 MB/sec execute 406 sec latency 684.307 ms 2023-07-21T22:38:10.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:10 smithi134 ceph-mon[97873]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T22:38:10.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:10 smithi093 ceph-mon[96579]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T22:38:11.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:10 smithi194 ceph-mon[98288]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T22:38:11.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 303370 29.83 MB/sec execute 407 sec latency 858.933 ms 2023-07-21T22:38:11.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:11 smithi093 ceph-mon[96579]: pgmap v693: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2023-07-21T22:38:12.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:11 smithi194 ceph-mon[98288]: pgmap v693: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2023-07-21T22:38:12.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:11 smithi134 ceph-mon[97873]: pgmap v693: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2023-07-21T22:38:12.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 303785 29.80 MB/sec execute 408 sec latency 601.048 ms 2023-07-21T22:38:13.541 INFO:tasks.workunit.client.0.smithi093.stdout: 10 304374 29.81 MB/sec execute 409 sec latency 92.548 ms 2023-07-21T22:38:13.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:13 smithi093 ceph-mon[96579]: pgmap v694: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 413 op/s 2023-07-21T22:38:14.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:13 smithi194 ceph-mon[98288]: pgmap v694: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 413 op/s 2023-07-21T22:38:14.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:13 smithi134 ceph-mon[97873]: pgmap v694: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 413 op/s 2023-07-21T22:38:14.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 304991 29.82 MB/sec execute 410 sec latency 177.994 ms 2023-07-21T22:38:15.544 INFO:tasks.workunit.client.0.smithi093.stdout: 10 305015 29.75 MB/sec execute 411 sec latency 1133.537 ms 2023-07-21T22:38:15.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:15 smithi093 ceph-mon[96579]: pgmap v695: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2023-07-21T22:38:16.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:15 smithi194 ceph-mon[98288]: pgmap v695: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2023-07-21T22:38:16.132 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:15 smithi134 ceph-mon[97873]: pgmap v695: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2023-07-21T22:38:16.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 305225 29.72 MB/sec execute 412 sec latency 1225.610 ms 2023-07-21T22:38:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 -- 172.21.15.93:0/3097535488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9718108120 msgr2=0x7f971810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 --2- 172.21.15.93:0/3097535488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9718108120 0x7f971810c580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9710013930 tx=0x7f9710013c40 comp rx=0 tx=0).stop 2023-07-21T22:38:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 -- 172.21.15.93:0/3097535488 shutdown_connections 2023-07-21T22:38:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 --2- 172.21.15.93:0/3097535488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9718108120 0x7f971810c580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 --2- 172.21.15.93:0/3097535488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f9718107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 --2- 172.21.15.93:0/3097535488 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f971806c7a0 0x7f9718111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.807+0000 7f971f6e7700 1 -- 172.21.15.93:0/3097535488 >> 172.21.15.93:0/3097535488 conn(0x7f971806bd70 msgr2=0x7f9718070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.815+0000 7f971f6e7700 1 -- 172.21.15.93:0/3097535488 shutdown_connections 2023-07-21T22:38:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.818+0000 7f971f6e7700 1 -- 172.21.15.93:0/3097535488 wait complete. 2023-07-21T22:38:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.819+0000 7f971f6e7700 1 Processor -- start 2023-07-21T22:38:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 -- start start 2023-07-21T22:38:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971806c7a0 0x7f971819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f971819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 0x7f97181a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a14b0 con 0x7f9718108120 2023-07-21T22:38:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a1620 con 0x7f971806c7a0 2023-07-21T22:38:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.822+0000 7f971f6e7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181a17e0 con 0x7f9718107780 2023-07-21T22:38:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.823+0000 7f971cc82700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f971819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.823+0000 7f971cc82700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f971819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36836/0 (socket says 172.21.15.93:36836) 2023-07-21T22:38:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.823+0000 7f971cc82700 1 -- 172.21.15.93:0/1581421745 learned_addr learned my addr 172.21.15.93:0/1581421745 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:16.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.823+0000 7f971d483700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971806c7a0 0x7f971819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.823+0000 7f971dc84700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 0x7f97181a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 -- 172.21.15.93:0/1581421745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 msgr2=0x7f971819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f971819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 -- 172.21.15.93:0/1581421745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971806c7a0 msgr2=0x7f971819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971806c7a0 0x7f971819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 -- 172.21.15.93:0/1581421745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9710009d00 con 0x7f9718108120 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f971dc84700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 0x7f97181a0e70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9710013930 tx=0x7f97100078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f970e7fc700 1 -- 172.21.15.93:0/1581421745 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710003e10 con 0x7f9718108120 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f970e7fc700 1 -- 172.21.15.93:0/1581421745 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9710015ea0 con 0x7f9718108120 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.824+0000 7f970e7fc700 1 -- 172.21.15.93:0/1581421745 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97100052d0 con 0x7f9718108120 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.825+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97181a1a60 con 0x7f9718108120 2023-07-21T22:38:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.825+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97181a1ce0 con 0x7f9718108120 2023-07-21T22:38:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.828+0000 7f970e7fc700 1 -- 172.21.15.93:0/1581421745 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9710007d50 con 0x7f9718108120 2023-07-21T22:38:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.829+0000 7f970e7fc700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 0x7f97040a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.829+0000 7f970e7fc700 1 -- 172.21.15.93:0/1581421745 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f971000e070 con 0x7f9718108120 2023-07-21T22:38:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.830+0000 7f972b86cb80 1 Processor -- start 2023-07-21T22:38:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 -- start start 2023-07-21T22:38:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4660 0x55c459dbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459cc3c50 con 0x55c459dc4660 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f971d483700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 0x7f97040a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459d169e0 con 0x55c459dc4210 2023-07-21T22:38:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.832+0000 7f972b86cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459d0fe70 con 0x55c459c1a6b0 2023-07-21T22:38:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.833+0000 7f971d483700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.833+0000 7f971d483700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36844/0 (socket says 172.21.15.93:36844) 2023-07-21T22:38:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.833+0000 7f971d483700 1 -- 172.21.15.93:0/1315020293 learned_addr learned my addr 172.21.15.93:0/1315020293 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 -- 172.21.15.93:0/1315020293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 msgr2=0x55c459dbdf40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 -- 172.21.15.93:0/1315020293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4660 msgr2=0x55c459dbe4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4660 0x55c459dbe4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 -- 172.21.15.93:0/1315020293 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c459b3dd80 con 0x55c459c1a6b0 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 0x7f97040a3830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9714009e80 tx=0x7f9714014570 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f971d483700 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9714014970 tx=0x7f971401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3d61c472c7251a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f97037fe700 1 -- 172.21.15.93:0/1315020293 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714021460 con 0x55c459c1a6b0 2023-07-21T22:38:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f97037fe700 1 -- 172.21.15.93:0/1315020293 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9714021a80 con 0x55c459c1a6b0 2023-07-21T22:38:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.834+0000 7f97037fe700 1 -- 172.21.15.93:0/1315020293 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97140168b0 con 0x55c459c1a6b0 2023-07-21T22:38:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.835+0000 7f972b86cb80 1 -- 172.21.15.93:0/1315020293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 msgr2=0x55c459dc3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.835+0000 7f972b86cb80 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9714014970 tx=0x7f971401b000 comp rx=0 tx=0).stop 2023-07-21T22:38:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 -- 172.21.15.93:0/1315020293 shutdown_connections 2023-07-21T22:38:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4660 0x55c459dbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459c1a6b0 0x55c459dc3cd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 --2- 172.21.15.93:0/1315020293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 -- 172.21.15.93:0/1315020293 >> 172.21.15.93:0/1315020293 conn(0x55c459db3ae0 msgr2=0x55c459db90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 -- 172.21.15.93:0/1315020293 shutdown_connections 2023-07-21T22:38:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 -- 172.21.15.93:0/1315020293 wait complete. 2023-07-21T22:38:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 -- start start 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.838+0000 7f972b86cb80 1 Processor -- start 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4210 0x55c459dc77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4660 0x55c459dc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459cafa60 con 0x55c459c1a6b0 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459caf760 con 0x55c459dc4660 2023-07-21T22:38:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459b938c0 con 0x55c459dc4210 2023-07-21T22:38:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f972b86cb80 10 client.0 _ll_register_callbacks cb 0x55c459cad700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49766/0 (socket says 172.21.15.93:49766) 2023-07-21T22:38:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 -- 172.21.15.93:0/394728985 learned_addr learned my addr 172.21.15.93:0/394728985 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971cc82700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4210 0x55c459dc77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 -- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4210 msgr2=0x55c459dc77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4210 0x55c459dc77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 -- 172.21.15.93:0/394728985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4660 msgr2=0x55c459dc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4660 0x55c459dc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.839+0000 7f971d483700 1 -- 172.21.15.93:0/394728985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9714014c00 con 0x55c459c1a6b0 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f971d483700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9714021ea0 tx=0x7f9714009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68feea0ccbada87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f9700ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714005e10 con 0x55c459c1a6b0 2023-07-21T22:38:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c459c69490 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f9714009970 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9714021460 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f972b86cb80 10 client.17838 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55c459b26c00 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.840+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971403a020 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9714003720 con 0x55c459c1a6b0 2023-07-21T22:38:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 10 client.17838 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 20 client.17838 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459dd79e0 0x55c459dd9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 4 client.17838 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c459d08e50 con 0x55c459dd79e0 2023-07-21T22:38:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.841+0000 7f971cc82700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459dd79e0 0x55c459dd9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.843+0000 7f9700ff9700 10 client.17838 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.843+0000 7f971cc82700 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459dd79e0 0x55c459dd9df0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38fecfadddeed00f server_cookie=44e16a19c275051f in_seq=0 out_seq=0 2023-07-21T22:38:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.846+0000 7f9700ff9700 1 -- 172.21.15.93:0/394728985 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c459d08e50 con 0x55c459dd79e0 2023-07-21T22:38:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.846+0000 7f9700ff9700 10 client.17838 handle_command_reply: tid=0 2023-07-21T22:38:16.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.846+0000 7f972b86cb80 1 client.17838 shutdown 2023-07-21T22:38:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.852+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 msgr2=0x55c459dca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.852+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9714021ea0 tx=0x7f9714009120 comp rx=0 tx=0).stop 2023-07-21T22:38:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 shutdown_connections 2023-07-21T22:38:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459dd79e0 0x55c459dd9df0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4660 0x55c459dc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dca570 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 --2- 172.21.15.93:0/394728985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4210 0x55c459dc77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 >> 172.21.15.93:0/394728985 conn(0x55c459db3ae0 msgr2=0x55c459db83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f9700ff9700 0 client.17838 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 shutdown_connections 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.853+0000 7f972b86cb80 1 -- 172.21.15.93:0/394728985 wait complete. 2023-07-21T22:38:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.854+0000 7f972b86cb80 20 client.17838 trim_cache size 0 max 16384 2023-07-21T22:38:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 Processor -- start 2023-07-21T22:38:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 -- start start 2023-07-21T22:38:16.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459dbea90 con 0x55c459c1a6b0 2023-07-21T22:38:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459db9f10 con 0x55c459dc4210 2023-07-21T22:38:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.878+0000 7f972b86cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459dc24b0 con 0x55c459dc4660 2023-07-21T22:38:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49778/0 (socket says 172.21.15.93:49778) 2023-07-21T22:38:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 -- 172.21.15.93:0/2962309381 learned_addr learned my addr 172.21.15.93:0/2962309381 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 -- 172.21.15.93:0/2962309381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 msgr2=0x55c459dca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 -- 172.21.15.93:0/2962309381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 msgr2=0x55c459dbdf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 -- 172.21.15.93:0/2962309381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c459b641f0 con 0x55c459c1a6b0 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.881+0000 7f971d483700 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9714000c00 tx=0x7f9714016490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcde9fef84359f3f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.882+0000 7f9703fff700 1 -- 172.21.15.93:0/2962309381 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714009800 con 0x55c459c1a6b0 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.882+0000 7f9703fff700 1 -- 172.21.15.93:0/2962309381 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f971403a040 con 0x55c459c1a6b0 2023-07-21T22:38:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.882+0000 7f9703fff700 1 -- 172.21.15.93:0/2962309381 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714016ba0 con 0x55c459c1a6b0 2023-07-21T22:38:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.882+0000 7f972b86cb80 1 -- 172.21.15.93:0/2962309381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 msgr2=0x55c459dceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:16.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.882+0000 7f972b86cb80 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9714000c00 tx=0x7f9714016490 comp rx=0 tx=0).stop 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 -- 172.21.15.93:0/2962309381 shutdown_connections 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459c1a6b0 0x55c459dceba0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 --2- 172.21.15.93:0/2962309381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459dc4210 0x55c459dbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 -- 172.21.15.93:0/2962309381 >> 172.21.15.93:0/2962309381 conn(0x55c459db3ae0 msgr2=0x55c459dc1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 -- 172.21.15.93:0/2962309381 shutdown_connections 2023-07-21T22:38:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 -- 172.21.15.93:0/2962309381 wait complete. 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 -- start start 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.883+0000 7f972b86cb80 1 Processor -- start 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459c1a6b0 0x55c459dc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dbf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459dbfc70 con 0x55c459dc4210 2023-07-21T22:38:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459dbfde0 con 0x55c459c1a6b0 2023-07-21T22:38:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459dbffa0 con 0x55c459dc4660 2023-07-21T22:38:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.884+0000 7f972b86cb80 10 client.0 _ll_register_callbacks cb 0x7f9714014db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.887+0000 7f971cc82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.887+0000 7f971cc82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49780/0 (socket says 172.21.15.93:49780) 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.887+0000 7f971cc82700 1 -- 172.21.15.93:0/4161382243 learned_addr learned my addr 172.21.15.93:0/4161382243 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 -- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 msgr2=0x55c459dbf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dbf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 -- 172.21.15.93:0/4161382243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459c1a6b0 msgr2=0x55c459dc09e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459c1a6b0 0x55c459dc09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 -- 172.21.15.93:0/4161382243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c459b26c00 con 0x55c459dc4210 2023-07-21T22:38:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.888+0000 7f971cc82700 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9708000c00 tx=0x7f9708008a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5b667aa3de564f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.893+0000 7f9700ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.893+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708016040 con 0x55c459dc4210 2023-07-21T22:38:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.893+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f9708004030 con 0x55c459dc4210 2023-07-21T22:38:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.896+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9714021a80 con 0x55c459dc4210 2023-07-21T22:38:16.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:16 smithi093 ceph-mon[96579]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2023-07-21T22:38:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.896+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97080041f0 con 0x55c459dc4210 2023-07-21T22:38:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.898+0000 7f972b86cb80 10 client.17844 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.898+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55c459b3c250 con 0x55c459dc4210 2023-07-21T22:38:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9714021a80 con 0x55c459dc4210 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f97080105a0 con 0x55c459dc4210 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 10 client.17844 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 10 client.17844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 20 client.17844 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459e4e770 0x55c459e50b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 4 client.17844 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.899+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c459ccaff0 con 0x55c459e4e770 2023-07-21T22:38:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.901+0000 7f971d483700 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459e4e770 0x55c459e50b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.902+0000 7f9700ff9700 10 client.17844 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:16.902+0000 7f971d483700 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459e4e770 0x55c459e50b80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3630661e6e67177 server_cookie=b411d228abb43f1a in_seq=0 out_seq=0 2023-07-21T22:38:17.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:16 smithi194 ceph-mon[98288]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2023-07-21T22:38:17.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:16 smithi134 ceph-mon[97873]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 484 op/s 2023-07-21T22:38:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f9700ff9700 1 -- 172.21.15.93:0/4161382243 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c459ccaff0 con 0x55c459e4e770 2023-07-21T22:38:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f9700ff9700 10 client.17844 handle_command_reply: tid=0 2023-07-21T22:38:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 client.17844 shutdown 2023-07-21T22:38:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 msgr2=0x55c459dc0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9708000c00 tx=0x7f9708008a50 comp rx=0 tx=0).stop 2023-07-21T22:38:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 shutdown_connections 2023-07-21T22:38:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55c459e4e770 0x55c459e50b80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c459dc4660 0x55c459dbf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c459c1a6b0 0x55c459dc09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 --2- 172.21.15.93:0/4161382243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c459dc4210 0x55c459dc0f20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.217+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 >> 172.21.15.93:0/4161382243 conn(0x55c459db3ae0 msgr2=0x55c459dc4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.220+0000 7f9700ff9700 0 client.17844 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.220+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 shutdown_connections 2023-07-21T22:38:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.220+0000 7f972b86cb80 1 -- 172.21.15.93:0/4161382243 wait complete. 2023-07-21T22:38:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.220+0000 7f972b86cb80 20 client.17844 trim_cache size 0 max 16384 2023-07-21T22:38:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:38:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:38:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:38:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:38:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 msgr2=0x7f97040a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 0x7f97040a3830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9714009e80 tx=0x7f9714014570 comp rx=0 tx=0).stop 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 msgr2=0x7f97181a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 0x7f97181a0e70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9710013930 tx=0x7f97100078a0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 shutdown_connections 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f97040a1380 0x7f97040a3830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9718108120 0x7f97181a0e70 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971806c7a0 0x7f971819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 --2- 172.21.15.93:0/1581421745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9718107780 0x7f971819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 >> 172.21.15.93:0/1581421745 conn(0x7f971806bd70 msgr2=0x7f971806f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 shutdown_connections 2023-07-21T22:38:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:17.221+0000 7f971f6e7700 1 -- 172.21.15.93:0/1581421745 wait complete. 2023-07-21T22:38:17.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 305473 29.68 MB/sec execute 413 sec latency 233.897 ms 2023-07-21T22:38:18.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 305971 29.67 MB/sec execute 414 sec latency 141.653 ms 2023-07-21T22:38:18.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:18 smithi093 ceph-mon[96579]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T22:38:18.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:18 smithi194 ceph-mon[98288]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T22:38:18.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:18 smithi134 ceph-mon[97873]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T22:38:19.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 306435 29.67 MB/sec execute 415 sec latency 115.040 ms 2023-07-21T22:38:19.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:19 smithi093 ceph-mon[96579]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 926 op/s 2023-07-21T22:38:19.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:19 smithi194 ceph-mon[98288]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 926 op/s 2023-07-21T22:38:19.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:19 smithi134 ceph-mon[97873]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 20 MiB/s wr, 926 op/s 2023-07-21T22:38:20.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 306880 29.64 MB/sec execute 416 sec latency 87.464 ms 2023-07-21T22:38:21.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 307470 29.65 MB/sec execute 417 sec latency 113.849 ms 2023-07-21T22:38:21.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:21 smithi194 ceph-mon[98288]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 780 op/s 2023-07-21T22:38:21.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:21 smithi134 ceph-mon[97873]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 780 op/s 2023-07-21T22:38:21.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:21 smithi093 ceph-mon[96579]: pgmap v699: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 780 op/s 2023-07-21T22:38:22.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 308000 29.63 MB/sec execute 418 sec latency 145.278 ms 2023-07-21T22:38:23.546 INFO:tasks.workunit.client.0.smithi093.stdout: 10 308678 29.65 MB/sec execute 419 sec latency 114.011 ms 2023-07-21T22:38:23.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:23 smithi194 ceph-mon[98288]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 766 op/s 2023-07-21T22:38:23.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:23 smithi134 ceph-mon[97873]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 766 op/s 2023-07-21T22:38:23.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:23 smithi093 ceph-mon[96579]: pgmap v700: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 18 MiB/s wr, 766 op/s 2023-07-21T22:38:24.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 309234 29.66 MB/sec execute 420 sec latency 118.882 ms 2023-07-21T22:38:25.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 309734 29.66 MB/sec execute 421 sec latency 179.484 ms 2023-07-21T22:38:25.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:25 smithi194 ceph-mon[98288]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 20 MiB/s wr, 790 op/s 2023-07-21T22:38:25.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:25 smithi134 ceph-mon[97873]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 20 MiB/s wr, 790 op/s 2023-07-21T22:38:25.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:25 smithi093 ceph-mon[96579]: pgmap v701: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 20 MiB/s wr, 790 op/s 2023-07-21T22:38:26.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 310270 29.66 MB/sec execute 422 sec latency 76.872 ms 2023-07-21T22:38:27.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 310725 29.66 MB/sec execute 423 sec latency 144.084 ms 2023-07-21T22:38:27.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:27 smithi194 ceph-mon[98288]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 766 op/s 2023-07-21T22:38:27.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:27 smithi134 ceph-mon[97873]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 766 op/s 2023-07-21T22:38:27.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:27 smithi093 ceph-mon[96579]: pgmap v702: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 766 op/s 2023-07-21T22:38:28.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 311172 29.63 MB/sec execute 424 sec latency 141.773 ms 2023-07-21T22:38:29.547 INFO:tasks.workunit.client.0.smithi093.stdout: 10 311767 29.63 MB/sec execute 425 sec latency 117.948 ms 2023-07-21T22:38:29.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:29 smithi194 ceph-mon[98288]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 830 op/s 2023-07-21T22:38:29.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:29 smithi134 ceph-mon[97873]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 830 op/s 2023-07-21T22:38:29.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:29 smithi093 ceph-mon[96579]: pgmap v703: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 830 op/s 2023-07-21T22:38:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 -- 172.21.15.93:0/2348760685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c10a0c0 msgr2=0x7f482c10c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 --2- 172.21.15.93:0/2348760685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c10a0c0 0x7f482c10c530 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f482400b0a0 tx=0x7f482400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 -- 172.21.15.93:0/2348760685 shutdown_connections 2023-07-21T22:38:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 --2- 172.21.15.93:0/2348760685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c10a0c0 0x7f482c10c530 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 --2- 172.21.15.93:0/2348760685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 --2- 172.21.15.93:0/2348760685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c101690 0x7f482c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 -- 172.21.15.93:0/2348760685 >> 172.21.15.93:0/2348760685 conn(0x7f482c0fa570 msgr2=0x7f482c0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 -- 172.21.15.93:0/2348760685 shutdown_connections 2023-07-21T22:38:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.365+0000 7f48326cf700 1 -- 172.21.15.93:0/2348760685 wait complete. 2023-07-21T22:38:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 Processor -- start 2023-07-21T22:38:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 -- start start 2023-07-21T22:38:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c101690 0x7f482c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 0x7f482c195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1961a0 con 0x7f482c101690 2023-07-21T22:38:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c196310 con 0x7f482c100d80 2023-07-21T22:38:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f48326cf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c1964d0 con 0x7f482c10a0c0 2023-07-21T22:38:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f482bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f482bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60956/0 (socket says 172.21.15.93:60956) 2023-07-21T22:38:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f482bfff700 1 -- 172.21.15.93:0/1748850803 learned_addr learned my addr 172.21.15.93:0/1748850803 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.366+0000 7f4830c6c700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 0x7f482c195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 -- 172.21.15.93:0/1748850803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 msgr2=0x7f482c195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 0x7f482c195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 -- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c101690 msgr2=0x7f482c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c101690 0x7f482c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 -- 172.21.15.93:0/1748850803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824009d00 con 0x7f482c100d80 2023-07-21T22:38:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f4830c6c700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 0x7f482c195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f482bfff700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9a0 tx=0x7f481c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f4829ffb700 1 -- 172.21.15.93:0/1748850803 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c009790 con 0x7f482c100d80 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f4829ffb700 1 -- 172.21.15.93:0/1748850803 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c00cbd0 con 0x7f482c100d80 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f4829ffb700 1 -- 172.21.15.93:0/1748850803 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c018680 con 0x7f482c100d80 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.367+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c1967b0 con 0x7f482c100d80 2023-07-21T22:38:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.368+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c1a2cc0 con 0x7f482c100d80 2023-07-21T22:38:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.372+0000 7f483e854b80 1 Processor -- start 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 -- start start 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753210 0x56493274cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753660 0x56493274d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564932652c50 con 0x5649325a96b0 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649326a59e0 con 0x564932753210 2023-07-21T22:38:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f483e854b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493269ee70 con 0x564932753660 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49800/0 (socket says 172.21.15.93:49800) 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 -- 172.21.15.93:0/2895816018 learned_addr learned my addr 172.21.15.93:0/2895816018 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4823fff700 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753660 0x56493274d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4829ffb700 1 -- 172.21.15.93:0/1748850803 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f481c00cd40 con 0x7f482c100d80 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 -- 172.21.15.93:0/2895816018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753660 msgr2=0x56493274d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753660 0x56493274d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 -- 172.21.15.93:0/2895816018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753210 msgr2=0x56493274cf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753210 0x56493274cf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.373+0000 7f4830c6c700 1 -- 172.21.15.93:0/2895816018 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649324ccd80 con 0x5649325a96b0 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4824009c20 tx=0x7f4824009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce692e877d528a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f48227fc700 1 -- 172.21.15.93:0/2895816018 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400e040 con 0x5649325a96b0 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f48227fc700 1 -- 172.21.15.93:0/2895816018 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482400bcb0 con 0x5649325a96b0 2023-07-21T22:38:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f48227fc700 1 -- 172.21.15.93:0/2895816018 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824005240 con 0x5649325a96b0 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f4829ffb700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 0x7f48180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.374+0000 7f4829ffb700 1 -- 172.21.15.93:0/1748850803 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f481c014070 con 0x7f482c100d80 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.375+0000 7f482bfff700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 0x7f48180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.375+0000 7f482bfff700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 0x7f48180a37f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f481c099f50 tx=0x7f481c005760 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 -- 172.21.15.93:0/2895816018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 msgr2=0x564932752cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4824009c20 tx=0x7f4824009c50 comp rx=0 tx=0).stop 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 -- 172.21.15.93:0/2895816018 shutdown_connections 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753660 0x56493274d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932752cd0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 --2- 172.21.15.93:0/2895816018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753210 0x56493274cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.377+0000 7f483e854b80 1 -- 172.21.15.93:0/2895816018 >> 172.21.15.93:0/2895816018 conn(0x564932742ae0 msgr2=0x5649327480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.378+0000 7f483e854b80 1 -- 172.21.15.93:0/2895816018 shutdown_connections 2023-07-21T22:38:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.378+0000 7f483e854b80 1 -- 172.21.15.93:0/2895816018 wait complete. 2023-07-21T22:38:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 -- start start 2023-07-21T22:38:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 Processor -- start 2023-07-21T22:38:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753210 0x564932756840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753660 0x564932756ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493263ea60 con 0x5649325a96b0 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493263e760 con 0x564932753660 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649325228c0 con 0x564932753210 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f483e854b80 10 client.0 _ll_register_callbacks cb 0x56493263c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49808/0 (socket says 172.21.15.93:49808) 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f4830c6c700 1 -- 172.21.15.93:0/2215370285 learned_addr learned my addr 172.21.15.93:0/2215370285 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.379+0000 7f4823fff700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753210 0x564932756840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753660 0x564932756ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 -- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753210 msgr2=0x564932756840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753210 0x564932756840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 -- 172.21.15.93:0/2215370285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753660 msgr2=0x564932756ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753660 0x564932756ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 -- 172.21.15.93:0/2215370285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c10d1f0 con 0x5649325a96b0 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4824007630 tx=0x7f4824006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa992aa8311d08be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f480b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482404d070 con 0x5649325a96b0 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f48240054f0 con 0x5649325a96b0 2023-07-21T22:38:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649325f8490 con 0x5649325a96b0 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f482400e050 con 0x5649325a96b0 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f483e854b80 10 client.17853 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.380+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x5649324b5c00 con 0x5649325a96b0 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649325f8490 con 0x5649325a96b0 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4824052070 con 0x5649325a96b0 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 10 client.17853 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 10 client.17853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 20 client.17853 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932760710 0x564932762b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 4 client.17853 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564932697e50 con 0x564932760710 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.381+0000 7f4823fff700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932760710 0x564932762b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.382+0000 7f480b7fe700 10 client.17853 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.382+0000 7f4823fff700 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932760710 0x564932762b20 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c24428e308c9466 server_cookie=92304ff9fae66dc3 in_seq=0 out_seq=0 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.384+0000 7f480b7fe700 1 -- 172.21.15.93:0/2215370285 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564932697e50 con 0x564932760710 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.384+0000 7f480b7fe700 10 client.17853 handle_command_reply: tid=0 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.384+0000 7f483e854b80 1 client.17853 shutdown 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 msgr2=0x564932759550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4824007630 tx=0x7f4824006000 comp rx=0 tx=0).stop 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 shutdown_connections 2023-07-21T22:38:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932760710 0x564932762b20 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932753660 0x564932756ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x564932759550 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 --2- 172.21.15.93:0/2215370285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932753210 0x564932756840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 >> 172.21.15.93:0/2215370285 conn(0x564932742ae0 msgr2=0x5649327473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f480b7fe700 0 client.17853 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 shutdown_connections 2023-07-21T22:38:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 1 -- 172.21.15.93:0/2215370285 wait complete. 2023-07-21T22:38:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.385+0000 7f483e854b80 20 client.17853 trim_cache size 0 max 16384 2023-07-21T22:38:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 Processor -- start 2023-07-21T22:38:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 -- start start 2023-07-21T22:38:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932742ae0 0x564932747b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649325a96b0 0x5649327590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493275d460 con 0x564932751100 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493274e160 con 0x5649325a96b0 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f483e854b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564932744c00 con 0x564932742ae0 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49816/0 (socket says 172.21.15.93:49816) 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f4830c6c700 1 -- 172.21.15.93:0/3007020596 learned_addr learned my addr 172.21.15.93:0/3007020596 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f4823fff700 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932742ae0 0x564932747b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.405+0000 7f4830c6c700 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649325a96b0 0x5649327590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 -- 172.21.15.93:0/3007020596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932742ae0 msgr2=0x564932747b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932742ae0 0x564932747b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 -- 172.21.15.93:0/3007020596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649325a96b0 msgr2=0x5649327590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649325a96b0 0x5649327590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 -- 172.21.15.93:0/3007020596 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649324f31f0 con 0x564932751100 2023-07-21T22:38:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4830c6c700 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4824029090 tx=0x7f4824000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b627277eb0a3515 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4822ffd700 1 -- 172.21.15.93:0/3007020596 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482404d070 con 0x564932751100 2023-07-21T22:38:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4822ffd700 1 -- 172.21.15.93:0/3007020596 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4824025910 con 0x564932751100 2023-07-21T22:38:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.406+0000 7f4822ffd700 1 -- 172.21.15.93:0/3007020596 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400e040 con 0x564932751100 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- 172.21.15.93:0/3007020596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 msgr2=0x564932767440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4824029090 tx=0x7f4824000f40 comp rx=0 tx=0).stop 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- 172.21.15.93:0/3007020596 shutdown_connections 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649325a96b0 0x5649327590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564932751100 0x564932767440 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 --2- 172.21.15.93:0/3007020596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932742ae0 0x564932747b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- 172.21.15.93:0/3007020596 >> 172.21.15.93:0/3007020596 conn(0x564932750d30 msgr2=0x56493275c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- 172.21.15.93:0/3007020596 shutdown_connections 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- 172.21.15.93:0/3007020596 wait complete. 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 -- start start 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.407+0000 7f483e854b80 1 Processor -- start 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932742ae0 0x56493274fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932751100 0x564932755620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564932750390 con 0x5649325a96b0 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564932750500 con 0x564932742ae0 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x564932755b60 con 0x564932751100 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f483e854b80 10 client.0 _ll_register_callbacks cb 0x7f48140095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49822/0 (socket says 172.21.15.93:49822) 2023-07-21T22:38:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 -- 172.21.15.93:0/2047842582 learned_addr learned my addr 172.21.15.93:0/2047842582 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4823fff700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932742ae0 0x56493274fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932751100 0x564932755620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 -- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932751100 msgr2=0x564932755620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932751100 0x564932755620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 -- 172.21.15.93:0/2047842582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932742ae0 msgr2=0x56493274fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932742ae0 0x56493274fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.408+0000 7f4830c6c700 1 -- 172.21.15.93:0/2047842582 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649324b5c00 con 0x5649325a96b0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f4830c6c700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f482400b070 tx=0x7f482400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48021d0365767df9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482404d040 con 0x5649325a96b0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4824025d00 con 0x5649325a96b0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564932659ff0 con 0x5649325a96b0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4824006000 con 0x5649325a96b0 2023-07-21T22:38:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 10 client.17859 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x5649324cb250 con 0x5649325a96b0 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564932659ff0 con 0x5649325a96b0 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4824052070 con 0x5649325a96b0 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 10 client.17859 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 20 client.17859 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932753210 0x5649327bd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 4 client.17859 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.409+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649324edf10 con 0x564932753210 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.410+0000 7f4823fff700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932753210 0x5649327bd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.410+0000 7f4823fff700 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932753210 0x5649327bd030 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37ba88708fe80e8 server_cookie=8d5ea2574a6fceb3 in_seq=0 out_seq=0 2023-07-21T22:38:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.410+0000 7f480b7fe700 10 client.17859 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:30.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 312401 29.65 MB/sec execute 426 sec latency 89.119 ms 2023-07-21T22:38:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.692+0000 7f480b7fe700 1 -- 172.21.15.93:0/2047842582 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649324edf10 con 0x564932753210 2023-07-21T22:38:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.692+0000 7f480b7fe700 10 client.17859 handle_command_reply: tid=0 2023-07-21T22:38:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.692+0000 7f483e854b80 1 client.17859 shutdown 2023-07-21T22:38:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 msgr2=0x56493274f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f482400b070 tx=0x7f482400bc10 comp rx=0 tx=0).stop 2023-07-21T22:38:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 shutdown_connections 2023-07-21T22:38:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x564932753210 0x5649327bd030 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564932751100 0x564932755620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649325a96b0 0x56493274f740 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 --2- 172.21.15.93:0/2047842582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564932742ae0 0x56493274fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.695+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 >> 172.21.15.93:0/2047842582 conn(0x564932750d30 msgr2=0x564932753aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f480b7fe700 0 client.17859 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 shutdown_connections 2023-07-21T22:38:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f483e854b80 1 -- 172.21.15.93:0/2047842582 wait complete. 2023-07-21T22:38:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f483e854b80 20 client.17859 trim_cache size 0 max 16384 2023-07-21T22:38:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:38:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:38:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:38:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:38:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 msgr2=0x7f48180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 0x7f48180a37f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f481c099f50 tx=0x7f481c005760 comp rx=0 tx=0).stop 2023-07-21T22:38:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 msgr2=0x7f482c19bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f481c00e9a0 tx=0x7f481c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.697+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 shutdown_connections 2023-07-21T22:38:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f482c10a0c0 0x7f482c195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f482c100d80 0x7f482c19bcd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f48180a1340 0x7f48180a37f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 --2- 172.21.15.93:0/1748850803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f482c101690 0x7f482c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 >> 172.21.15.93:0/1748850803 conn(0x7f482c0fa570 msgr2=0x7f482c101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 shutdown_connections 2023-07-21T22:38:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:30.698+0000 7f48326cf700 1 -- 172.21.15.93:0/1748850803 wait complete. 2023-07-21T22:38:31.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 312998 29.64 MB/sec execute 427 sec latency 246.476 ms 2023-07-21T22:38:31.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:31 smithi194 ceph-mon[98288]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 408 op/s 2023-07-21T22:38:31.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:31 smithi134 ceph-mon[97873]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 408 op/s 2023-07-21T22:38:31.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:31 smithi093 ceph-mon[96579]: pgmap v704: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 408 op/s 2023-07-21T22:38:32.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 313628 29.66 MB/sec execute 428 sec latency 82.856 ms 2023-07-21T22:38:33.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 314130 29.67 MB/sec execute 429 sec latency 170.768 ms 2023-07-21T22:38:33.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:33 smithi194 ceph-mon[98288]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 462 op/s 2023-07-21T22:38:33.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:33 smithi134 ceph-mon[97873]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 462 op/s 2023-07-21T22:38:33.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:33 smithi093 ceph-mon[96579]: pgmap v705: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 462 op/s 2023-07-21T22:38:34.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 314534 29.65 MB/sec execute 430 sec latency 73.837 ms 2023-07-21T22:38:35.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 315039 29.64 MB/sec execute 431 sec latency 98.790 ms 2023-07-21T22:38:35.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:35 smithi194 ceph-mon[98288]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 609 op/s 2023-07-21T22:38:35.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:35 smithi134 ceph-mon[97873]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 609 op/s 2023-07-21T22:38:35.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:35 smithi093 ceph-mon[96579]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 609 op/s 2023-07-21T22:38:36.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 315551 29.63 MB/sec execute 432 sec latency 100.227 ms 2023-07-21T22:38:37.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 316247 29.63 MB/sec execute 433 sec latency 74.511 ms 2023-07-21T22:38:37.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:37 smithi194 ceph-mon[98288]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 453 op/s 2023-07-21T22:38:37.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:37 smithi134 ceph-mon[97873]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 453 op/s 2023-07-21T22:38:37.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:37 smithi093 ceph-mon[96579]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.4 MiB/s wr, 453 op/s 2023-07-21T22:38:38.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 316894 29.65 MB/sec execute 434 sec latency 138.674 ms 2023-07-21T22:38:39.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 317405 29.64 MB/sec execute 435 sec latency 276.212 ms 2023-07-21T22:38:39.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:39 smithi194 ceph-mon[98288]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 626 op/s 2023-07-21T22:38:39.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:39 smithi134 ceph-mon[97873]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 626 op/s 2023-07-21T22:38:39.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:39 smithi093 ceph-mon[96579]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 626 op/s 2023-07-21T22:38:40.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 317847 29.65 MB/sec execute 436 sec latency 72.040 ms 2023-07-21T22:38:41.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 318308 29.65 MB/sec execute 437 sec latency 169.441 ms 2023-07-21T22:38:41.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:41 smithi194 ceph-mon[98288]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 451 op/s 2023-07-21T22:38:41.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:41 smithi134 ceph-mon[97873]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 451 op/s 2023-07-21T22:38:41.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:41 smithi093 ceph-mon[96579]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.4 MiB/s wr, 451 op/s 2023-07-21T22:38:42.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 318700 29.62 MB/sec execute 438 sec latency 83.350 ms 2023-07-21T22:38:43.548 INFO:tasks.workunit.client.0.smithi093.stdout: 10 319254 29.61 MB/sec execute 439 sec latency 88.726 ms 2023-07-21T22:38:43.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:43 smithi194 ceph-mon[98288]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 488 op/s 2023-07-21T22:38:43.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:43 smithi134 ceph-mon[97873]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 488 op/s 2023-07-21T22:38:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.876+0000 7f4e06034700 1 -- 172.21.15.93:0/3030336433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010cff0 msgr2=0x7f4e001137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.876+0000 7f4e06034700 1 --2- 172.21.15.93:0/3030336433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010cff0 0x7f4e001137c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4df40099c0 tx=0x7f4df4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 -- 172.21.15.93:0/3030336433 shutdown_connections 2023-07-21T22:38:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 --2- 172.21.15.93:0/3030336433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010cff0 0x7f4e001137c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 --2- 172.21.15.93:0/3030336433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 --2- 172.21.15.93:0/3030336433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010c640 0x7f4e0010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 -- 172.21.15.93:0/3030336433 >> 172.21.15.93:0/3030336433 conn(0x7f4e0006bd70 msgr2=0x7f4e00070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 -- 172.21.15.93:0/3030336433 shutdown_connections 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.877+0000 7f4e06034700 1 -- 172.21.15.93:0/3030336433 wait complete. 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 Processor -- start 2023-07-21T22:38:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 -- start start 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010c640 0x7f4e0011c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 0x7f4e00116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00116a60 con 0x7f4e0010cff0 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00116bd0 con 0x7f4e0010b970 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.878+0000 7f4e06034700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00116d90 con 0x7f4e0010c640 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 0x7f4e00116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:60996/0 (socket says 172.21.15.93:60996) 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dff7fe700 1 -- 172.21.15.93:0/2802685380 learned_addr learned my addr 172.21.15.93:0/2802685380 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 -- 172.21.15.93:0/2802685380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010c640 msgr2=0x7f4e0011c900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010c640 0x7f4e0011c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 -- 172.21.15.93:0/2802685380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 msgr2=0x7f4e0011c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 -- 172.21.15.93:0/2802685380 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40095f0 con 0x7f4e0010cff0 2023-07-21T22:38:43.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.879+0000 7f4dfffff700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 0x7f4e00116320 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4df40098e0 tx=0x7f4df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.880+0000 7f4dfcff9700 1 -- 172.21.15.93:0/2802685380 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df401d070 con 0x7f4e0010cff0 2023-07-21T22:38:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.880+0000 7f4dfcff9700 1 -- 172.21.15.93:0/2802685380 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df4004100 con 0x7f4e0010cff0 2023-07-21T22:38:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.880+0000 7f4dfcff9700 1 -- 172.21.15.93:0/2802685380 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4004a00 con 0x7f4e0010cff0 2023-07-21T22:38:43.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.880+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e001b9ef0 con 0x7f4e0010cff0 2023-07-21T22:38:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.880+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e001ba260 con 0x7f4e0010cff0 2023-07-21T22:38:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.885+0000 7f4e121b9b80 1 Processor -- start 2023-07-21T22:38:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.886+0000 7f4dfcff9700 1 -- 172.21.15.93:0/2802685380 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4df4004b60 con 0x7f4e0010cff0 2023-07-21T22:38:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.887+0000 7f4dfcff9700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 0x7f4de80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.887+0000 7f4dfcff9700 1 -- 172.21.15.93:0/2802685380 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4df40cd8c0 con 0x7f4e0010cff0 2023-07-21T22:38:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.887+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 0x7f4de80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.887+0000 7f4e121b9b80 1 -- start start 2023-07-21T22:38:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55527cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab555224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab55427c50 con 0x55ab55528210 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5547a9e0 con 0x55ab55528660 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab55473e70 con 0x55ab5537e6b0 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 0x7f4de80a37b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e00117bf0 tx=0x7f4df0006c60 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49842/0 (socket says 172.21.15.93:49842) 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 -- 172.21.15.93:0/1332826273 learned_addr learned my addr 172.21.15.93:0/1332826273 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 -- 172.21.15.93:0/1332826273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 msgr2=0x55ab55527cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55527cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 -- 172.21.15.93:0/1332826273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 msgr2=0x55ab555224a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab555224a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.888+0000 7f4dfeffd700 1 -- 172.21.15.93:0/1332826273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab552a1d80 con 0x55ab55528210 2023-07-21T22:38:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.889+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4df00158f0 tx=0x7f4df000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103ac550c3f5ceb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.889+0000 7f4de5ffb700 1 -- 172.21.15.93:0/1332826273 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df001a070 con 0x55ab55528210 2023-07-21T22:38:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.889+0000 7f4de5ffb700 1 -- 172.21.15.93:0/1332826273 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df0012970 con 0x55ab55528210 2023-07-21T22:38:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.889+0000 7f4de5ffb700 1 -- 172.21.15.93:0/1332826273 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0013950 con 0x55ab55528210 2023-07-21T22:38:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.890+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1332826273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 msgr2=0x55ab55521f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.890+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4df00158f0 tx=0x7f4df000e000 comp rx=0 tx=0).stop 2023-07-21T22:38:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1332826273 shutdown_connections 2023-07-21T22:38:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab555224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55527cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1332826273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55521f40 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1332826273 >> 172.21.15.93:0/1332826273 conn(0x55ab55517ae0 msgr2=0x55ab5551d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1332826273 shutdown_connections 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1332826273 wait complete. 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.892+0000 7f4e121b9b80 1 -- start start 2023-07-21T22:38:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 Processor -- start 2023-07-21T22:38:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab5552e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528210 0x55ab5552b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab55413a60 con 0x55ab55528660 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab55413760 con 0x55ab55528210 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab552f78c0 con 0x55ab5537e6b0 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4e121b9b80 10 client.0 _ll_register_callbacks cb 0x55ab55411700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab5552e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab5552e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36926/0 (socket says 172.21.15.93:36926) 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dff7fe700 1 -- 172.21.15.93:0/4094260006 learned_addr learned my addr 172.21.15.93:0/4094260006 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528210 0x55ab5552b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.893+0000 7f4dfffff700 1 -- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 msgr2=0x55ab5552e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4dfffff700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab5552e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4dfffff700 1 -- 172.21.15.93:0/4094260006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528210 msgr2=0x55ab5552b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4dfffff700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528210 0x55ab5552b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4dfffff700 1 -- 172.21.15.93:0/4094260006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df0015660 con 0x55ab55528660 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4dfffff700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552bd30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4df40cb010 tx=0x7f4df409a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bf1b802c5543c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df409ea50 con 0x55ab55528660 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab553cd490 con 0x55ab55528660 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4df40a2de0 con 0x55ab55528660 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df40b3120 con 0x55ab55528660 2023-07-21T22:38:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4e121b9b80 10 client.17868 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x55ab5528ac00 con 0x55ab55528660 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.894+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab553cd490 con 0x55ab55528660 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4df4096810 con 0x55ab55528660 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 10 client.17868 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 20 client.17868 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab5553b790 0x55ab5553dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 4 client.17868 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab5546ce50 con 0x55ab5553b790 2023-07-21T22:38:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.895+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab5553b790 0x55ab5553dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.896+0000 7f4ddeffd700 10 client.17868 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.896+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab5553b790 0x55ab5553dba0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d071b556742dda server_cookie=d8c0d295d383102f in_seq=0 out_seq=0 2023-07-21T22:38:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.896+0000 7f4ddeffd700 1 -- 172.21.15.93:0/4094260006 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab5546ce50 con 0x55ab5553b790 2023-07-21T22:38:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.896+0000 7f4ddeffd700 10 client.17868 handle_command_reply: tid=0 2023-07-21T22:38:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.896+0000 7f4e121b9b80 1 client.17868 shutdown 2023-07-21T22:38:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 msgr2=0x55ab5552bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552bd30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4df40cb010 tx=0x7f4df409a650 comp rx=0 tx=0).stop 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 shutdown_connections 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab5553b790 0x55ab5553dba0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552bd30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab5552e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4ddeffd700 0 client.17868 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4094260006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528210 0x55ab5552b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 >> 172.21.15.93:0/4094260006 conn(0x55ab55517ae0 msgr2=0x55ab5551c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 shutdown_connections 2023-07-21T22:38:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4094260006 wait complete. 2023-07-21T22:38:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.897+0000 7f4e121b9b80 20 client.17868 trim_cache size 0 max 16384 2023-07-21T22:38:43.946 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:43 smithi093 ceph-mon[96579]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 488 op/s 2023-07-21T22:38:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 Processor -- start 2023-07-21T22:38:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 -- start start 2023-07-21T22:38:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 0x55ab55529180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5551e7a0 con 0x55ab55528660 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5551df70 con 0x55ab5537e6b0 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5552cbc0 con 0x55ab55528210 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4dfffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4dff7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32784/0 (socket says 172.21.15.93:32784) 2023-07-21T22:38:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4dfeffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 0x55ab55529180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.918+0000 7f4dff7fe700 1 -- 172.21.15.93:0/4164026844 learned_addr learned my addr 172.21.15.93:0/4164026844 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:43.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 -- 172.21.15.93:0/4164026844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 msgr2=0x55ab55529180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 0x55ab55529180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 -- 172.21.15.93:0/4164026844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 msgr2=0x55ab5552e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 -- 172.21.15.93:0/4164026844 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab552c81f0 con 0x55ab5537e6b0 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 0x55ab55529180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000f80 tx=0x7f4df8009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a389dfeb5372afde server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4de67fc700 1 -- 172.21.15.93:0/4164026844 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df801b150 con 0x55ab5537e6b0 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4de67fc700 1 -- 172.21.15.93:0/4164026844 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df800e030 con 0x55ab5537e6b0 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.919+0000 7f4de67fc700 1 -- 172.21.15.93:0/4164026844 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8017d60 con 0x55ab5537e6b0 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4164026844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 msgr2=0x55ab55521f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000f80 tx=0x7f4df8009790 comp rx=0 tx=0).stop 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4164026844 shutdown_connections 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528660 0x55ab5552e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab5537e6b0 0x55ab55521f40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/4164026844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab55528210 0x55ab55529180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4164026844 >> 172.21.15.93:0/4164026844 conn(0x55ab55517ae0 msgr2=0x55ab55528a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4164026844 shutdown_connections 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.920+0000 7f4e121b9b80 1 -- 172.21.15.93:0/4164026844 wait complete. 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 -- start start 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 Processor -- start 2023-07-21T22:38:43.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55531220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab55531ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab55533210 con 0x55ab55528210 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5558f620 con 0x55ab55528660 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5558f760 con 0x55ab5537e6b0 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4e121b9b80 10 client.0 _ll_register_callbacks cb 0x7f4df409a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4dfeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4dfeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49862/0 (socket says 172.21.15.93:49862) 2023-07-21T22:38:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4dfeffd700 1 -- 172.21.15.93:0/1670159932 learned_addr learned my addr 172.21.15.93:0/1670159932 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.921+0000 7f4dfffff700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab55531ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 -- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 msgr2=0x55ab55531220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:38:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55531220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 -- 172.21.15.93:0/1670159932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 msgr2=0x55ab55531760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 -- 172.21.15.93:0/1670159932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab552a1d80 con 0x55ab55528660 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfeffd700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4dfffff700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab55531ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4df409a720 tx=0x7f4df409a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cababa1a273879f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40a2b50 con 0x55ab55528660 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4df4020a10 con 0x55ab55528660 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.922+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4df0012970 con 0x55ab55528660 2023-07-21T22:38:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df40a2de0 con 0x55ab55528660 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 10 client.27194 fetch_fsmap learned FSMap version 86 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x7f4df80075b0 con 0x55ab55528660 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df0012970 con 0x55ab55528660 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 86) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4df40c8050 con 0x55ab55528660 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 86 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 20 client.27194 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab55590ab0 0x55ab55592f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 4 client.27194 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.923+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab5542eff0 con 0x55ab55590ab0 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.925+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab55590ab0 0x55ab55592f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.925+0000 7f4ddeffd700 10 client.27194 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:43.925+0000 7f4dff7fe700 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab55590ab0 0x55ab55592f40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1be7978593b9070 server_cookie=59874fb51ee84d66 in_seq=0 out_seq=0 2023-07-21T22:38:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.204+0000 7f4ddeffd700 1 -- 172.21.15.93:0/1670159932 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab5542eff0 con 0x55ab55590ab0 2023-07-21T22:38:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.204+0000 7f4ddeffd700 10 client.27194 handle_command_reply: tid=0 2023-07-21T22:38:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.204+0000 7f4e121b9b80 1 client.27194 shutdown 2023-07-21T22:38:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 msgr2=0x55ab55531ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab55531ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4df409a720 tx=0x7f4df409a560 comp rx=0 tx=0).stop 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 shutdown_connections 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ab55590ab0 0x55ab55592f40 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ab55528660 0x55ab55531ca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4ddeffd700 0 client.27194 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ab5537e6b0 0x55ab55531220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 --2- 172.21.15.93:0/1670159932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ab55528210 0x55ab55531760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 >> 172.21.15.93:0/1670159932 conn(0x55ab55517ae0 msgr2=0x55ab555292c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 shutdown_connections 2023-07-21T22:38:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.205+0000 7f4e121b9b80 1 -- 172.21.15.93:0/1670159932 wait complete. 2023-07-21T22:38:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.207+0000 7f4e121b9b80 20 client.27194 trim_cache size 0 max 16384 2023-07-21T22:38:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:38:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:38:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:38:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:38:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 msgr2=0x7f4de80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 0x7f4de80a37b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e00117bf0 tx=0x7f4df0006c60 comp rx=0 tx=0).stop 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 msgr2=0x7f4e00116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 0x7f4e00116320 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4df40098e0 tx=0x7f4df4004730 comp rx=0 tx=0).stop 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 shutdown_connections 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4de80a1300 0x7f4de80a37b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e0010cff0 0x7f4e00116320 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e0010b970 0x7f4e0011c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 --2- 172.21.15.93:0/2802685380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e0010c640 0x7f4e0011c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 >> 172.21.15.93:0/2802685380 conn(0x7f4e0006bd70 msgr2=0x7f4e00109670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 shutdown_connections 2023-07-21T22:38:44.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:44.208+0000 7f4e06034700 1 -- 172.21.15.93:0/2802685380 wait complete. 2023-07-21T22:38:44.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 319642 29.57 MB/sec execute 440 sec latency 157.527 ms 2023-07-21T22:38:45.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 320468 29.60 MB/sec execute 441 sec latency 90.421 ms 2023-07-21T22:38:45.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:45 smithi194 ceph-mon[98288]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2023-07-21T22:38:45.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:45 smithi134 ceph-mon[97873]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2023-07-21T22:38:45.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:45 smithi093 ceph-mon[96579]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 576 op/s 2023-07-21T22:38:46.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 321175 29.64 MB/sec execute 442 sec latency 60.018 ms 2023-07-21T22:38:47.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 321692 29.65 MB/sec execute 443 sec latency 67.725 ms 2023-07-21T22:38:47.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:47 smithi194 ceph-mon[98288]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2023-07-21T22:38:47.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:47 smithi134 ceph-mon[97873]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2023-07-21T22:38:47.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:47 smithi093 ceph-mon[96579]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.7 MiB/s wr, 374 op/s 2023-07-21T22:38:48.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 322072 29.63 MB/sec execute 444 sec latency 83.123 ms 2023-07-21T22:38:49.552 INFO:tasks.workunit.client.0.smithi093.stdout: 10 322491 29.61 MB/sec execute 445 sec latency 92.233 ms 2023-07-21T22:38:49.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:49 smithi194 ceph-mon[98288]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 587 op/s 2023-07-21T22:38:49.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:49 smithi134 ceph-mon[97873]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 587 op/s 2023-07-21T22:38:49.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:49 smithi093 ceph-mon[96579]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 587 op/s 2023-07-21T22:38:50.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 322814 29.57 MB/sec execute 446 sec latency 355.868 ms 2023-07-21T22:38:51.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 322832 29.50 MB/sec execute 447 sec latency 1222.038 ms 2023-07-21T22:38:51.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:51 smithi134 ceph-mon[97873]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 408 op/s 2023-07-21T22:38:51.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:51 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:38:51.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:51 smithi093 ceph-mon[96579]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 408 op/s 2023-07-21T22:38:51.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:51 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:38:52.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:51 smithi194 ceph-mon[98288]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.6 MiB/s wr, 408 op/s 2023-07-21T22:38:52.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:51 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:38:52.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 323072 29.46 MB/sec execute 448 sec latency 1308.700 ms 2023-07-21T22:38:53.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 323399 29.43 MB/sec execute 449 sec latency 164.302 ms 2023-07-21T22:38:53.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:53 smithi134 ceph-mon[97873]: pgmap v715: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 482 op/s 2023-07-21T22:38:53.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:53 smithi093 ceph-mon[96579]: pgmap v715: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 482 op/s 2023-07-21T22:38:54.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:53 smithi194 ceph-mon[98288]: pgmap v715: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 482 op/s 2023-07-21T22:38:54.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 323799 29.40 MB/sec execute 450 sec latency 147.855 ms 2023-07-21T22:38:55.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 324456 29.44 MB/sec execute 451 sec latency 109.011 ms 2023-07-21T22:38:55.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:55 smithi134 ceph-mon[97873]: pgmap v716: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 785 op/s 2023-07-21T22:38:55.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:55 smithi093 ceph-mon[96579]: pgmap v716: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 785 op/s 2023-07-21T22:38:56.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:55 smithi194 ceph-mon[98288]: pgmap v716: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 785 op/s 2023-07-21T22:38:56.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 325018 29.46 MB/sec execute 452 sec latency 71.940 ms 2023-07-21T22:38:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 -- 172.21.15.93:0/1703467488 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 msgr2=0x7fbd2c102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 --2- 172.21.15.93:0/1703467488 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c102d50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbd200099c0 tx=0x7fbd20009cd0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 -- 172.21.15.93:0/1703467488 shutdown_connections 2023-07-21T22:38:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 --2- 172.21.15.93:0/1703467488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 0x7fbd2c10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 --2- 172.21.15.93:0/1703467488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c0fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 --2- 172.21.15.93:0/1703467488 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c102d50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 -- 172.21.15.93:0/1703467488 >> 172.21.15.93:0/1703467488 conn(0x7fbd2c075f90 msgr2=0x7fbd2c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 -- 172.21.15.93:0/1703467488 shutdown_connections 2023-07-21T22:38:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.360+0000 7fbd3227d700 1 -- 172.21.15.93:0/1703467488 wait complete. 2023-07-21T22:38:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 Processor -- start 2023-07-21T22:38:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 -- start start 2023-07-21T22:38:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 0x7fbd2c1949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c195020 con 0x7fbd2c0fed90 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c195190 con 0x7fbd2c103430 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.361+0000 7fbd3227d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c195350 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36954/0 (socket says 172.21.15.93:36954) 2023-07-21T22:38:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 -- 172.21.15.93:0/3929393954 learned_addr learned my addr 172.21.15.93:0/3929393954 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd3081a700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 0x7fbd2c1949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 -- 172.21.15.93:0/3929393954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 msgr2=0x7fbd2c1949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 0x7fbd2c1949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 -- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 msgr2=0x7fbd2c19b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c19b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 -- 172.21.15.93:0/3929393954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200095f0 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e9a0 tx=0x7fbd1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd297fa700 1 -- 172.21.15.93:0/3929393954 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c009790 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd297fa700 1 -- 172.21.15.93:0/3929393954 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd1c010430 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c195630 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd297fa700 1 -- 172.21.15.93:0/3929393954 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c018680 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.409 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.362+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c0fbc80 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.365+0000 7fbd297fa700 1 -- 172.21.15.93:0/3929393954 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbd1c0105a0 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.366+0000 7fbd297fa700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 0x7fbd140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.366+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 0x7fbd140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.366+0000 7fbd297fa700 1 -- 172.21.15.93:0/3929393954 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbd1c014070 con 0x7fbd2c0fe3f0 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.366+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 0x7fbd140a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098e0 tx=0x7fbd20000bc0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 Processor -- start 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 -- start start 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a39cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 0x562b23a344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23939c50 con 0x562b23a3a210 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2398c9e0 con 0x562b23a3a660 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23985e70 con 0x562b238906b0 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49884/0 (socket says 172.21.15.93:49884) 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a39cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2b7fe700 1 -- 172.21.15.93:0/749516484 learned_addr learned my addr 172.21.15.93:0/749516484 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:57.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3081a700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 0x562b23a344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 -- 172.21.15.93:0/749516484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 msgr2=0x562b23a344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 0x562b23a344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 -- 172.21.15.93:0/749516484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 msgr2=0x562b23a33f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2bfff700 1 -- 172.21.15.93:0/749516484 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b237b3d80 con 0x562b238906b0 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.367+0000 7fbd3081a700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 0x562b23a344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:38:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.368+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a39cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0ca630 tx=0x7fbd1c099a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c93387f38790953 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.368+0000 7fbd127fc700 1 -- 172.21.15.93:0/749516484 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0a2400 con 0x562b238906b0 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.368+0000 7fbd127fc700 1 -- 172.21.15.93:0/749516484 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd1c010880 con 0x562b238906b0 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.368+0000 7fbd127fc700 1 -- 172.21.15.93:0/749516484 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0a1f70 con 0x562b238906b0 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- 172.21.15.93:0/749516484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 msgr2=0x562b23a39cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a39cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0ca630 tx=0x7fbd1c099a00 comp rx=0 tx=0).stop 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- 172.21.15.93:0/749516484 shutdown_connections 2023-07-21T22:38:57.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a660 0x562b23a344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a39cd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/749516484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a210 0x562b23a33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- 172.21.15.93:0/749516484 >> 172.21.15.93:0/749516484 conn(0x562b23a29ae0 msgr2=0x562b23a2f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- 172.21.15.93:0/749516484 shutdown_connections 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- 172.21.15.93:0/749516484 wait complete. 2023-07-21T22:38:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 -- start start 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.369+0000 7fbd3e402b80 1 Processor -- start 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a3d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a3df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23925a60 con 0x562b23a3a660 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23925760 con 0x562b23a3a210 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b238098c0 con 0x562b238906b0 2023-07-21T22:38:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3e402b80 10 client.0 _ll_register_callbacks cb 0x562b23923700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36968/0 (socket says 172.21.15.93:36968) 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd2bfff700 1 -- 172.21.15.93:0/3447316548 learned_addr learned my addr 172.21.15.93:0/3447316548 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a3df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a3d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 -- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 msgr2=0x562b23a405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 -- 172.21.15.93:0/3447316548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 msgr2=0x562b23a3d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a3d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.370+0000 7fbd3081a700 1 -- 172.21.15.93:0/3447316548 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c10b840 con 0x562b23a3a660 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3081a700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a3df30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c19b850 tx=0x7fbd2400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389362fa4641893e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd240077e0 con 0x562b23a3a660 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbd24006e40 con 0x562b23a3a660 2023-07-21T22:38:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b238df490 con 0x562b23a3a660 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd24007940 con 0x562b23a3a660 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 10 client.17886 fetch_fsmap learned FSMap version 87 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x7fbd2c0fbc80 con 0x562b23a3a660 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b238df490 con 0x562b23a3a660 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 87) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbd24014070 con 0x562b23a3a660 2023-07-21T22:38:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 10 client.17886 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T22:38:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 20 client.17886 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23a4db00 0x562b23a4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 4 client.17886 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.371+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b2397ee50 con 0x562b23a4db00 2023-07-21T22:38:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.373+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23a4db00 0x562b23a4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.374+0000 7fbd0b7fe700 10 client.17886 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.374+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23a4db00 0x562b23a4ff10 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9b4fc9c8bd44aa server_cookie=5d1450ac4d28fd70 in_seq=0 out_seq=0 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/3447316548 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562b2397ee50 con 0x562b23a4db00 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd0b7fe700 10 client.17886 handle_command_reply: tid=0 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 client.17886 shutdown 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 msgr2=0x562b23a3df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a3df30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c19b850 tx=0x7fbd2400aba0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 shutdown_connections 2023-07-21T22:38:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23a4db00 0x562b23a4ff10 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a3df30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/3447316548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a3d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd0b7fe700 0 client.17886 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 >> 172.21.15.93:0/3447316548 conn(0x562b23a29ae0 msgr2=0x562b23a2e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 shutdown_connections 2023-07-21T22:38:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.376+0000 7fbd3e402b80 1 -- 172.21.15.93:0/3447316548 wait complete. 2023-07-21T22:38:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.377+0000 7fbd3e402b80 20 client.17886 trim_cache size 0 max 16384 2023-07-21T22:38:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 Processor -- start 2023-07-21T22:38:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 -- start start 2023-07-21T22:38:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a2eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a34840 con 0x562b23a3a660 2023-07-21T22:38:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a3e290 con 0x562b23a3a210 2023-07-21T22:38:57.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a3be10 con 0x562b238906b0 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd3081a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a2eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.396+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a44ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:36976/0 (socket says 172.21.15.93:36976) 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2bfff700 1 -- 172.21.15.93:0/2966394801 learned_addr learned my addr 172.21.15.93:0/2966394801 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd3081a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a2eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49900/0 (socket says 172.21.15.93:49900) 2023-07-21T22:38:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 -- 172.21.15.93:0/2966394801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 msgr2=0x562b23a44ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a44ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 -- 172.21.15.93:0/2966394801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 msgr2=0x562b23a2eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a2eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 -- 172.21.15.93:0/2966394801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b2379cc00 con 0x562b23a3a210 2023-07-21T22:38:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.397+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a40140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd20006010 tx=0x7fbd20022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc65b65cc1bbc072 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.402+0000 7fbd12ffd700 1 -- 172.21.15.93:0/2966394801 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2002c030 con 0x562b23a3a210 2023-07-21T22:38:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.402+0000 7fbd12ffd700 1 -- 172.21.15.93:0/2966394801 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd2002d040 con 0x562b23a3a210 2023-07-21T22:38:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.402+0000 7fbd12ffd700 1 -- 172.21.15.93:0/2966394801 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000f800 con 0x562b23a3a210 2023-07-21T22:38:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- 172.21.15.93:0/2966394801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 msgr2=0x562b23a40140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a40140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbd20006010 tx=0x7fbd20022000 comp rx=0 tx=0).stop 2023-07-21T22:38:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- 172.21.15.93:0/2966394801 shutdown_connections 2023-07-21T22:38:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a2eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b238906b0 0x562b23a44ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/2966394801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b23a3a210 0x562b23a40140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- 172.21.15.93:0/2966394801 >> 172.21.15.93:0/2966394801 conn(0x562b23a29ae0 msgr2=0x562b23a379d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- 172.21.15.93:0/2966394801 shutdown_connections 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- 172.21.15.93:0/2966394801 wait complete. 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.403+0000 7fbd3e402b80 1 -- start start 2023-07-21T22:38:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 Processor -- start 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b23a3a210 0x562b23a36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a35c70 con 0x562b23a3a660 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a35de0 con 0x562b238906b0 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b23a35fa0 con 0x562b23a3a210 2023-07-21T22:38:57.449 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3e402b80 10 client.0 _ll_register_callbacks cb 0x562b23923700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd2bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32832/0 (socket says 172.21.15.93:32832) 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd2bfff700 1 -- 172.21.15.93:0/644272176 learned_addr learned my addr 172.21.15.93:0/644272176 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b23a3a210 0x562b23a36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.404+0000 7fbd3081a700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 -- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b23a3a210 msgr2=0x562b23a36fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b23a3a210 0x562b23a36fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 -- 172.21.15.93:0/644272176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 msgr2=0x562b23a35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a35730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 -- 172.21.15.93:0/644272176 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd20003c80 con 0x562b238906b0 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd2bfff700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0cabb0 tx=0x7fbd1c099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5baf20a79e56cd35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd0b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c099910 con 0x562b238906b0 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fbd1c09a410 con 0x562b238906b0 2023-07-21T22:38:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd1c010880 con 0x562b238906b0 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd1c0b8260 con 0x562b238906b0 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd3e402b80 10 client.27206 fetch_fsmap learned FSMap version 87 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.405+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x562b237da1f0 con 0x562b238906b0 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd1c010880 con 0x562b238906b0 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 87) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbd1c094bc0 con 0x562b238906b0 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 10 client.27206 fetch_fsmap finished waiting for FSMap version 87 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:38:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 20 client.27206 populate_metadata read hostname 'smithi093' 2023-07-21T22:38:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23aa26b0 0x562b23aa4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 4 client.27206 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b23940ff0 con 0x562b23aa26b0 2023-07-21T22:38:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.406+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23aa26b0 0x562b23aa4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.407+0000 7fbd2b7fe700 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23aa26b0 0x562b23aa4a90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1276e2ffac86b0f2 server_cookie=af7fef25325dd0e7 in_seq=0 out_seq=0 2023-07-21T22:38:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.407+0000 7fbd0b7fe700 10 client.27206 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:57.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 325394 29.44 MB/sec execute 453 sec latency 120.409 ms 2023-07-21T22:38:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.616+0000 7fbd0b7fe700 1 -- 172.21.15.93:0/644272176 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b23940ff0 con 0x562b23aa26b0 2023-07-21T22:38:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.616+0000 7fbd0b7fe700 10 client.27206 handle_command_reply: tid=0 2023-07-21T22:38:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.616+0000 7fbd3e402b80 1 client.27206 shutdown 2023-07-21T22:38:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 msgr2=0x562b23a36aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0cabb0 tx=0x7fbd1c099eb0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 shutdown_connections 2023-07-21T22:38:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x562b23aa26b0 0x562b23aa4a90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b23a3a660 0x562b23a35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd0b7fe700 0 client.27206 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:38:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b238906b0 0x562b23a36aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 --2- 172.21.15.93:0/644272176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b23a3a210 0x562b23a36fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.617+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 >> 172.21.15.93:0/644272176 conn(0x562b23a29ae0 msgr2=0x562b23a38970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 shutdown_connections 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3e402b80 1 -- 172.21.15.93:0/644272176 wait complete. 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3e402b80 20 client.27206 trim_cache size 0 max 16384 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:38:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:38:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:38:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 msgr2=0x7fbd140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 0x7fbd140a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd200098e0 tx=0x7fbd20000bc0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 msgr2=0x7fbd2c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00e9a0 tx=0x7fbd1c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 shutdown_connections 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c103430 0x7fbd2c1949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd2c0fe3f0 0x7fbd2c19ab50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fbd140a12c0 0x7fbd140a3770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 --2- 172.21.15.93:0/3929393954 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd2c0fed90 0x7fbd2c19b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.618+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 >> 172.21.15.93:0/3929393954 conn(0x7fbd2c075f90 msgr2=0x7fbd2c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.619+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 shutdown_connections 2023-07-21T22:38:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:38:57.619+0000 7fbd3227d700 1 -- 172.21.15.93:0/3929393954 wait complete. 2023-07-21T22:38:57.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:57 smithi134 ceph-mon[97873]: pgmap v717: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 20 MiB/s wr, 680 op/s 2023-07-21T22:38:57.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:57 smithi093 ceph-mon[96579]: pgmap v717: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 20 MiB/s wr, 680 op/s 2023-07-21T22:38:58.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:57 smithi194 ceph-mon[98288]: pgmap v717: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 20 MiB/s wr, 680 op/s 2023-07-21T22:38:58.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 325742 29.40 MB/sec execute 454 sec latency 179.564 ms 2023-07-21T22:38:59.557 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326245 29.40 MB/sec execute 455 sec latency 89.721 ms 2023-07-21T22:38:59.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:38:59 smithi134 ceph-mon[97873]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 955 op/s 2023-07-21T22:38:59.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:38:59 smithi093 ceph-mon[96579]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 955 op/s 2023-07-21T22:39:00.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:38:59 smithi194 ceph-mon[98288]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 955 op/s 2023-07-21T22:39:00.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326595 29.37 MB/sec execute 456 sec latency 445.492 ms 2023-07-21T22:39:01.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326595 29.31 MB/sec execute 457 sec latency 1445.578 ms 2023-07-21T22:39:01.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:01 smithi134 ceph-mon[97873]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 736 op/s 2023-07-21T22:39:01.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:01 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:01.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:01 smithi093 ceph-mon[96579]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 736 op/s 2023-07-21T22:39:01.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:01 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:02.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:01 smithi194 ceph-mon[98288]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 19 MiB/s wr, 736 op/s 2023-07-21T22:39:02.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:01 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:02.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326595 29.24 MB/sec execute 458 sec latency 2445.671 ms 2023-07-21T22:39:03.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326694 29.18 MB/sec execute 459 sec latency 2845.290 ms 2023-07-21T22:39:03.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:03 smithi134 ceph-mon[97873]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 754 op/s 2023-07-21T22:39:03.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:03 smithi093 ceph-mon[96579]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 754 op/s 2023-07-21T22:39:04.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:03 smithi194 ceph-mon[98288]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 19 MiB/s wr, 754 op/s 2023-07-21T22:39:04.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326696 29.12 MB/sec execute 460 sec latency 1435.112 ms 2023-07-21T22:39:05.558 INFO:tasks.workunit.client.0.smithi093.stdout: 10 326937 29.07 MB/sec execute 461 sec latency 1996.407 ms 2023-07-21T22:39:05.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:05 smithi134 ceph-mon[97873]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 767 op/s 2023-07-21T22:39:05.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:05 smithi093 ceph-mon[96579]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 767 op/s 2023-07-21T22:39:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:05 smithi194 ceph-mon[98288]: pgmap v721: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 18 MiB/s wr, 767 op/s 2023-07-21T22:39:06.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 327208 29.04 MB/sec execute 462 sec latency 203.403 ms 2023-07-21T22:39:06.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:06 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:07.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:06 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:07.131 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:06 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:07.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 327324 29.00 MB/sec execute 463 sec latency 202.221 ms 2023-07-21T22:39:07.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:07 smithi093 ceph-mon[96579]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-07-21T22:39:08.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:07 smithi194 ceph-mon[98288]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-07-21T22:39:08.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:07 smithi134 ceph-mon[97873]: pgmap v722: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 437 op/s 2023-07-21T22:39:08.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 327687 29.00 MB/sec execute 464 sec latency 184.724 ms 2023-07-21T22:39:08.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:08 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:08.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:08 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:08.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:08 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:39:09.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:08 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:09.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:08 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:09.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:08 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:39:09.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:08 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:09.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:08 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:09.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:08 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:39:09.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 328177 29.00 MB/sec execute 465 sec latency 130.518 ms 2023-07-21T22:39:09.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:09 smithi093 ceph-mon[96579]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2023-07-21T22:39:10.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:09 smithi194 ceph-mon[98288]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2023-07-21T22:39:10.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:09 smithi134 ceph-mon[97873]: pgmap v723: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.4 MiB/s wr, 648 op/s 2023-07-21T22:39:10.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 328357 28.95 MB/sec execute 466 sec latency 235.317 ms 2023-07-21T22:39:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/1309869705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 msgr2=0x7f5e041067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/1309869705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e041067b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b0a0 tx=0x7f5dfc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/1309869705 shutdown_connections 2023-07-21T22:39:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/1309869705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 0x7f5e04112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/1309869705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e0410c1c0 0x7f5e0410c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/1309869705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e041067b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.796+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/1309869705 >> 172.21.15.93:0/1309869705 conn(0x7f5e04075a20 msgr2=0x7f5e04077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.797+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/1309869705 shutdown_connections 2023-07-21T22:39:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.797+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/1309869705 wait complete. 2023-07-21T22:39:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.798+0000 7f5e0ba5b700 1 Processor -- start 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.798+0000 7f5e0ba5b700 1 -- start start 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 0x7f5e04080ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e04079720 0x7f5e04079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0407a360 con 0x7f5e04106340 2023-07-21T22:39:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0407a4d0 con 0x7f5e04079720 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e0ba5b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0407e4b0 con 0x7f5e04106cf0 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49916/0 (socket says 172.21.15.93:49916) 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 -- 172.21.15.93:0/409741760 learned_addr learned my addr 172.21.15.93:0/409741760 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 0x7f5e04080ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 -- 172.21.15.93:0/409741760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 msgr2=0x7f5e04080ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 0x7f5e04080ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 -- 172.21.15.93:0/409741760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e04079720 msgr2=0x7f5e04079b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e04079720 0x7f5e04079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.799+0000 7f5e097f7700 1 -- 172.21.15.93:0/409741760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc009d00 con 0x7f5e04106340 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5e097f7700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000b480 tx=0x7f5e0000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5dfa7fc700 1 -- 172.21.15.93:0/409741760 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000118f0 con 0x7f5e04106340 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5dfa7fc700 1 -- 172.21.15.93:0/409741760 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e00010380 con 0x7f5e04106340 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0407e790 con 0x7f5e04106340 2023-07-21T22:39:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5dfa7fc700 1 -- 172.21.15.93:0/409741760 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000f540 con 0x7f5e04106340 2023-07-21T22:39:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.800+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0407eca0 con 0x7f5e04106340 2023-07-21T22:39:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.803+0000 7f5dfa7fc700 1 -- 172.21.15.93:0/409741760 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5e000104f0 con 0x7f5e04106340 2023-07-21T22:39:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.804+0000 7f5dfa7fc700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 0x7f5df00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 Processor -- start 2023-07-21T22:39:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5dfa7fc700 1 -- 172.21.15.93:0/409741760 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5e00098bf0 con 0x7f5e04106340 2023-07-21T22:39:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 -- start start 2023-07-21T22:39:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6247836b0 0x55b62492ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624926f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62482cc50 con 0x55b62492d660 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62487f9e0 con 0x55b62492d210 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.805+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624878e70 con 0x55b6247836b0 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e08ff6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624926f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e08ff6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624926f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32854/0 (socket says 172.21.15.93:32854) 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e08ff6700 1 -- 172.21.15.93:0/3642993557 learned_addr learned my addr 172.21.15.93:0/3642993557 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 -- 172.21.15.93:0/3642993557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6247836b0 msgr2=0x55b62492ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6247836b0 0x55b62492ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 -- 172.21.15.93:0/3642993557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 msgr2=0x55b624926f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624926f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 -- 172.21.15.93:0/3642993557 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6246a6d80 con 0x55b62492d660 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.806+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249274a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009bd0 tx=0x7f5df4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc14fa119a41f31 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.807+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 0x7f5df00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.807+0000 7f5def7fe700 1 -- 172.21.15.93:0/3642993557 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4004230 con 0x55b62492d660 2023-07-21T22:39:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.807+0000 7f5def7fe700 1 -- 172.21.15.93:0/3642993557 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df400eda0 con 0x55b62492d660 2023-07-21T22:39:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.807+0000 7f5def7fe700 1 -- 172.21.15.93:0/3642993557 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005020 con 0x55b62492d660 2023-07-21T22:39:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 -- 172.21.15.93:0/3642993557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 msgr2=0x55b6249274a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249274a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009bd0 tx=0x7f5df4009ee0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 -- 172.21.15.93:0/3642993557 shutdown_connections 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249274a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b6247836b0 0x55b62492ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/3642993557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624926f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.808+0000 7f5e17be0b80 1 -- 172.21.15.93:0/3642993557 >> 172.21.15.93:0/3642993557 conn(0x55b62491cae0 msgr2=0x55b6249220e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.809+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 0x7f5df00a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009610 tx=0x7f5dfc00b3b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.813+0000 7f5e17be0b80 1 -- 172.21.15.93:0/3642993557 shutdown_connections 2023-07-21T22:39:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.816+0000 7f5e17be0b80 1 -- 172.21.15.93:0/3642993557 wait complete. 2023-07-21T22:39:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.816+0000 7f5e17be0b80 1 -- start start 2023-07-21T22:39:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 Processor -- start 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6247836b0 0x55b624930870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624930db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624818a60 con 0x55b62492d660 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624818760 con 0x55b6247836b0 2023-07-21T22:39:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6246fc8c0 con 0x55b62492d210 2023-07-21T22:39:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e17be0b80 10 client.0 _ll_register_callbacks cb 0x55b624816700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e09ff8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e09ff8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49932/0 (socket says 172.21.15.93:49932) 2023-07-21T22:39:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.817+0000 7f5e09ff8700 1 -- 172.21.15.93:0/164536762 learned_addr learned my addr 172.21.15.93:0/164536762 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 -- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 msgr2=0x55b624930db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:39:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624930db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 -- 172.21.15.93:0/164536762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6247836b0 msgr2=0x55b624930870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6247836b0 0x55b624930870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 -- 172.21.15.93:0/164536762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df4009880 con 0x55b62492d660 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ba0 tx=0x7f5df40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bf4801c3248ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5decff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005430 con 0x55b62492d660 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5df4014070 con 0x55b62492d660 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6247d2490 con 0x55b62492d660 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.818+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df4004940 con 0x55b62492d660 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 10 client.17904 fetch_fsmap learned FSMap version 88 2023-07-21T22:39:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55b62468fc00 con 0x55b62492d660 2023-07-21T22:39:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df402f020 con 0x55b62492d660 2023-07-21T22:39:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 88) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5df4010430 con 0x55b62492d660 2023-07-21T22:39:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 10 client.17904 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T22:39:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 20 client.17904 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b624951a90 0x55b624953ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 4 client.17904 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:39:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.819+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b624871e50 con 0x55b624951a90 2023-07-21T22:39:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.820+0000 7f5e097f7700 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b624951a90 0x55b624953ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.829+0000 7f5decff9700 10 client.17904 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.829+0000 7f5e097f7700 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b624951a90 0x55b624953ea0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3503cc12472b6a server_cookie=130c0f05ad2c7d47 in_seq=0 out_seq=0 2023-07-21T22:39:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.846+0000 7f5decff9700 1 -- 172.21.15.93:0/164536762 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b624871e50 con 0x55b624951a90 2023-07-21T22:39:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.846+0000 7f5decff9700 10 client.17904 handle_command_reply: tid=0 2023-07-21T22:39:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.846+0000 7f5e17be0b80 1 client.17904 shutdown 2023-07-21T22:39:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.846+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 msgr2=0x55b6249312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.846+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ba0 tx=0x7f5df40048a0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 shutdown_connections 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b624951a90 0x55b624953ea0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b62492d660 0x55b6249312f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6247836b0 0x55b624930870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/164536762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624930db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 >> 172.21.15.93:0/164536762 conn(0x55b62491cae0 msgr2=0x55b6249213e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5decff9700 0 client.17904 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 shutdown_connections 2023-07-21T22:39:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.847+0000 7f5e17be0b80 1 -- 172.21.15.93:0/164536762 wait complete. 2023-07-21T22:39:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.848+0000 7f5e17be0b80 20 client.17904 trim_cache size 0 max 16384 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 Processor -- start 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 -- start start 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b62492c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624948bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624927f80 con 0x55b6247836b0 2023-07-21T22:39:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624927d60 con 0x55b62492d210 2023-07-21T22:39:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.873+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62492c980 con 0x55b62492d660 2023-07-21T22:39:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37018/0 (socket says 172.21.15.93:37018) 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 -- 172.21.15.93:0/1186971890 learned_addr learned my addr 172.21.15.93:0/1186971890 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 -- 172.21.15.93:0/1186971890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 msgr2=0x55b624948bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624948bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 -- 172.21.15.93:0/1186971890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 msgr2=0x55b62492c1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b62492c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 -- 172.21.15.93:0/1186971890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6246cd1f0 con 0x55b62492d660 2023-07-21T22:39:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009730 tx=0x7f5df400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb95210e093d17a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.874+0000 7f5deffff700 1 -- 172.21.15.93:0/1186971890 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4009290 con 0x55b62492d660 2023-07-21T22:39:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.875+0000 7f5deffff700 1 -- 172.21.15.93:0/1186971890 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df402f070 con 0x55b62492d660 2023-07-21T22:39:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.875+0000 7f5deffff700 1 -- 172.21.15.93:0/1186971890 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4018760 con 0x55b62492d660 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.875+0000 7f5e17be0b80 1 -- 172.21.15.93:0/1186971890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 msgr2=0x55b624926f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.875+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009730 tx=0x7f5df400ef10 comp rx=0 tx=0).stop 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 -- 172.21.15.93:0/1186971890 shutdown_connections 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d660 0x55b624926f40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b62492c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/1186971890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d210 0x55b624948bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 -- 172.21.15.93:0/1186971890 >> 172.21.15.93:0/1186971890 conn(0x55b62491cae0 msgr2=0x55b624930830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 -- 172.21.15.93:0/1186971890 shutdown_connections 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 -- 172.21.15.93:0/1186971890 wait complete. 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 -- start start 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.876+0000 7f5e17be0b80 1 Processor -- start 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624928730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 0x55b624928c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62492a4f0 con 0x55b6247836b0 2023-07-21T22:39:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6249291b0 con 0x55b62492d660 2023-07-21T22:39:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b624929370 con 0x55b62492d210 2023-07-21T22:39:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.877+0000 7f5e17be0b80 10 client.0 _ll_register_callbacks cb 0x7f5df4009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49944/0 (socket says 172.21.15.93:49944) 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 -- 172.21.15.93:0/2342175136 learned_addr learned my addr 172.21.15.93:0/2342175136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 0x55b624928c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624928730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 -- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 msgr2=0x55b624928730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624928730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 -- 172.21.15.93:0/2342175136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 msgr2=0x55b624928c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 0x55b624928c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 -- 172.21.15.93:0/2342175136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b62468fc00 con 0x55b6247836b0 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e09ff8700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 0x55b624928c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5e097f7700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5e00095660 tx=0x7f5e00015300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff61e844ec3544c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5decff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:39:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.878+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00098e50 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f5e00098fb0 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df400eda0 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e000b1200 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 10 client.17907 fetch_fsmap learned FSMap version 88 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55b6246a5250 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df400eda0 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 88) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e000a0bb0 con 0x55b6247836b0 2023-07-21T22:39:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 10 client.17907 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.879+0000 7f5e17be0b80 20 client.17907 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b6249a6bc0 0x55b6249a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5e17be0b80 4 client.17907 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b624833ff0 con 0x55b6249a6bc0 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b6249a6bc0 0x55b6249a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5decff9700 10 client.17907 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:10.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:10.880+0000 7f5e08ff6700 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b6249a6bc0 0x55b6249a8fd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8b8e463e28a8a5 server_cookie=8aff244c3d55bc16 in_seq=0 out_seq=0 2023-07-21T22:39:10.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:10 smithi093 ceph-mon[96579]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T22:39:11.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:10 smithi194 ceph-mon[98288]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T22:39:11.135 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:10 smithi134 ceph-mon[97873]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T22:39:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.327+0000 7f5decff9700 1 -- 172.21.15.93:0/2342175136 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b624833ff0 con 0x55b6249a6bc0 2023-07-21T22:39:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.327+0000 7f5decff9700 10 client.17907 handle_command_reply: tid=0 2023-07-21T22:39:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.327+0000 7f5e17be0b80 1 client.17907 shutdown 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 msgr2=0x55b624929e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5e00095660 tx=0x7f5e00015300 comp rx=0 tx=0).stop 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 shutdown_connections 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55b6249a6bc0 0x55b6249a8fd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b62492d660 0x55b624928c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b6247836b0 0x55b624929e70 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 --2- 172.21.15.93:0/2342175136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62492d210 0x55b624928730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 >> 172.21.15.93:0/2342175136 conn(0x55b62491cae0 msgr2=0x55b62492edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5decff9700 0 client.17907 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 shutdown_connections 2023-07-21T22:39:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.328+0000 7f5e17be0b80 1 -- 172.21.15.93:0/2342175136 wait complete. 2023-07-21T22:39:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.329+0000 7f5e17be0b80 20 client.17907 trim_cache size 0 max 16384 2023-07-21T22:39:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:39:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:39:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:39:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:39:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.329+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 msgr2=0x7f5df00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.329+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 0x7f5df00a37f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009610 tx=0x7f5dfc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.329+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 msgr2=0x7f5e04080920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.329+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000b480 tx=0x7f5e0000b840 comp rx=0 tx=0).stop 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 shutdown_connections 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e04079720 0x7f5e04079b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5df00a1340 0x7f5df00a37f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e04106340 0x7f5e04080920 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 --2- 172.21.15.93:0/409741760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5e04106cf0 0x7f5e04080ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 >> 172.21.15.93:0/409741760 conn(0x7f5e04075a20 msgr2=0x7f5e04076f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 shutdown_connections 2023-07-21T22:39:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:11.330+0000 7f5e0ba5b700 1 -- 172.21.15.93:0/409741760 wait complete. 2023-07-21T22:39:11.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 328493 28.89 MB/sec execute 467 sec latency 275.477 ms 2023-07-21T22:39:11.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:11 smithi093 ceph-mon[96579]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-07-21T22:39:12.092 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:11 smithi194 ceph-mon[98288]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-07-21T22:39:12.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:11 smithi134 ceph-mon[97873]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 435 op/s 2023-07-21T22:39:12.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 328797 28.88 MB/sec execute 468 sec latency 133.129 ms 2023-07-21T22:39:13.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 329381 28.88 MB/sec execute 469 sec latency 111.592 ms 2023-07-21T22:39:13.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:13 smithi093 ceph-mon[96579]: pgmap v726: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-07-21T22:39:14.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:13 smithi194 ceph-mon[98288]: pgmap v726: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-07-21T22:39:14.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:13 smithi134 ceph-mon[97873]: pgmap v726: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-07-21T22:39:14.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 329975 28.87 MB/sec execute 470 sec latency 114.666 ms 2023-07-21T22:39:15.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 330580 28.90 MB/sec execute 471 sec latency 90.745 ms 2023-07-21T22:39:16.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:15 smithi194 ceph-mon[98288]: pgmap v727: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T22:39:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:15 smithi134 ceph-mon[97873]: pgmap v727: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T22:39:16.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:15 smithi093 ceph-mon[96579]: pgmap v727: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T22:39:16.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 331113 28.90 MB/sec execute 472 sec latency 80.976 ms 2023-07-21T22:39:17.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:16 smithi194 ceph-mon[98288]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 553 op/s 2023-07-21T22:39:17.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:16 smithi134 ceph-mon[97873]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 553 op/s 2023-07-21T22:39:17.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:16 smithi093 ceph-mon[96579]: pgmap v728: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 553 op/s 2023-07-21T22:39:17.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 331536 28.89 MB/sec execute 473 sec latency 81.150 ms 2023-07-21T22:39:18.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 332010 28.86 MB/sec execute 474 sec latency 84.445 ms 2023-07-21T22:39:18.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:18 smithi093 ceph-mon[96579]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T22:39:18.837 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:18 smithi194 ceph-mon[98288]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T22:39:18.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:18 smithi134 ceph-mon[97873]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T22:39:19.559 INFO:tasks.workunit.client.0.smithi093.stdout: 10 332491 28.86 MB/sec execute 475 sec latency 175.369 ms 2023-07-21T22:39:19.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:19 smithi194 ceph-mon[98288]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 544 op/s 2023-07-21T22:39:19.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:19 smithi134 ceph-mon[97873]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 544 op/s 2023-07-21T22:39:19.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:19 smithi093 ceph-mon[96579]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 544 op/s 2023-07-21T22:39:20.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 333102 28.87 MB/sec execute 476 sec latency 116.166 ms 2023-07-21T22:39:21.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 333681 28.88 MB/sec execute 477 sec latency 116.775 ms 2023-07-21T22:39:21.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:21 smithi194 ceph-mon[98288]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2023-07-21T22:39:21.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:21 smithi134 ceph-mon[97873]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2023-07-21T22:39:21.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:21 smithi093 ceph-mon[96579]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2023-07-21T22:39:22.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 334208 28.89 MB/sec execute 478 sec latency 75.870 ms 2023-07-21T22:39:23.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 334663 28.87 MB/sec execute 479 sec latency 113.593 ms 2023-07-21T22:39:23.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:23 smithi194 ceph-mon[98288]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 435 op/s 2023-07-21T22:39:23.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:23 smithi134 ceph-mon[97873]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 435 op/s 2023-07-21T22:39:23.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:23 smithi093 ceph-mon[96579]: pgmap v732: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.9 MiB/s wr, 435 op/s 2023-07-21T22:39:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 -- 172.21.15.93:0/4292584178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10a530 msgr2=0x7f4cdc10a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/4292584178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10a530 0x7f4cdc10a9a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009990 tx=0x7f4ccc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 -- 172.21.15.93:0/4292584178 shutdown_connections 2023-07-21T22:39:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/4292584178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc117320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/4292584178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/4292584178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10a530 0x7f4cdc10a9a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 -- 172.21.15.93:0/4292584178 >> 172.21.15.93:0/4292584178 conn(0x7f4cdc06a940 msgr2=0x7f4cdc06ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 -- 172.21.15.93:0/4292584178 shutdown_connections 2023-07-21T22:39:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.478+0000 7f4ce13d8700 1 -- 172.21.15.93:0/4292584178 wait complete. 2023-07-21T22:39:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 Processor -- start 2023-07-21T22:39:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 -- start start 2023-07-21T22:39:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10a530 0x7f4cdc1a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0af0 con 0x7f4cdc10aee0 2023-07-21T22:39:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0c60 con 0x7f4cdc110560 2023-07-21T22:39:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4ce13d8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0e20 con 0x7f4cdc10a530 2023-07-21T22:39:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32886/0 (socket says 172.21.15.93:32886) 2023-07-21T22:39:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/3864886445 learned_addr learned my addr 172.21.15.93:0/3864886445 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:24.528 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49958/0 (socket says 172.21.15.93:49958) 2023-07-21T22:39:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.479+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10a530 0x7f4cdc1a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/3864886445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10a530 msgr2=0x7f4cdc1a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.529 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10a530 0x7f4cdc1a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 msgr2=0x7f4cdc1a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/3864886445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc0095f0 con 0x7f4cdc110560 2023-07-21T22:39:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:39:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4007ee0 tx=0x7f4cd400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cc3fff700 1 -- 172.21.15.93:0/3864886445 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400dd20 con 0x7f4cdc110560 2023-07-21T22:39:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc1a1100 con 0x7f4cdc110560 2023-07-21T22:39:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cc3fff700 1 -- 172.21.15.93:0/3864886445 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd400f040 con 0x7f4cdc110560 2023-07-21T22:39:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4cc3fff700 1 -- 172.21.15.93:0/3864886445 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40048a0 con 0x7f4cdc110560 2023-07-21T22:39:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.480+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc1ad610 con 0x7f4cdc110560 2023-07-21T22:39:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.484+0000 7f4ced55db80 1 Processor -- start 2023-07-21T22:39:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.484+0000 7f4ced55db80 1 -- start start 2023-07-21T22:39:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311cffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00660 0x563311cfa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311bffc50 con 0x563311d00210 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311c529e0 con 0x563311d00660 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4ced55db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311c4be70 con 0x563311b566b0 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311cffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49964/0 (socket says 172.21.15.93:49964) 2023-07-21T22:39:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 -- 172.21.15.93:0/2227030313 learned_addr learned my addr 172.21.15.93:0/2227030313 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00660 0x563311cfa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 -- 172.21.15.93:0/2227030313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 msgr2=0x563311cffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311cffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 -- 172.21.15.93:0/2227030313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00660 msgr2=0x563311cfa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00660 0x563311cfa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 -- 172.21.15.93:0/2227030313 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563311a79d80 con 0x563311d00210 2023-07-21T22:39:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.485+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009d00 tx=0x7f4cd0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f717272a46f4a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.486+0000 7f4cc17fa700 1 -- 172.21.15.93:0/2227030313 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0004230 con 0x563311d00210 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.486+0000 7f4cc17fa700 1 -- 172.21.15.93:0/2227030313 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd0004390 con 0x563311d00210 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.486+0000 7f4cc17fa700 1 -- 172.21.15.93:0/2227030313 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0019750 con 0x563311d00210 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.486+0000 7f4cc3fff700 1 -- 172.21.15.93:0/3864886445 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4cd4003a00 con 0x7f4cdc110560 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4cc3fff700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 0x7f4cc40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- 172.21.15.93:0/2227030313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 msgr2=0x563311cf9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009d00 tx=0x7f4cd0011720 comp rx=0 tx=0).stop 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4cc3fff700 1 -- 172.21.15.93:0/3864886445 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4cd40cc550 con 0x7f4cdc110560 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 0x7f4cc40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- 172.21.15.93:0/2227030313 shutdown_connections 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00660 0x563311cfa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311cffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 --2- 172.21.15.93:0/2227030313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00210 0x563311cf9f40 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- 172.21.15.93:0/2227030313 >> 172.21.15.93:0/2227030313 conn(0x563311cefae0 msgr2=0x563311cf50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- 172.21.15.93:0/2227030313 shutdown_connections 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- 172.21.15.93:0/2227030313 wait complete. 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 0x7f4cc40a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563311d00a70 tx=0x7f4ccc005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.487+0000 7f4ced55db80 1 -- start start 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 Processor -- start 2023-07-21T22:39:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311d06530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 0x563311d03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311beba60 con 0x563311d00660 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311beb760 con 0x563311d00210 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311acf8c0 con 0x563311b566b0 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4ced55db80 10 client.0 _ll_register_callbacks cb 0x563311be9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 0x563311d03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49972/0 (socket says 172.21.15.93:49972) 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/4087162695 learned_addr learned my addr 172.21.15.93:0/4087162695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311d06530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 -- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 msgr2=0x563311d06530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311d06530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 -- 172.21.15.93:0/4087162695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 msgr2=0x563311d03d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cda7fc700 1 -- 172.21.15.93:0/4087162695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd0009960 con 0x563311d00210 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.488+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 0x563311d03740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00b2e0 tx=0x7f4ccc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771c55a76e073b57 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4cbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00f7b0 con 0x563311d00210 2023-07-21T22:39:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563311ba5490 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4ccc00f910 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ccc00fad0 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4ced55db80 10 client.27224 fetch_fsmap learned FSMap version 88 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.489+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x563311a62c00 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ccc040020 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 88) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4ccc028400 con 0x563311d00210 2023-07-21T22:39:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 10 client.27224 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 10 client.27224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 20 client.27224 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d13a20 0x563311d15e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 4 client.27224 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.490+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563311c44e50 con 0x563311d13a20 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.492+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d13a20 0x563311d15e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.492+0000 7f4cbeff5700 10 client.27224 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.492+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d13a20 0x563311d15e30 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90ace3ee4ac95cc server_cookie=6c86811532f0c6a2 in_seq=0 out_seq=0 2023-07-21T22:39:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4cbeff5700 1 -- 172.21.15.93:0/4087162695 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563311c44e50 con 0x563311d13a20 2023-07-21T22:39:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4cbeff5700 10 client.27224 handle_command_reply: tid=0 2023-07-21T22:39:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 client.27224 shutdown 2023-07-21T22:39:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 msgr2=0x563311d03740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 0x563311d03740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00b2e0 tx=0x7f4ccc009310 comp rx=0 tx=0).stop 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 shutdown_connections 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d13a20 0x563311d15e30 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311d00660 0x563311d03d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311b566b0 0x563311d06530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 --2- 172.21.15.93:0/4087162695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311d00210 0x563311d03740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4cbeff5700 0 client.27224 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 >> 172.21.15.93:0/4087162695 conn(0x563311cefae0 msgr2=0x563311cf43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.494+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 shutdown_connections 2023-07-21T22:39:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.495+0000 7f4ced55db80 1 -- 172.21.15.93:0/4087162695 wait complete. 2023-07-21T22:39:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.495+0000 7f4ced55db80 20 client.27224 trim_cache size 0 max 16384 2023-07-21T22:39:24.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 335197 28.88 MB/sec execute 480 sec latency 100.609 ms 2023-07-21T22:39:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 Processor -- start 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 -- start start 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d01330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d02600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311d10450 con 0x563311b566b0 2023-07-21T22:39:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311c44e50 con 0x563311cefae0 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4ced55db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311cfad30 con 0x563311d00210 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d01330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cda7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49976/0 (socket says 172.21.15.93:49976) 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cda7fc700 1 -- 172.21.15.93:0/3714627170 learned_addr learned my addr 172.21.15.93:0/3714627170 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d02600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 -- 172.21.15.93:0/3714627170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 msgr2=0x563311d02600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d02600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 -- 172.21.15.93:0/3714627170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 msgr2=0x563311d0ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cdaffd700 1 -- 172.21.15.93:0/3714627170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563311aa01f0 con 0x563311cefae0 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.517+0000 7f4cda7fc700 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.518+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d01330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cd001b040 tx=0x7f4cd0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ab9ce036e5a2fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.518+0000 7f4cc1ffb700 1 -- 172.21.15.93:0/3714627170 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd002e070 con 0x563311cefae0 2023-07-21T22:39:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.518+0000 7f4cc1ffb700 1 -- 172.21.15.93:0/3714627170 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd0009eb0 con 0x563311cefae0 2023-07-21T22:39:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.518+0000 7f4cc1ffb700 1 -- 172.21.15.93:0/3714627170 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0003c00 con 0x563311cefae0 2023-07-21T22:39:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- 172.21.15.93:0/3714627170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 msgr2=0x563311d01330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d01330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4cd001b040 tx=0x7f4cd0011720 comp rx=0 tx=0).stop 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- 172.21.15.93:0/3714627170 shutdown_connections 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d02600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d01330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 --2- 172.21.15.93:0/3714627170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d0ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- 172.21.15.93:0/3714627170 >> 172.21.15.93:0/3714627170 conn(0x563311d10080 msgr2=0x563311d01ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- 172.21.15.93:0/3714627170 shutdown_connections 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- 172.21.15.93:0/3714627170 wait complete. 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 -- start start 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.519+0000 7f4ced55db80 1 Processor -- start 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311d0a710 con 0x563311b566b0 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311d0a880 con 0x563311cefae0 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563311d0aa40 con 0x563311d00210 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4ced55db80 10 client.0 _ll_register_callbacks cb 0x7f4ccc0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37064/0 (socket says 172.21.15.93:37064) 2023-07-21T22:39:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/1191876186 learned_addr learned my addr 172.21.15.93:0/1191876186 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/1191876186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 msgr2=0x563311d07240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d07240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/1191876186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 msgr2=0x563311d06d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.520+0000 7f4cdb7fe700 1 -- 172.21.15.93:0/1191876186 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563311a62c00 con 0x563311d00210 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.521+0000 7f4cdb7fe700 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40c9d50 tx=0x7f4cd4099030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63aa946e994521eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.521+0000 7f4cbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.521+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40a1a70 con 0x563311d00210 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.521+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4cd4009410 con 0x563311d00210 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.521+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cd0004390 con 0x563311d00210 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.522+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd40840b0 con 0x563311d00210 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.522+0000 7f4ced55db80 10 client.27142 fetch_fsmap learned FSMap version 88 2023-07-21T22:39:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.522+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x563311a79d80 con 0x563311d00210 2023-07-21T22:39:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd0004390 con 0x563311d00210 2023-07-21T22:39:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 88) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4cd40b7020 con 0x563311d00210 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 10 client.27142 fetch_fsmap finished waiting for FSMap version 88 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 10 client.27142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 20 client.27142 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d00660 0x563311d628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 4 client.27142 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:39:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.524+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563311c06ff0 con 0x563311d00660 2023-07-21T22:39:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.567+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d00660 0x563311d628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:24.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.567+0000 7f4cdaffd700 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d00660 0x563311d628e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781fb409724caab5 server_cookie=468c5294b2d32b63 in_seq=0 out_seq=0 2023-07-21T22:39:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.567+0000 7f4cbeff5700 10 client.27142 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4cbeff5700 1 -- 172.21.15.93:0/1191876186 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563311c06ff0 con 0x563311d00660 2023-07-21T22:39:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4cbeff5700 10 client.27142 handle_command_reply: tid=0 2023-07-21T22:39:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 client.27142 shutdown 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 msgr2=0x563311d0a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40c9d50 tx=0x7f4cd4099030 comp rx=0 tx=0).stop 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 shutdown_connections 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563311d00660 0x563311d628e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563311d00210 0x563311d0a1d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4cbeff5700 0 client.27142 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563311b566b0 0x563311d06d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 --2- 172.21.15.93:0/1191876186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563311cefae0 0x563311d07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 >> 172.21.15.93:0/1191876186 conn(0x563311d10080 msgr2=0x563311cf43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 shutdown_connections 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 1 -- 172.21.15.93:0/1191876186 wait complete. 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.879+0000 7f4ced55db80 20 client.27142 trim_cache size 0 max 16384 2023-07-21T22:39:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:39:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:39:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:39:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.880+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 msgr2=0x7f4cc40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.880+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 0x7f4cc40a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563311d00a70 tx=0x7f4ccc005fd0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.880+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 msgr2=0x7f4cdc1a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.880+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4007ee0 tx=0x7f4cd400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 shutdown_connections 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cdc110560 0x7f4cdc1a0480 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cdc10a530 0x7f4cdc1a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f4cc40a1340 0x7f4cc40a37f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 --2- 172.21.15.93:0/3864886445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cdc10aee0 0x7f4cdc1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.881+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 >> 172.21.15.93:0/3864886445 conn(0x7f4cdc06a940 msgr2=0x7f4cdc108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.882+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 shutdown_connections 2023-07-21T22:39:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:24.882+0000 7f4ce13d8700 1 -- 172.21.15.93:0/3864886445 wait complete. 2023-07-21T22:39:25.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 335534 28.85 MB/sec execute 481 sec latency 267.033 ms 2023-07-21T22:39:25.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:25 smithi194 ceph-mon[98288]: pgmap v733: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.5 MiB/s wr, 465 op/s 2023-07-21T22:39:25.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:25 smithi134 ceph-mon[97873]: pgmap v733: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.5 MiB/s wr, 465 op/s 2023-07-21T22:39:25.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:25 smithi093 ceph-mon[96579]: pgmap v733: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.5 MiB/s wr, 465 op/s 2023-07-21T22:39:26.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 335955 28.83 MB/sec execute 482 sec latency 308.102 ms 2023-07-21T22:39:27.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 336452 28.82 MB/sec execute 483 sec latency 116.978 ms 2023-07-21T22:39:27.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:27 smithi194 ceph-mon[98288]: pgmap v734: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 489 op/s 2023-07-21T22:39:27.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:27 smithi134 ceph-mon[97873]: pgmap v734: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 489 op/s 2023-07-21T22:39:27.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:27 smithi093 ceph-mon[96579]: pgmap v734: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 489 op/s 2023-07-21T22:39:28.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 336969 28.83 MB/sec execute 484 sec latency 69.752 ms 2023-07-21T22:39:29.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 337460 28.84 MB/sec execute 485 sec latency 101.558 ms 2023-07-21T22:39:29.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:29 smithi194 ceph-mon[98288]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-21T22:39:29.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:29 smithi134 ceph-mon[97873]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-21T22:39:29.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:29 smithi093 ceph-mon[96579]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 704 op/s 2023-07-21T22:39:30.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 337988 28.81 MB/sec execute 486 sec latency 106.283 ms 2023-07-21T22:39:31.560 INFO:tasks.workunit.client.0.smithi093.stdout: 10 338349 28.79 MB/sec execute 487 sec latency 412.340 ms 2023-07-21T22:39:31.845 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:31 smithi194 ceph-mon[98288]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.4 MiB/s wr, 481 op/s 2023-07-21T22:39:31.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:31 smithi134 ceph-mon[97873]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.4 MiB/s wr, 481 op/s 2023-07-21T22:39:31.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:31 smithi093 ceph-mon[96579]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.4 MiB/s wr, 481 op/s 2023-07-21T22:39:32.561 INFO:tasks.workunit.client.0.smithi093.stdout: 10 338378 28.73 MB/sec execute 488 sec latency 1013.620 ms 2023-07-21T22:39:33.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 338758 28.74 MB/sec execute 489 sec latency 811.376 ms 2023-07-21T22:39:33.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:33 smithi194 ceph-mon[98288]: pgmap v737: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 539 op/s 2023-07-21T22:39:33.882 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:33 smithi134 ceph-mon[97873]: pgmap v737: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 539 op/s 2023-07-21T22:39:33.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:33 smithi093 ceph-mon[96579]: pgmap v737: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 539 op/s 2023-07-21T22:39:34.562 INFO:tasks.workunit.client.0.smithi093.stdout: 10 339072 28.72 MB/sec execute 490 sec latency 167.899 ms 2023-07-21T22:39:35.562 INFO:tasks.workunit.client.0.smithi093.stdout: 10 339642 28.71 MB/sec execute 491 sec latency 82.760 ms 2023-07-21T22:39:35.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:35 smithi194 ceph-mon[98288]: pgmap v738: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-07-21T22:39:35.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:35 smithi134 ceph-mon[97873]: pgmap v738: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-07-21T22:39:35.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:35 smithi093 ceph-mon[96579]: pgmap v738: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 21 MiB/s wr, 767 op/s 2023-07-21T22:39:36.562 INFO:tasks.workunit.client.0.smithi093.stdout: 10 340106 28.72 MB/sec execute 492 sec latency 119.785 ms 2023-07-21T22:39:36.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:36 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:36.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:36 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:36.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:36 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:37.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 340531 28.70 MB/sec execute 493 sec latency 1119.816 ms 2023-07-21T22:39:37.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:37 smithi194 ceph-mon[98288]: pgmap v739: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 597 op/s 2023-07-21T22:39:37.883 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:37 smithi134 ceph-mon[97873]: pgmap v739: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 597 op/s 2023-07-21T22:39:37.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:37 smithi093 ceph-mon[96579]: pgmap v739: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 17 MiB/s wr, 597 op/s 2023-07-21T22:39:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 -- 172.21.15.93:0/3712079798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 msgr2=0x7f2368102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 --2- 172.21.15.93:0/3712079798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368102380 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f236000b0d0 tx=0x7f236000b3e0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 -- 172.21.15.93:0/3712079798 shutdown_connections 2023-07-21T22:39:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 --2- 172.21.15.93:0/3712079798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2368106ff0 0x7f236810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 --2- 172.21.15.93:0/3712079798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368102380 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 --2- 172.21.15.93:0/3712079798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2368102950 0x7f2368106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 -- 172.21.15.93:0/3712079798 >> 172.21.15.93:0/3712079798 conn(0x7f23680fd800 msgr2=0x7f23680ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 -- 172.21.15.93:0/3712079798 shutdown_connections 2023-07-21T22:39:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.060+0000 7f236ebcc700 1 -- 172.21.15.93:0/3712079798 wait complete. 2023-07-21T22:39:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 Processor -- start 2023-07-21T22:39:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 -- start start 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2368102950 0x7f2368198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2368106ff0 0x7f236819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819d1f0 con 0x7f2368101fb0 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819d360 con 0x7f2368102950 2023-07-21T22:39:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236ebcc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819d520 con 0x7f2368106ff0 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236c968700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236c968700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49994/0 (socket says 172.21.15.93:49994) 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.061+0000 7f236c968700 1 -- 172.21.15.93:0/1008899240 learned_addr learned my addr 172.21.15.93:0/1008899240 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 -- 172.21.15.93:0/1008899240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2368106ff0 msgr2=0x7f236819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2368106ff0 0x7f236819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 -- 172.21.15.93:0/1008899240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2368102950 msgr2=0x7f2368198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2368102950 0x7f2368198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 -- 172.21.15.93:0/1008899240 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360009d00 con 0x7f2368101fb0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236c968700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f23600078a0 tx=0x7f2360007980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f2365ffb700 1 -- 172.21.15.93:0/1008899240 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000e040 con 0x7f2368101fb0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236819d7a0 con 0x7f2368101fb0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f2365ffb700 1 -- 172.21.15.93:0/1008899240 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2360003e10 con 0x7f2368101fb0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f2365ffb700 1 -- 172.21.15.93:0/1008899240 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012a70 con 0x7f2368101fb0 2023-07-21T22:39:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.062+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681a5040 con 0x7f2368101fb0 2023-07-21T22:39:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.064+0000 7f2365ffb700 1 -- 172.21.15.93:0/1008899240 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f23600056a0 con 0x7f2368101fb0 2023-07-21T22:39:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.065+0000 7f2365ffb700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 0x7f23500a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.065+0000 7f2365ffb700 1 -- 172.21.15.93:0/1008899240 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f23600ce380 con 0x7f2368101fb0 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.065+0000 7f2367fff700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 0x7f23500a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.066+0000 7f2367fff700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 0x7f23500a37f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2358005950 tx=0x7f23580058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 Processor -- start 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 -- start start 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19660 0x56293af134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ae18c50 con 0x56293ad6f6b0 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ae6b9e0 con 0x56293af19210 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f237ad51b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ae64e70 con 0x56293af19660 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f236c968700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f236c968700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50002/0 (socket says 172.21.15.93:50002) 2023-07-21T22:39:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f236c968700 1 -- 172.21.15.93:0/3802462932 learned_addr learned my addr 172.21.15.93:0/3802462932 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.067+0000 7f2367fff700 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 -- 172.21.15.93:0/3802462932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19660 msgr2=0x56293af134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19660 0x56293af134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 -- 172.21.15.93:0/3802462932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 msgr2=0x56293af12f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 -- 172.21.15.93:0/3802462932 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293ac92d80 con 0x56293ad6f6b0 2023-07-21T22:39:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f236c968700 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f23600a3370 tx=0x7f236009b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86821b3b758e2a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f234effd700 1 -- 172.21.15.93:0/3802462932 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360022310 con 0x56293ad6f6b0 2023-07-21T22:39:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f234effd700 1 -- 172.21.15.93:0/3802462932 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f236009b4e0 con 0x56293ad6f6b0 2023-07-21T22:39:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.068+0000 7f234effd700 1 -- 172.21.15.93:0/3802462932 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360097d90 con 0x56293ad6f6b0 2023-07-21T22:39:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.069+0000 7f237ad51b80 1 -- 172.21.15.93:0/3802462932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 msgr2=0x56293af18cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.069+0000 7f237ad51b80 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f23600a3370 tx=0x7f236009b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 -- 172.21.15.93:0/3802462932 shutdown_connections 2023-07-21T22:39:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19660 0x56293af134a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293ad6f6b0 0x56293af18cd0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 --2- 172.21.15.93:0/3802462932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 -- 172.21.15.93:0/3802462932 >> 172.21.15.93:0/3802462932 conn(0x56293af08ae0 msgr2=0x56293af0e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 -- 172.21.15.93:0/3802462932 shutdown_connections 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 -- 172.21.15.93:0/3802462932 wait complete. 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 -- start start 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.072+0000 7f237ad51b80 1 Processor -- start 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af19ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af1a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ae04a60 con 0x56293af19660 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ae04760 con 0x56293ad6f6b0 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293ace88c0 con 0x56293af19210 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f237ad51b80 10 client.0 _ll_register_callbacks cb 0x56293ae02700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f236c968700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f236c968700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af21b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32938/0 (socket says 172.21.15.93:32938) 2023-07-21T22:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.073+0000 7f236c968700 1 -- 172.21.15.93:0/1682290290 learned_addr learned my addr 172.21.15.93:0/1682290290 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f236d169700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af1a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af19ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 -- 172.21.15.93:0/1682290290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 msgr2=0x56293af21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af21b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 -- 172.21.15.93:0/1682290290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 msgr2=0x56293af1a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af1a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.075+0000 7f2367fff700 1 -- 172.21.15.93:0/1682290290 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681a5040 con 0x56293af19210 2023-07-21T22:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.079+0000 7f2367fff700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af19ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2368199820 tx=0x7f2358004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afdd54dd18806e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.079+0000 7f2347fff700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:39:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.079+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358010e50 con 0x56293af19210 2023-07-21T22:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.079+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f235801e3f0 con 0x56293af19210 2023-07-21T22:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.079+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56293adbe490 con 0x56293af19210 2023-07-21T22:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.080+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f235800dea0 con 0x56293af19210 2023-07-21T22:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.080+0000 7f237ad51b80 10 client.27148 fetch_fsmap learned FSMap version 89 2023-07-21T22:39:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.080+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x56293ac7bc00 con 0x56293af19210 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.080+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56293adbe490 con 0x56293af19210 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2358024020 con 0x56293af19210 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 10 client.27148 fetch_fsmap finished waiting for FSMap version 89 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 10 client.27148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 20 client.27148 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293af28790 0x56293af2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 4 client.27148 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:39:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.081+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56293ae5de50 con 0x56293af28790 2023-07-21T22:39:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.082+0000 7f236c968700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293af28790 0x56293af2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.083+0000 7f236c968700 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293af28790 0x56293af2aba0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4efab0df1d3696d server_cookie=c353195f7c048cf3 in_seq=0 out_seq=0 2023-07-21T22:39:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.083+0000 7f2347fff700 10 client.27148 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.084+0000 7f2347fff700 1 -- 172.21.15.93:0/1682290290 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56293ae5de50 con 0x56293af28790 2023-07-21T22:39:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.084+0000 7f2347fff700 10 client.27148 handle_command_reply: tid=0 2023-07-21T22:39:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.084+0000 7f237ad51b80 1 client.27148 shutdown 2023-07-21T22:39:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.084+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 msgr2=0x56293af19ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.084+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af19ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2368199820 tx=0x7f2358004750 comp rx=0 tx=0).stop 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 shutdown_connections 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293af28790 0x56293af2aba0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af1a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 --2- 172.21.15.93:0/1682290290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af19ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 >> 172.21.15.93:0/1682290290 conn(0x56293af08ae0 msgr2=0x56293af0d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.086+0000 7f2347fff700 0 client.27148 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.087+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 shutdown_connections 2023-07-21T22:39:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.087+0000 7f237ad51b80 1 -- 172.21.15.93:0/1682290290 wait complete. 2023-07-21T22:39:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.087+0000 7f237ad51b80 20 client.27148 trim_cache size 0 max 16384 2023-07-21T22:39:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 Processor -- start 2023-07-21T22:39:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 -- start start 2023-07-21T22:39:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 0x56293af1a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af13a70 con 0x56293af19660 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af25ee0 con 0x56293af19210 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f237ad51b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af0ecb0 con 0x56293ad6f6b0 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.111+0000 7f236d169700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50018/0 (socket says 172.21.15.93:50018) 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 -- 172.21.15.93:0/2720745931 learned_addr learned my addr 172.21.15.93:0/2720745931 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236c968700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 0x56293af1a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f2367fff700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 -- 172.21.15.93:0/2720745931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 msgr2=0x56293af1a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 0x56293af1a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 -- 172.21.15.93:0/2720745931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 msgr2=0x56293af12f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 -- 172.21.15.93:0/2720745931 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293af2f7d0 con 0x56293af19660 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236c968700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 0x56293af1a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f236d169700 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f235c009860 tx=0x7f235c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9989d577ea1685b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f234f7fe700 1 -- 172.21.15.93:0/2720745931 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c013680 con 0x56293af19660 2023-07-21T22:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f234f7fe700 1 -- 172.21.15.93:0/2720745931 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f235c013ca0 con 0x56293af19660 2023-07-21T22:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.112+0000 7f234f7fe700 1 -- 172.21.15.93:0/2720745931 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c012d80 con 0x56293af19660 2023-07-21T22:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 -- 172.21.15.93:0/2720745931 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 msgr2=0x56293af39430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f235c009860 tx=0x7f235c009b70 comp rx=0 tx=0).stop 2023-07-21T22:39:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 -- 172.21.15.93:0/2720745931 shutdown_connections 2023-07-21T22:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af39430 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293ad6f6b0 0x56293af1a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 --2- 172.21.15.93:0/2720745931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293af19210 0x56293af12f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.113+0000 7f237ad51b80 1 -- 172.21.15.93:0/2720745931 >> 172.21.15.93:0/2720745931 conn(0x56293af08ae0 msgr2=0x56293af19aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.116+0000 7f237ad51b80 1 -- 172.21.15.93:0/2720745931 shutdown_connections 2023-07-21T22:39:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.116+0000 7f237ad51b80 1 -- 172.21.15.93:0/2720745931 wait complete. 2023-07-21T22:39:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 -- start start 2023-07-21T22:39:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 Processor -- start 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af36410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af36950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af36e90 con 0x56293af19660 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af37000 con 0x56293ad6f6b0 2023-07-21T22:39:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293af371c0 con 0x56293af19210 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.117+0000 7f237ad51b80 10 client.0 _ll_register_callbacks cb 0x7f235800da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32952/0 (socket says 172.21.15.93:32952) 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 -- 172.21.15.93:0/490058835 learned_addr learned my addr 172.21.15.93:0/490058835 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 -- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 msgr2=0x56293af36410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af36410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 -- 172.21.15.93:0/490058835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 msgr2=0x56293af36950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af36950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 -- 172.21.15.93:0/490058835 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56293ac92d80 con 0x56293ad6f6b0 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f236c968700 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2360097f50 tx=0x7f2360097bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b700f2c4650041 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.118+0000 7f2347fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236009f510 con 0x56293ad6f6b0 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236009b4e0 con 0x56293ad6f6b0 2023-07-21T22:39:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2360099880 con 0x56293ad6f6b0 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23600860b0 con 0x56293ad6f6b0 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 10 client.27239 fetch_fsmap learned FSMap version 89 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x7f235c009540 con 0x56293ad6f6b0 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f236009b4e0 con 0x56293ad6f6b0 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2360019070 con 0x56293ad6f6b0 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 89 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 10 client.27239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 20 client.27239 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293afa2ee0 0x56293afa5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 4 client.27239 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:39:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.119+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56293ae1fff0 con 0x56293afa2ee0 2023-07-21T22:39:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.120+0000 7f2367fff700 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293afa2ee0 0x56293afa5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.121+0000 7f2347fff700 10 client.27239 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.121+0000 7f2367fff700 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293afa2ee0 0x56293afa5330 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554207ec6f25adea server_cookie=50ee27cb4335fcf0 in_seq=0 out_seq=0 2023-07-21T22:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:39:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f2347fff700 1 -- 172.21.15.93:0/490058835 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56293ae1fff0 con 0x56293afa2ee0 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f2347fff700 10 client.27239 handle_command_reply: tid=0 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 client.27239 shutdown 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 msgr2=0x56293af39440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2360097f50 tx=0x7f2360097bf0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 shutdown_connections 2023-07-21T22:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x56293afa2ee0 0x56293afa5330 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56293af19660 0x56293af36950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56293ad6f6b0 0x56293af39440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f2347fff700 0 client.27239 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 --2- 172.21.15.93:0/490058835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56293af19210 0x56293af36410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 >> 172.21.15.93:0/490058835 conn(0x56293af08ae0 msgr2=0x56293af221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 shutdown_connections 2023-07-21T22:39:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 1 -- 172.21.15.93:0/490058835 wait complete. 2023-07-21T22:39:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.322+0000 7f237ad51b80 20 client.27239 trim_cache size 0 max 16384 2023-07-21T22:39:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 msgr2=0x7f23500a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 0x7f23500a37f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2358005950 tx=0x7f23580058e0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 msgr2=0x7f2368198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f23600078a0 tx=0x7f2360007980 comp rx=0 tx=0).stop 2023-07-21T22:39:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 shutdown_connections 2023-07-21T22:39:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2368106ff0 0x7f236819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23500a1340 0x7f23500a37f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2368101fb0 0x7f2368198000 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 --2- 172.21.15.93:0/1008899240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2368102950 0x7f2368198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 >> 172.21.15.93:0/1008899240 conn(0x7f23680fd800 msgr2=0x7f23681058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 shutdown_connections 2023-07-21T22:39:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:38.323+0000 7f236ebcc700 1 -- 172.21.15.93:0/1008899240 wait complete. 2023-07-21T22:39:38.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 340909 28.69 MB/sec execute 494 sec latency 2119.842 ms 2023-07-21T22:39:39.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 341423 28.66 MB/sec execute 495 sec latency 3119.869 ms 2023-07-21T22:39:39.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:39 smithi194 ceph-mon[98288]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 857 op/s 2023-07-21T22:39:39.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:39 smithi134 ceph-mon[97873]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 857 op/s 2023-07-21T22:39:39.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:39 smithi093 ceph-mon[96579]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 857 op/s 2023-07-21T22:39:40.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 342068 28.68 MB/sec execute 496 sec latency 4119.900 ms 2023-07-21T22:39:41.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 342689 28.70 MB/sec execute 497 sec latency 5119.931 ms 2023-07-21T22:39:41.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:41 smithi194 ceph-mon[98288]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T22:39:41.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:41 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:41.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:41 smithi134 ceph-mon[97873]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T22:39:41.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:41 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:41.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:41 smithi093 ceph-mon[96579]: pgmap v741: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 602 op/s 2023-07-21T22:39:41.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:41 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:42.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 343124 28.69 MB/sec execute 498 sec latency 6119.962 ms 2023-07-21T22:39:43.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 343531 28.69 MB/sec execute 499 sec latency 7120.001 ms 2023-07-21T22:39:43.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:43 smithi194 ceph-mon[98288]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 631 op/s 2023-07-21T22:39:43.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:43 smithi134 ceph-mon[97873]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 631 op/s 2023-07-21T22:39:43.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:43 smithi093 ceph-mon[96579]: pgmap v742: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 17 MiB/s wr, 631 op/s 2023-07-21T22:39:44.564 INFO:tasks.workunit.client.0.smithi093.stdout: 10 343940 28.65 MB/sec execute 500 sec latency 8120.032 ms 2023-07-21T22:39:45.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 344449 28.65 MB/sec execute 501 sec latency 8770.706 ms 2023-07-21T22:39:45.854 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:45 smithi194 ceph-mon[98288]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 768 op/s 2023-07-21T22:39:45.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:45 smithi134 ceph-mon[97873]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 768 op/s 2023-07-21T22:39:45.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:45 smithi093 ceph-mon[96579]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 18 MiB/s wr, 768 op/s 2023-07-21T22:39:46.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 345117 28.66 MB/sec execute 502 sec latency 94.821 ms 2023-07-21T22:39:46.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:46 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:46.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:46 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:46.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:46 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:47.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 345725 28.67 MB/sec execute 503 sec latency 101.366 ms 2023-07-21T22:39:47.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:47 smithi194 ceph-mon[98288]: pgmap v744: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:39:47.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:47 smithi134 ceph-mon[97873]: pgmap v744: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:39:47.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:47 smithi093 ceph-mon[96579]: pgmap v744: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:39:48.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 346259 28.69 MB/sec execute 504 sec latency 93.104 ms 2023-07-21T22:39:49.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 346601 28.67 MB/sec execute 505 sec latency 159.864 ms 2023-07-21T22:39:49.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:49 smithi194 ceph-mon[98288]: pgmap v745: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 679 op/s 2023-07-21T22:39:49.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:49 smithi134 ceph-mon[97873]: pgmap v745: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 679 op/s 2023-07-21T22:39:49.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:49 smithi093 ceph-mon[96579]: pgmap v745: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 679 op/s 2023-07-21T22:39:50.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 347123 28.66 MB/sec execute 506 sec latency 83.439 ms 2023-07-21T22:39:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.493+0000 7f2ba6f77700 1 -- 172.21.15.93:0/2957281364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba010e2a0 msgr2=0x7f2ba010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.493+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/2957281364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba010e2a0 0x7f2ba010e670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098f0 tx=0x7f2b90009c00 comp rx=0 tx=0).stop 2023-07-21T22:39:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.496+0000 7f2ba6f77700 1 -- 172.21.15.93:0/2957281364 shutdown_connections 2023-07-21T22:39:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.496+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/2957281364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006d3a0 0x7f2ba011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.496+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/2957281364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba010e2a0 0x7f2ba010e670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.496+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/2957281364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba006c910 0x7f2ba006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.496+0000 7f2ba6f77700 1 -- 172.21.15.93:0/2957281364 >> 172.21.15.93:0/2957281364 conn(0x7f2ba006bd70 msgr2=0x7f2ba0070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.499+0000 7f2ba6f77700 1 -- 172.21.15.93:0/2957281364 shutdown_connections 2023-07-21T22:39:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.499+0000 7f2ba6f77700 1 -- 172.21.15.93:0/2957281364 wait complete. 2023-07-21T22:39:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 Processor -- start 2023-07-21T22:39:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 -- start start 2023-07-21T22:39:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 0x7f2ba0116ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba006d3a0 0x7f2ba0117430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 0x7f2ba01a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01a1570 con 0x7f2ba006c910 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01a16e0 con 0x7f2ba006d3a0 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.500+0000 7f2ba6f77700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01a18a0 con 0x7f2ba010e2a0 2023-07-21T22:39:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 0x7f2ba01a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 0x7f2ba01a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37116/0 (socket says 172.21.15.93:37116) 2023-07-21T22:39:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba5514700 1 -- 172.21.15.93:0/715016288 learned_addr learned my addr 172.21.15.93:0/715016288 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 0x7f2ba0116ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2b9ffff700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba006d3a0 0x7f2ba0117430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 -- 172.21.15.93:0/715016288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 msgr2=0x7f2ba01a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 0x7f2ba01a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 -- 172.21.15.93:0/715016288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba006d3a0 msgr2=0x7f2ba0117430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba006d3a0 0x7f2ba0117430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.501+0000 7f2ba4d13700 1 -- 172.21.15.93:0/715016288 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009520 con 0x7f2ba006c910 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 0x7f2ba0116ef0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009230 tx=0x7f2b9000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2b9dffb700 1 -- 172.21.15.93:0/715016288 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9001c070 con 0x7f2ba006c910 2023-07-21T22:39:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba01a1b20 con 0x7f2ba006c910 2023-07-21T22:39:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2b9dffb700 1 -- 172.21.15.93:0/715016288 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9000baa0 con 0x7f2ba006c910 2023-07-21T22:39:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2b9dffb700 1 -- 172.21.15.93:0/715016288 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900038e0 con 0x7f2ba006c910 2023-07-21T22:39:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.502+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba01a92b0 con 0x7f2ba006c910 2023-07-21T22:39:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.505+0000 7f2b9dffb700 1 -- 172.21.15.93:0/715016288 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2b90020030 con 0x7f2ba006c910 2023-07-21T22:39:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.506+0000 7f2b9dffb700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 0x7f2b880a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.506+0000 7f2b9dffb700 1 -- 172.21.15.93:0/715016288 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2b900cda40 con 0x7f2ba006c910 2023-07-21T22:39:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.506+0000 7f2b9ffff700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 0x7f2b880a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.506+0000 7f2b9ffff700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 0x7f2b880a3830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2ba01185c0 tx=0x7f2b94009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 Processor -- start 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 -- start start 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25278c50 con 0x55bb25379660 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb252cb9e0 con 0x55bb25379210 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb252c4e70 con 0x55bb251cf6b0 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50044/0 (socket says 172.21.15.93:50044) 2023-07-21T22:39:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 -- 172.21.15.93:0/387272346 learned_addr learned my addr 172.21.15.93:0/387272346 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 -- 172.21.15.93:0/387272346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 msgr2=0x55bb25378cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 -- 172.21.15.93:0/387272346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 msgr2=0x55bb25372f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.507+0000 7f2ba5514700 1 -- 172.21.15.93:0/387272346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb250f2d80 con 0x55bb25379660 2023-07-21T22:39:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.509+0000 7f2ba5514700 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801a520 tx=0x7f2b9801a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fcddc688976a6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.509+0000 7f2b86ffd700 1 -- 172.21.15.93:0/387272346 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98018e90 con 0x55bb25379660 2023-07-21T22:39:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.509+0000 7f2b86ffd700 1 -- 172.21.15.93:0/387272346 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b98020040 con 0x55bb25379660 2023-07-21T22:39:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.509+0000 7f2b86ffd700 1 -- 172.21.15.93:0/387272346 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98028ca0 con 0x55bb25379660 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/387272346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 msgr2=0x55bb253734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801a520 tx=0x7f2b9801a9f0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/387272346 shutdown_connections 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb253734a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/387272346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/387272346 >> 172.21.15.93:0/387272346 conn(0x55bb25368ae0 msgr2=0x55bb2536e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/387272346 shutdown_connections 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/387272346 wait complete. 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 -- start start 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.510+0000 7f2bb30fcb80 1 Processor -- start 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb2537cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25264a60 con 0x55bb25379660 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25264760 con 0x55bb25379210 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb251488c0 con 0x55bb251cf6b0 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2bb30fcb80 10 client.0 _ll_register_callbacks cb 0x55bb25262700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb2537cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50052/0 (socket says 172.21.15.93:50052) 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb2537cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32980/0 (socket says 172.21.15.93:32980) 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 -- 172.21.15.93:0/569825647 learned_addr learned my addr 172.21.15.93:0/569825647 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 -- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 msgr2=0x55bb2537c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 -- 172.21.15.93:0/569825647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 msgr2=0x55bb2537cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb2537cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.511+0000 7f2ba5514700 1 -- 172.21.15.93:0/569825647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9801a200 con 0x55bb25379660 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2ba5514700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801c010 tx=0x7f2b98003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa2c7c71f4c5756 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98028b10 con 0x55bb25379660 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2b98028c70 con 0x55bb25379660 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb2521e490 con 0x55bb25379660 2023-07-21T22:39:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b98028e30 con 0x55bb25379660 2023-07-21T22:39:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2bb30fcb80 10 client.17949 fetch_fsmap learned FSMap version 91 2023-07-21T22:39:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55bb250dbc00 con 0x55bb25379660 2023-07-21T22:39:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.512+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb2521e490 con 0x55bb25379660 2023-07-21T22:39:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2b98003f30 con 0x55bb25379660 2023-07-21T22:39:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 10 client.17949 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:39:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 10 client.17949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 20 client.17949 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb2538ca40 0x55bb2538ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 4 client.17949 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb252bde50 con 0x55bb2538ca40 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.514+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb2538ca40 0x55bb2538ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.515+0000 7f2b7ffff700 10 client.17949 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.515+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb2538ca40 0x55bb2538ee50 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08721b7ad546e66 server_cookie=5dab0ecf1c07dd29 in_seq=0 out_seq=0 2023-07-21T22:39:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.516+0000 7f2b7ffff700 1 -- 172.21.15.93:0/569825647 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb252bde50 con 0x55bb2538ca40 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.516+0000 7f2b7ffff700 10 client.17949 handle_command_reply: tid=0 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.516+0000 7f2bb30fcb80 1 client.17949 shutdown 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.516+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 msgr2=0x55bb2537d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.516+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801c010 tx=0x7f2b98003d40 comp rx=0 tx=0).stop 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 shutdown_connections 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb2538ca40 0x55bb2538ee50 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379660 0x55bb2537d320 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2b7ffff700 0 client.17949 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/569825647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379210 0x55bb2537cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 >> 172.21.15.93:0/569825647 conn(0x55bb25368ae0 msgr2=0x55bb2536d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 shutdown_connections 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/569825647 wait complete. 2023-07-21T22:39:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.518+0000 7f2bb30fcb80 20 client.17949 trim_cache size 0 max 16384 2023-07-21T22:39:51.565 INFO:tasks.workunit.client.0.smithi093.stdout: 10 347640 28.66 MB/sec execute 507 sec latency 93.110 ms 2023-07-21T22:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.540+0000 7f2bb30fcb80 1 Processor -- start 2023-07-21T22:39:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.540+0000 7f2bb30fcb80 1 -- start start 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.540+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb25383ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2536ac00 con 0x55bb25379210 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25373b20 con 0x55bb25379660 2023-07-21T22:39:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25373480 con 0x55bb251cf6b0 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37136/0 (socket says 172.21.15.93:37136) 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 -- 172.21.15.93:0/2855320690 learned_addr learned my addr 172.21.15.93:0/2855320690 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba5514700 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb25383ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 -- 172.21.15.93:0/2855320690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 msgr2=0x55bb25383ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb25383ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 -- 172.21.15.93:0/2855320690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 msgr2=0x55bb25372f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 -- 172.21.15.93:0/2855320690 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb25393a80 con 0x55bb251cf6b0 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.541+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b90097610 tx=0x7f2b9000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e765be75f3e32f9a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.542+0000 7f2b877fe700 1 -- 172.21.15.93:0/2855320690 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9009ebd0 con 0x55bb251cf6b0 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.542+0000 7f2b877fe700 1 -- 172.21.15.93:0/2855320690 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b900ba040 con 0x55bb251cf6b0 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.542+0000 7f2b877fe700 1 -- 172.21.15.93:0/2855320690 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900050d0 con 0x55bb251cf6b0 2023-07-21T22:39:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2855320690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 msgr2=0x55bb25378180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b90097610 tx=0x7f2b9000fa70 comp rx=0 tx=0).stop 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2855320690 shutdown_connections 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb25383ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb25378180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2855320690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb25372f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2855320690 >> 172.21.15.93:0/2855320690 conn(0x55bb25368ae0 msgr2=0x55bb25379b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.543+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2855320690 shutdown_connections 2023-07-21T22:39:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2855320690 wait complete. 2023-07-21T22:39:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 -- start start 2023-07-21T22:39:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 Processor -- start 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb2537ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb25380860 con 0x55bb25379210 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb253dfe30 con 0x55bb25379660 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb253dff70 con 0x55bb251cf6b0 2023-07-21T22:39:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.544+0000 7f2bb30fcb80 10 client.0 _ll_register_callbacks cb 0x55bb25262700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb2537ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:32996/0 (socket says 172.21.15.93:32996) 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2b9ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb2537ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50064/0 (socket says 172.21.15.93:50064) 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 -- 172.21.15.93:0/2918688157 learned_addr learned my addr 172.21.15.93:0/2918688157 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 -- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 msgr2=0x55bb2537fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 -- 172.21.15.93:0/2918688157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 msgr2=0x55bb2537ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb2537ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 -- 172.21.15.93:0/2918688157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba01a92b0 con 0x55bb25379660 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2ba5514700 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801aa10 tx=0x7f2b98003fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c97beb7ca8a1bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9801adb0 con 0x55bb25379660 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b98020040 con 0x55bb25379660 2023-07-21T22:39:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.545+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2b98004880 con 0x55bb25379660 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b98004380 con 0x55bb25379660 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 10 client.27254 fetch_fsmap learned FSMap version 91 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55bb251191f0 con 0x55bb25379660 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b98051080 con 0x55bb25379660 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2b9804b030 con 0x55bb25379660 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 10 client.27254 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:39:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 10 client.27254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 20 client.27254 populate_metadata read hostname 'smithi093' 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb253e1200 0x55bb253e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 4 client.27254 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bb2527fff0 con 0x55bb253e1200 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.546+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb253e1200 0x55bb253e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.547+0000 7f2b7ffff700 10 client.27254 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.547+0000 7f2ba4d13700 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb253e1200 0x55bb253e3610 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc071173f817e7a server_cookie=e20c4ba5224706b3 in_seq=0 out_seq=0 2023-07-21T22:39:51.839 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:51 smithi194 ceph-mon[98288]: pgmap v746: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2023-07-21T22:39:51.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:51 smithi134 ceph-mon[97873]: pgmap v746: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2023-07-21T22:39:51.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:51 smithi093 ceph-mon[96579]: pgmap v746: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 393 op/s 2023-07-21T22:39:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2b7ffff700 1 -- 172.21.15.93:0/2918688157 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bb2527fff0 con 0x55bb253e1200 2023-07-21T22:39:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2b7ffff700 10 client.27254 handle_command_reply: tid=0 2023-07-21T22:39:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 client.27254 shutdown 2023-07-21T22:39:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 msgr2=0x55bb253df8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b9801aa10 tx=0x7f2b98003fe0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 shutdown_connections 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55bb253e1200 0x55bb253e3610 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb25379660 0x55bb253df8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2b7ffff700 0 client.27254 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bb251cf6b0 0x55bb2537fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 --2- 172.21.15.93:0/2918688157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55bb25379210 0x55bb2537ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 >> 172.21.15.93:0/2918688157 conn(0x55bb25368ae0 msgr2=0x55bb25377ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 shutdown_connections 2023-07-21T22:39:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.906+0000 7f2bb30fcb80 1 -- 172.21.15.93:0/2918688157 wait complete. 2023-07-21T22:39:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:39:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:39:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:39:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:39:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.907+0000 7f2bb30fcb80 20 client.27254 trim_cache size 0 max 16384 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.907+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 msgr2=0x7f2b880a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.907+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 0x7f2b880a3830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2ba01185c0 tx=0x7f2b94009260 comp rx=0 tx=0).stop 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.907+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 msgr2=0x7f2ba0116ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.907+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 0x7f2ba0116ef0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009230 tx=0x7f2b9000fa00 comp rx=0 tx=0).stop 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 shutdown_connections 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ba010e2a0 0x7f2ba01a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f2b880a1380 0x7f2b880a3830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ba006c910 0x7f2ba0116ef0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 --2- 172.21.15.93:0/715016288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba006d3a0 0x7f2ba0117430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 >> 172.21.15.93:0/715016288 conn(0x7f2ba006bd70 msgr2=0x7f2ba0070780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 shutdown_connections 2023-07-21T22:39:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:39:51.909+0000 7f2ba6f77700 1 -- 172.21.15.93:0/715016288 wait complete. 2023-07-21T22:39:52.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 347995 28.64 MB/sec execute 508 sec latency 281.939 ms 2023-07-21T22:39:53.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 348638 28.64 MB/sec execute 509 sec latency 85.888 ms 2023-07-21T22:39:53.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:53 smithi194 ceph-mon[98288]: pgmap v747: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.4 MiB/s wr, 476 op/s 2023-07-21T22:39:53.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:53 smithi134 ceph-mon[97873]: pgmap v747: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.4 MiB/s wr, 476 op/s 2023-07-21T22:39:53.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:53 smithi093 ceph-mon[96579]: pgmap v747: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.4 MiB/s wr, 476 op/s 2023-07-21T22:39:54.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 349327 28.66 MB/sec execute 510 sec latency 82.014 ms 2023-07-21T22:39:55.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 349699 28.65 MB/sec execute 511 sec latency 123.782 ms 2023-07-21T22:39:55.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:55 smithi194 ceph-mon[98288]: pgmap v748: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 620 op/s 2023-07-21T22:39:55.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:55 smithi134 ceph-mon[97873]: pgmap v748: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 620 op/s 2023-07-21T22:39:55.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:55 smithi093 ceph-mon[96579]: pgmap v748: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 620 op/s 2023-07-21T22:39:56.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 350274 28.66 MB/sec execute 512 sec latency 99.731 ms 2023-07-21T22:39:57.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 350761 28.65 MB/sec execute 513 sec latency 100.845 ms 2023-07-21T22:39:57.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:57 smithi134 ceph-mon[97873]: pgmap v749: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2023-07-21T22:39:57.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:57 smithi093 ceph-mon[96579]: pgmap v749: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2023-07-21T22:39:58.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:57 smithi194 ceph-mon[98288]: pgmap v749: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.4 MiB/s wr, 425 op/s 2023-07-21T22:39:58.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 351291 28.66 MB/sec execute 514 sec latency 84.047 ms 2023-07-21T22:39:59.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 351893 28.65 MB/sec execute 515 sec latency 80.895 ms 2023-07-21T22:39:59.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:39:59 smithi134 ceph-mon[97873]: pgmap v750: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 604 op/s 2023-07-21T22:39:59.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:39:59 smithi093 ceph-mon[96579]: pgmap v750: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 604 op/s 2023-07-21T22:40:00.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:39:59 smithi194 ceph-mon[98288]: pgmap v750: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 604 op/s 2023-07-21T22:40:00.440 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:39:59.999+0000 7ff4dc340700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:40:00.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352299 28.64 MB/sec execute 516 sec latency 229.774 ms 2023-07-21T22:40:00.880 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:00 smithi134 ceph-mon[97873]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:40:00.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:00 smithi093 ceph-mon[96579]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:40:01.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:00 smithi194 ceph-mon[98288]: overall HEALTH_ERR 2 MDSs report damaged metadata 2023-07-21T22:40:01.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352308 28.59 MB/sec execute 517 sec latency 1229.808 ms 2023-07-21T22:40:01.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:01 smithi134 ceph-mon[97873]: pgmap v751: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 434 op/s 2023-07-21T22:40:01.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:01 smithi093 ceph-mon[96579]: pgmap v751: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 434 op/s 2023-07-21T22:40:02.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:01 smithi194 ceph-mon[98288]: pgmap v751: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.2 MiB/s wr, 434 op/s 2023-07-21T22:40:02.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352308 28.53 MB/sec execute 518 sec latency 2229.901 ms 2023-07-21T22:40:03.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352308 28.48 MB/sec execute 519 sec latency 3230.005 ms 2023-07-21T22:40:03.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:03 smithi134 ceph-mon[97873]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 509 op/s 2023-07-21T22:40:03.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:03 smithi093 ceph-mon[96579]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 509 op/s 2023-07-21T22:40:04.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:03 smithi194 ceph-mon[98288]: pgmap v752: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.9 MiB/s wr, 509 op/s 2023-07-21T22:40:04.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352592 28.45 MB/sec execute 520 sec latency 3508.879 ms 2023-07-21T22:40:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 -- 172.21.15.93:0/364283791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0000fea30 msgr2=0x7fb0000feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 --2- 172.21.15.93:0/364283791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0000fea30 0x7fb0000feea0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7faff800b0a0 tx=0x7faff800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 -- 172.21.15.93:0/364283791 shutdown_connections 2023-07-21T22:40:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 --2- 172.21.15.93:0/364283791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0000ff3e0 0x7fb00010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 --2- 172.21.15.93:0/364283791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb000104a30 0x7fb000104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 --2- 172.21.15.93:0/364283791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0000fea30 0x7fb0000feea0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 -- 172.21.15.93:0/364283791 >> 172.21.15.93:0/364283791 conn(0x7fb000074760 msgr2=0x7fb000074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 -- 172.21.15.93:0/364283791 shutdown_connections 2023-07-21T22:40:05.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.075+0000 7fb005fff700 1 -- 172.21.15.93:0/364283791 wait complete. 2023-07-21T22:40:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 Processor -- start 2023-07-21T22:40:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 -- start start 2023-07-21T22:40:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0000fea30 0x7fb00019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0000ff3e0 0x7fb00019d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00010ad80 con 0x7fb000104a30 2023-07-21T22:40:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00010aef0 con 0x7fb0000fea30 2023-07-21T22:40:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.076+0000 7fb005fff700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00010b0b0 con 0x7fb0000ff3e0 2023-07-21T22:40:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50080/0 (socket says 172.21.15.93:50080) 2023-07-21T22:40:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 -- 172.21.15.93:0/3726638358 learned_addr learned my addr 172.21.15.93:0/3726638358 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 -- 172.21.15.93:0/3726638358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0000ff3e0 msgr2=0x7fb00019d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0000ff3e0 0x7fb00019d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 -- 172.21.15.93:0/3726638358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0000fea30 msgr2=0x7fb00019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0000fea30 0x7fb00019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 -- 172.21.15.93:0/3726638358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8009d00 con 0x7fb000104a30 2023-07-21T22:40:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffffff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7faff400e940 tx=0x7faff400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7faffcff9700 1 -- 172.21.15.93:0/3726638358 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40097b0 con 0x7fb000104a30 2023-07-21T22:40:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00010b330 con 0x7fb000104a30 2023-07-21T22:40:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.078+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000199d50 con 0x7fb000104a30 2023-07-21T22:40:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.079+0000 7faffcff9700 1 -- 172.21.15.93:0/3726638358 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff4004cf0 con 0x7fb000104a30 2023-07-21T22:40:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.079+0000 7faffcff9700 1 -- 172.21.15.93:0/3726638358 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40055d0 con 0x7fb000104a30 2023-07-21T22:40:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 Processor -- start 2023-07-21T22:40:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 -- start start 2023-07-21T22:40:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7447f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb734dc50 con 0x55ccb72a46b0 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb73a09e0 con 0x55ccb744e660 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.083+0000 7fb012184b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7399e70 con 0x55ccb744e210 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33014/0 (socket says 172.21.15.93:33014) 2023-07-21T22:40:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 -- 172.21.15.93:0/288204031 learned_addr learned my addr 172.21.15.93:0/288204031 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7fafff7fe700 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7447f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 -- 172.21.15.93:0/288204031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 msgr2=0x55ccb7447f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7447f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 -- 172.21.15.93:0/288204031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 msgr2=0x55ccb744dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 -- 172.21.15.93:0/288204031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccb71c7d80 con 0x55ccb744e660 2023-07-21T22:40:05.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7fafff7fe700 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7447f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:40:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.084+0000 7faffeffd700 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faff800b780 tx=0x7faff8005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caed9e7840ca9754 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.085+0000 7fafe5ffb700 1 -- 172.21.15.93:0/288204031 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800e070 con 0x55ccb744e660 2023-07-21T22:40:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.085+0000 7fafe5ffb700 1 -- 172.21.15.93:0/288204031 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff8009460 con 0x55ccb744e660 2023-07-21T22:40:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.085+0000 7fafe5ffb700 1 -- 172.21.15.93:0/288204031 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80054f0 con 0x55ccb744e660 2023-07-21T22:40:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7faffcff9700 1 -- 172.21.15.93:0/3726638358 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faff400cbd0 con 0x7fb000104a30 2023-07-21T22:40:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 -- 172.21.15.93:0/288204031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 msgr2=0x55ccb74484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faff800b780 tx=0x7faff8005a60 comp rx=0 tx=0).stop 2023-07-21T22:40:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 -- 172.21.15.93:0/288204031 shutdown_connections 2023-07-21T22:40:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7faffcff9700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 0x7fafe80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74484a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 --2- 172.21.15.93:0/288204031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7447f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7fb012184b80 1 -- 172.21.15.93:0/288204031 >> 172.21.15.93:0/288204031 conn(0x55ccb743dae0 msgr2=0x55ccb74430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7faffcff9700 1 -- 172.21.15.93:0/3726638358 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faff4014070 con 0x7fb000104a30 2023-07-21T22:40:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.086+0000 7faffeffd700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 0x7fafe80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.087+0000 7fb012184b80 1 -- 172.21.15.93:0/288204031 shutdown_connections 2023-07-21T22:40:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.087+0000 7fb012184b80 1 -- 172.21.15.93:0/288204031 wait complete. 2023-07-21T22:40:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.087+0000 7faffeffd700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 0x7fafe80a37f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faff8003ea0 tx=0x7faff8006000 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.087+0000 7fb012184b80 1 -- start start 2023-07-21T22:40:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.087+0000 7fb012184b80 1 Processor -- start 2023-07-21T22:40:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb72a46b0 0x55ccb7454520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e660 0x55ccb7451dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7339a60 con 0x55ccb744e210 2023-07-21T22:40:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7339760 con 0x55ccb72a46b0 2023-07-21T22:40:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb721d8c0 con 0x55ccb744e660 2023-07-21T22:40:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7fb012184b80 10 client.0 _ll_register_callbacks cb 0x55ccb7337700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7faffeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7faffeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50092/0 (socket says 172.21.15.93:50092) 2023-07-21T22:40:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7faffeffd700 1 -- 172.21.15.93:0/4179125642 learned_addr learned my addr 172.21.15.93:0/4179125642 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.089+0000 7faffffff700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e660 0x55ccb7451dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 -- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e660 msgr2=0x55ccb7451dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e660 0x55ccb7451dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 -- 172.21.15.93:0/4179125642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb72a46b0 msgr2=0x55ccb7454520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb72a46b0 0x55ccb7454520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 -- 172.21.15.93:0/4179125642 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000067900 con 0x55ccb744e210 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7faffeffd700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7faff80060b0 tx=0x7faff80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f71c4b64eb9410 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7fafdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8029070 con 0x55ccb744e210 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faff8005520 con 0x55ccb744e210 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccb72f3490 con 0x55ccb744e210 2023-07-21T22:40:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.090+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7faff800e070 con 0x55ccb744e210 2023-07-21T22:40:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 10 client.17961 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55ccb71b0c00 con 0x55ccb744e210 2023-07-21T22:40:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff803b020 con 0x55ccb744e210 2023-07-21T22:40:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faff8023400 con 0x55ccb744e210 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 10 client.17961 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 10 client.17961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 20 client.17961 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb7461770 0x55ccb7463b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 4 client.17961 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:40:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccb7392e50 con 0x55ccb7461770 2023-07-21T22:40:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.091+0000 7fafff7fe700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb7461770 0x55ccb7463b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.092+0000 7fafdeffd700 10 client.17961 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.092+0000 7fafff7fe700 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb7461770 0x55ccb7463b80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30808a944c12c17a server_cookie=90f1c9171fcd45de in_seq=0 out_seq=0 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.092+0000 7fafdeffd700 1 -- 172.21.15.93:0/4179125642 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ccb7392e50 con 0x55ccb7461770 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.092+0000 7fafdeffd700 10 client.17961 handle_command_reply: tid=0 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.093+0000 7fb012184b80 1 client.17961 shutdown 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 msgr2=0x55ccb74517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7faff80060b0 tx=0x7faff80095d0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 shutdown_connections 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb7461770 0x55ccb7463b80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e660 0x55ccb7451dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fafdeffd700 0 client.17961 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb72a46b0 0x55ccb7454520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 --2- 172.21.15.93:0/4179125642 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e210 0x55ccb74517a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 >> 172.21.15.93:0/4179125642 conn(0x55ccb743dae0 msgr2=0x55ccb74423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 shutdown_connections 2023-07-21T22:40:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 1 -- 172.21.15.93:0/4179125642 wait complete. 2023-07-21T22:40:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.094+0000 7fb012184b80 20 client.17961 trim_cache size 0 max 16384 2023-07-21T22:40:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 Processor -- start 2023-07-21T22:40:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 -- start start 2023-07-21T22:40:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb74435e0 con 0x55ccb72a46b0 2023-07-21T22:40:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb744c3e0 con 0x55ccb744e660 2023-07-21T22:40:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.115+0000 7fb012184b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb745e400 con 0x55ccb744e210 2023-07-21T22:40:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37180/0 (socket says 172.21.15.93:37180) 2023-07-21T22:40:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 -- 172.21.15.93:0/4091550221 learned_addr learned my addr 172.21.15.93:0/4091550221 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7fafff7fe700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffffff700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 -- 172.21.15.93:0/4091550221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 msgr2=0x55ccb74430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 -- 172.21.15.93:0/4091550221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 msgr2=0x55ccb744a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 -- 172.21.15.93:0/4091550221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccb71ee1f0 con 0x55ccb744e210 2023-07-21T22:40:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7fafff7fe700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.116+0000 7faffeffd700 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faff800b780 tx=0x7faff8003ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41adb5e6e07dcd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.117+0000 7fafe67fc700 1 -- 172.21.15.93:0/4091550221 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8029070 con 0x55ccb744e210 2023-07-21T22:40:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.117+0000 7fafe67fc700 1 -- 172.21.15.93:0/4091550221 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff803b070 con 0x55ccb744e210 2023-07-21T22:40:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.117+0000 7fafe67fc700 1 -- 172.21.15.93:0/4091550221 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800e040 con 0x55ccb744e210 2023-07-21T22:40:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- 172.21.15.93:0/4091550221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 msgr2=0x55ccb7442b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faff800b780 tx=0x7faff8003ae0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- 172.21.15.93:0/4091550221 shutdown_connections 2023-07-21T22:40:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e660 0x55ccb74430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb72a46b0 0x55ccb744a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- 172.21.15.93:0/4091550221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb744e210 0x55ccb7442b60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- 172.21.15.93:0/4091550221 >> 172.21.15.93:0/4091550221 conn(0x55ccb743dae0 msgr2=0x55ccb744b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- 172.21.15.93:0/4091550221 shutdown_connections 2023-07-21T22:40:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- 172.21.15.93:0/4091550221 wait complete. 2023-07-21T22:40:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- start start 2023-07-21T22:40:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 Processor -- start 2023-07-21T22:40:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e210 0x55ccb7458ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e660 0x55ccb7456b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.118+0000 7fb012184b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7457110 con 0x55ccb744e660 2023-07-21T22:40:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fb012184b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7457280 con 0x55ccb744e210 2023-07-21T22:40:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fb012184b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccb7457440 con 0x55ccb72a46b0 2023-07-21T22:40:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fb012184b80 10 client.0 _ll_register_callbacks cb 0x7faff800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37184/0 (socket says 172.21.15.93:37184) 2023-07-21T22:40:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 -- 172.21.15.93:0/1560419930 learned_addr learned my addr 172.21.15.93:0/1560419930 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7faffffff700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e660 0x55ccb7456b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 -- 172.21.15.93:0/1560419930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e210 msgr2=0x55ccb7458ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e210 0x55ccb7458ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 -- 172.21.15.93:0/1560419930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e660 msgr2=0x55ccb7456b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e660 0x55ccb7456b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 -- 172.21.15.93:0/1560419930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff000e040 con 0x55ccb72a46b0 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafff7fe700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55ccb7464490 tx=0x7faff0016e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=378dd1ba21518a0c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.119+0000 7fafdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff00160d0 con 0x55ccb72a46b0 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faff0016230 con 0x55ccb72a46b0 2023-07-21T22:40:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faff8009460 con 0x55ccb72a46b0 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7faff00163f0 con 0x55ccb72a46b0 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fb012184b80 10 client.27178 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.121+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55ccb71b0c00 con 0x55ccb72a46b0 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff8009460 con 0x55ccb72a46b0 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faff0026070 con 0x55ccb72a46b0 2023-07-21T22:40:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 10 client.27178 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 10 client.27178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 20 client.27178 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb74b68d0 0x55ccb74b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 4 client.27178 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:40:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ccb7354ff0 con 0x55ccb74b68d0 2023-07-21T22:40:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.122+0000 7faffeffd700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb74b68d0 0x55ccb74b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.123+0000 7fafdeffd700 10 client.27178 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.123+0000 7faffeffd700 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb74b68d0 0x55ccb74b8ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1d06a7eece41e1 server_cookie=6f8819afe27e3786 in_seq=0 out_seq=0 2023-07-21T22:40:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fafdeffd700 1 -- 172.21.15.93:0/1560419930 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ccb7354ff0 con 0x55ccb74b68d0 2023-07-21T22:40:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fafdeffd700 10 client.27178 handle_command_reply: tid=0 2023-07-21T22:40:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 client.27178 shutdown 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 msgr2=0x55ccb7458590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55ccb7464490 tx=0x7faff0016e20 comp rx=0 tx=0).stop 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 shutdown_connections 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55ccb74b68d0 0x55ccb74b8ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ccb744e660 0x55ccb7456b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ccb72a46b0 0x55ccb7458590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 --2- 172.21.15.93:0/1560419930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccb744e210 0x55ccb7458ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 >> 172.21.15.93:0/1560419930 conn(0x55ccb743dae0 msgr2=0x55ccb744ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fafdeffd700 0 client.27178 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 shutdown_connections 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 1 -- 172.21.15.93:0/1560419930 wait complete. 2023-07-21T22:40:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.417+0000 7fb012184b80 20 client.27178 trim_cache size 0 max 16384 2023-07-21T22:40:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:40:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:40:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:40:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.418+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 msgr2=0x7fafe80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.418+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 0x7fafe80a37f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faff8003ea0 tx=0x7faff8006000 comp rx=0 tx=0).stop 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.418+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 msgr2=0x7fb00010a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.418+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7faff400e940 tx=0x7faff400ed00 comp rx=0 tx=0).stop 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 shutdown_connections 2023-07-21T22:40:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fafe80a1340 0x7fafe80a37f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb000104a30 0x7fb00010a770 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0000fea30 0x7fb00019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 --2- 172.21.15.93:0/3726638358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0000ff3e0 0x7fb00019d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 >> 172.21.15.93:0/3726638358 conn(0x7fb000074760 msgr2=0x7fb000100560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 shutdown_connections 2023-07-21T22:40:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:05.419+0000 7fb005fff700 1 -- 172.21.15.93:0/3726638358 wait complete. 2023-07-21T22:40:05.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352714 28.41 MB/sec execute 521 sec latency 336.579 ms 2023-07-21T22:40:05.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:05 smithi134 ceph-mon[97873]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 518 op/s 2023-07-21T22:40:05.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:05 smithi093 ceph-mon[96579]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 518 op/s 2023-07-21T22:40:06.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:05 smithi194 ceph-mon[98288]: pgmap v753: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 518 op/s 2023-07-21T22:40:06.563 INFO:tasks.workunit.client.0.smithi093.stdout: 10 352918 28.38 MB/sec execute 522 sec latency 173.330 ms 2023-07-21T22:40:07.570 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353046 28.33 MB/sec execute 523 sec latency 585.222 ms 2023-07-21T22:40:07.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:07 smithi093 ceph-mon[96579]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 345 op/s 2023-07-21T22:40:08.092 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:07 smithi194 ceph-mon[98288]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 345 op/s 2023-07-21T22:40:08.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:07 smithi134 ceph-mon[97873]: pgmap v754: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.1 MiB/s wr, 345 op/s 2023-07-21T22:40:08.570 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353069 28.28 MB/sec execute 524 sec latency 1387.630 ms 2023-07-21T22:40:09.570 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353251 28.24 MB/sec execute 525 sec latency 1554.935 ms 2023-07-21T22:40:09.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 748 op/s 2023-07-21T22:40:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:40:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:09.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:09 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 748 op/s 2023-07-21T22:40:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:40:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:09 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 16 MiB/s wr, 748 op/s 2023-07-21T22:40:10.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:40:10.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.137 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.138 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:09 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:10.570 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353431 28.21 MB/sec execute 526 sec latency 232.279 ms 2023-07-21T22:40:11.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:10 smithi194 ceph-mon[98288]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T22:40:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:10 smithi134 ceph-mon[97873]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T22:40:11.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:10 smithi093 ceph-mon[96579]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T22:40:11.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353606 28.17 MB/sec execute 527 sec latency 243.626 ms 2023-07-21T22:40:12.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:11 smithi194 ceph-mon[98288]: pgmap v757: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-21T22:40:12.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:11 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:11 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:11 smithi134 ceph-mon[97873]: pgmap v757: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-21T22:40:12.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:11 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:11 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:11 smithi093 ceph-mon[96579]: pgmap v757: 97 pgs: 97 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-21T22:40:12.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:11 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:11 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:12.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 353820 28.14 MB/sec execute 528 sec latency 210.465 ms 2023-07-21T22:40:13.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 354166 28.13 MB/sec execute 529 sec latency 147.164 ms 2023-07-21T22:40:14.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:13 smithi194 ceph-mon[98288]: pgmap v758: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 689 op/s 2023-07-21T22:40:14.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:13 smithi134 ceph-mon[97873]: pgmap v758: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 689 op/s 2023-07-21T22:40:14.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:13 smithi093 ceph-mon[96579]: pgmap v758: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 17 MiB/s wr, 689 op/s 2023-07-21T22:40:14.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 354641 28.12 MB/sec execute 530 sec latency 131.587 ms 2023-07-21T22:40:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:14 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:40:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:14 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:40:15.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:14 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:15.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:14 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:40:15.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:14 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:40:15.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:14 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:15.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:14 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:40:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:14 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:40:15.189 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:14 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:40:15.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 355274 28.14 MB/sec execute 531 sec latency 79.644 ms 2023-07-21T22:40:16.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:15 smithi194 ceph-mon[98288]: pgmap v759: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:15 smithi134 ceph-mon[97873]: pgmap v759: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:16.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:15 smithi093 ceph-mon[96579]: pgmap v759: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:16.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 355814 28.14 MB/sec execute 532 sec latency 109.984 ms 2023-07-21T22:40:17.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 356256 28.14 MB/sec execute 533 sec latency 98.275 ms 2023-07-21T22:40:18.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:17 smithi194 ceph-mon[98288]: pgmap v760: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:18.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:17 smithi134 ceph-mon[97873]: pgmap v760: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:18.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:17 smithi093 ceph-mon[96579]: pgmap v760: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 20 MiB/s wr, 874 op/s 2023-07-21T22:40:18.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 356741 28.12 MB/sec execute 534 sec latency 103.847 ms 2023-07-21T22:40:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 -- 172.21.15.93:0/1497560120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7540ff470 msgr2=0x7fa75410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 --2- 172.21.15.93:0/1497560120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7540ff470 0x7fa75410b900 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b0a0 tx=0x7fa74c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 -- 172.21.15.93:0/1497560120 shutdown_connections 2023-07-21T22:40:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 --2- 172.21.15.93:0/1497560120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7540ff470 0x7fa75410b900 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 --2- 172.21.15.93:0/1497560120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa754104ad0 0x7fa754104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 --2- 172.21.15.93:0/1497560120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540feac0 0x7fa7540fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.572+0000 7fa75a423700 1 -- 172.21.15.93:0/1497560120 >> 172.21.15.93:0/1497560120 conn(0x7fa754075bd0 msgr2=0x7fa754075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.575+0000 7fa75a423700 1 -- 172.21.15.93:0/1497560120 shutdown_connections 2023-07-21T22:40:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.575+0000 7fa75a423700 1 -- 172.21.15.93:0/1497560120 wait complete. 2023-07-21T22:40:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.575+0000 7fa75a423700 1 Processor -- start 2023-07-21T22:40:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 -- start start 2023-07-21T22:40:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7540feac0 0x7fa75410a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540ff470 0x7fa75410a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75410ae90 con 0x7fa754104ad0 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541b1210 con 0x7fa7540ff470 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa75a423700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541b1350 con 0x7fa7540feac0 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa753fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7540feac0 0x7fa75410a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50120/0 (socket says 172.21.15.93:50120) 2023-07-21T22:40:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 -- 172.21.15.93:0/3676957285 learned_addr learned my addr 172.21.15.93:0/3676957285 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7537fe700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540ff470 0x7fa75410a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 -- 172.21.15.93:0/3676957285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7540feac0 msgr2=0x7fa75410a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7540feac0 0x7fa75410a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 -- 172.21.15.93:0/3676957285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540ff470 msgr2=0x7fa75410a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540ff470 0x7fa75410a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.576+0000 7fa7589c0700 1 -- 172.21.15.93:0/3676957285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c009d00 con 0x7fa754104ad0 2023-07-21T22:40:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa7589c0700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006070 tx=0x7fa74c007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa7517fa700 1 -- 172.21.15.93:0/3676957285 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00e040 con 0x7fa754104ad0 2023-07-21T22:40:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7541b1520 con 0x7fa754104ad0 2023-07-21T22:40:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa7517fa700 1 -- 172.21.15.93:0/3676957285 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa74c003e10 con 0x7fa754104ad0 2023-07-21T22:40:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa7517fa700 1 -- 172.21.15.93:0/3676957285 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c01da90 con 0x7fa754104ad0 2023-07-21T22:40:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.577+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541b19a0 con 0x7fa754104ad0 2023-07-21T22:40:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.580+0000 7fa7517fa700 1 -- 172.21.15.93:0/3676957285 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa74c005730 con 0x7fa754104ad0 2023-07-21T22:40:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.581+0000 7fa7517fa700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 0x7fa73c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.581+0000 7fa7517fa700 1 -- 172.21.15.93:0/3676957285 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa74c0ce1d0 con 0x7fa754104ad0 2023-07-21T22:40:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 Processor -- start 2023-07-21T22:40:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 -- start start 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c4c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84b51c50 con 0x560a84c52210 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84ba49e0 con 0x560a84c52660 2023-07-21T22:40:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84b9de70 con 0x560a84aa86b0 2023-07-21T22:40:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa753fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa753fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37204/0 (socket says 172.21.15.93:37204) 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7537fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50126/0 (socket says 172.21.15.93:50126) 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa753fff700 1 -- 172.21.15.93:0/2329991060 learned_addr learned my addr 172.21.15.93:0/2329991060 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.582+0000 7fa7589c0700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c4c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 -- 172.21.15.93:0/2329991060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 msgr2=0x560a84c51cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 -- 172.21.15.93:0/2329991060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 msgr2=0x560a84c4c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c4c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 -- 172.21.15.93:0/2329991060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a849cbd80 con 0x560a84c52210 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa753fff700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7589c0700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c4c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa7537fe700 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa75410bc10 tx=0x7fa74800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f21fbce23ae3940 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa753fff700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 0x7fa73c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa73a7fc700 1 -- 172.21.15.93:0/2329991060 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748010c70 con 0x560a84c52210 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa73a7fc700 1 -- 172.21.15.93:0/2329991060 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa748015070 con 0x560a84c52210 2023-07-21T22:40:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.583+0000 7fa73a7fc700 1 -- 172.21.15.93:0/2329991060 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748005020 con 0x560a84c52210 2023-07-21T22:40:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.584+0000 7fa753fff700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 0x7fa73c0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa744009f00 tx=0x7fa744008040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.584+0000 7fa7665a8b80 1 -- 172.21.15.93:0/2329991060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 msgr2=0x560a84c4bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa75410bc10 tx=0x7fa74800bf50 comp rx=0 tx=0).stop 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 -- 172.21.15.93:0/2329991060 shutdown_connections 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c4c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84aa86b0 0x560a84c51cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/2329991060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c4bf40 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 -- 172.21.15.93:0/2329991060 >> 172.21.15.93:0/2329991060 conn(0x560a84c41ae0 msgr2=0x560a84c470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 -- 172.21.15.93:0/2329991060 shutdown_connections 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 -- 172.21.15.93:0/2329991060 wait complete. 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 -- start start 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.585+0000 7fa7665a8b80 1 Processor -- start 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c55dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84b3da60 con 0x560a84aa86b0 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84b3d760 con 0x560a84c52660 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84a218c0 con 0x560a84c52210 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7665a8b80 10 client.0 _ll_register_callbacks cb 0x560a84b3b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50130/0 (socket says 172.21.15.93:50130) 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 -- 172.21.15.93:0/3806437307 learned_addr learned my addr 172.21.15.93:0/3806437307 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa7589c0700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c55dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 -- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 msgr2=0x560a84c557c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c557c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 -- 172.21.15.93:0/3806437307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 msgr2=0x560a84c55dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c55dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.586+0000 7fa753fff700 1 -- 172.21.15.93:0/3806437307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7440097e0 con 0x560a84aa86b0 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa753fff700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa7440159d0 tx=0x7fa744004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bcc05606ff0ab1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7337fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744015d90 con 0x560a84aa86b0 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa7440147c0 con 0x560a84aa86b0 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a84af7490 con 0x560a84aa86b0 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa744005a00 con 0x560a84aa86b0 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7665a8b80 10 client.17979 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x7fa7541b19a0 con 0x560a84aa86b0 2023-07-21T22:40:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.587+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a84af7490 con 0x560a84aa86b0 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa744006380 con 0x560a84aa86b0 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 10 client.17979 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 10 client.17979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 20 client.17979 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84c65ab0 0x560a84c67ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 4 client.17979 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.588+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a84b96e50 con 0x560a84c65ab0 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.591+0000 7fa7537fe700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84c65ab0 0x560a84c67ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.592+0000 7fa7537fe700 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84c65ab0 0x560a84c67ec0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a06719151fcb218 server_cookie=1e8154c4ffc3b0e3 in_seq=0 out_seq=0 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.592+0000 7fa7337fe700 10 client.17979 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7337fe700 1 -- 172.21.15.93:0/3806437307 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a84b96e50 con 0x560a84c65ab0 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7337fe700 10 client.17979 handle_command_reply: tid=0 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 client.17979 shutdown 2023-07-21T22:40:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 msgr2=0x560a84c58540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa7440159d0 tx=0x7fa744004770 comp rx=0 tx=0).stop 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 shutdown_connections 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84c65ab0 0x560a84c67ec0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84c52660 0x560a84c55dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84aa86b0 0x560a84c58540 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/3806437307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 >> 172.21.15.93:0/3806437307 conn(0x560a84c41ae0 msgr2=0x560a84c463e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7337fe700 0 client.17979 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 shutdown_connections 2023-07-21T22:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 1 -- 172.21.15.93:0/3806437307 wait complete. 2023-07-21T22:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.593+0000 7fa7665a8b80 20 client.17979 trim_cache size 0 max 16384 2023-07-21T22:40:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 Processor -- start 2023-07-21T22:40:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 -- start start 2023-07-21T22:40:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52660 0x560a84c58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c4cd30 con 0x560a84c52210 2023-07-21T22:40:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c43c00 con 0x560a84aa86b0 2023-07-21T22:40:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.617+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c434f0 con 0x560a84c52660 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33066/0 (socket says 172.21.15.93:33066) 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 -- 172.21.15.93:0/648407162 learned_addr learned my addr 172.21.15.93:0/648407162 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa7537fe700 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 -- 172.21.15.93:0/648407162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52660 msgr2=0x560a84c58070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52660 0x560a84c58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 -- 172.21.15.93:0/648407162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 msgr2=0x560a84c53140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c53140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 -- 172.21.15.93:0/648407162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74800b960 con 0x560a84aa86b0 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa753fff700 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa744006e70 tx=0x7fa744015f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd12b8273ba947a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa73affd700 1 -- 172.21.15.93:0/648407162 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744006950 con 0x560a84aa86b0 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa73affd700 1 -- 172.21.15.93:0/648407162 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa744014b50 con 0x560a84aa86b0 2023-07-21T22:40:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.618+0000 7fa73affd700 1 -- 172.21.15.93:0/648407162 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744035b60 con 0x560a84aa86b0 2023-07-21T22:40:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.619+0000 7fa7665a8b80 1 -- 172.21.15.93:0/648407162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 msgr2=0x560a84c5cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.619+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa744006e70 tx=0x7fa744015f10 comp rx=0 tx=0).stop 2023-07-21T22:40:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 -- 172.21.15.93:0/648407162 shutdown_connections 2023-07-21T22:40:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52660 0x560a84c58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c5cb40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/648407162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52210 0x560a84c53140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 -- 172.21.15.93:0/648407162 >> 172.21.15.93:0/648407162 conn(0x560a84c41ae0 msgr2=0x560a84c52a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.620+0000 7fa7665a8b80 1 -- 172.21.15.93:0/648407162 shutdown_connections 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 -- 172.21.15.93:0/648407162 wait complete. 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 -- start start 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 Processor -- start 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c4eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c4dcb0 con 0x560a84c52660 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c4ddf0 con 0x560a84aa86b0 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a84c4dfb0 con 0x560a84c52210 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.621+0000 7fa7665a8b80 10 client.0 _ll_register_callbacks cb 0x7fa744012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa753fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c4eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50144/0 (socket says 172.21.15.93:50144) 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 -- 172.21.15.93:0/4247942989 learned_addr learned my addr 172.21.15.93:0/4247942989 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7537fe700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 -- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 msgr2=0x560a84c4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c4f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 -- 172.21.15.93:0/4247942989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 msgr2=0x560a84c4eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c4eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 -- 172.21.15.93:0/4247942989 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541b19a0 con 0x560a84c52660 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa753fff700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7589c0700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa74c09b2c0 tx=0x7fa74c009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8828f34c508a0d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7337fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0cbd10 con 0x560a84c52660 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.622+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa74c09f360 con 0x560a84c52660 2023-07-21T22:40:18.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa748015070 con 0x560a84c52660 2023-07-21T22:40:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa74c0b2020 con 0x560a84c52660 2023-07-21T22:40:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 10 client.17985 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x560a849b4c00 con 0x560a84c52660 2023-07-21T22:40:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa748015070 con 0x560a84c52660 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa74c09a820 con 0x560a84c52660 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 10 client.17985 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 10 client.17985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 20 client.17985 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84cba620 0x560a84cbca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 4 client.17985 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.623+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a84b58ff0 con 0x560a84cba620 2023-07-21T22:40:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.624+0000 7fa753fff700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84cba620 0x560a84cbca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.625+0000 7fa7337fe700 10 client.17985 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.625+0000 7fa753fff700 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84cba620 0x560a84cbca00 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9beddadd49c30804 server_cookie=7154490ee3c187d4 in_seq=0 out_seq=0 2023-07-21T22:40:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7337fe700 1 -- 172.21.15.93:0/4247942989 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a84b58ff0 con 0x560a84cba620 2023-07-21T22:40:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7337fe700 10 client.17985 handle_command_reply: tid=0 2023-07-21T22:40:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 client.17985 shutdown 2023-07-21T22:40:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 msgr2=0x560a84c4d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa74c09b2c0 tx=0x7fa74c009a30 comp rx=0 tx=0).stop 2023-07-21T22:40:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 shutdown_connections 2023-07-21T22:40:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x560a84cba620 0x560a84cbca00 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x560a84c52660 0x560a84c4d730 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a84aa86b0 0x560a84c4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 --2- 172.21.15.93:0/4247942989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560a84c52210 0x560a84c4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7337fe700 0 client.17985 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 >> 172.21.15.93:0/4247942989 conn(0x560a84c41ae0 msgr2=0x560a84c532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 shutdown_connections 2023-07-21T22:40:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 1 -- 172.21.15.93:0/4247942989 wait complete. 2023-07-21T22:40:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.925+0000 7fa7665a8b80 20 client.17985 trim_cache size 0 max 16384 2023-07-21T22:40:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:40:18.990 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:40:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:40:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:40:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 msgr2=0x7fa73c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 0x7fa73c0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa744009f00 tx=0x7fa744008040 comp rx=0 tx=0).stop 2023-07-21T22:40:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 msgr2=0x7fa7541b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa74c006070 tx=0x7fa74c007b60 comp rx=0 tx=0).stop 2023-07-21T22:40:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 shutdown_connections 2023-07-21T22:40:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa73c0a12c0 0x7fa73c0a3770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa754104ad0 0x7fa7541b0cd0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7540feac0 0x7fa75410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 --2- 172.21.15.93:0/3676957285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7540ff470 0x7fa75410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.926+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 >> 172.21.15.93:0/3676957285 conn(0x7fa754075bd0 msgr2=0x7fa7540fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.927+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 shutdown_connections 2023-07-21T22:40:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:18.927+0000 7fa75a423700 1 -- 172.21.15.93:0/3676957285 wait complete. 2023-07-21T22:40:19.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 357162 28.12 MB/sec execute 535 sec latency 215.254 ms 2023-07-21T22:40:20.089 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:19 smithi194 ceph-mon[98288]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 608 op/s 2023-07-21T22:40:20.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:19 smithi134 ceph-mon[97873]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 608 op/s 2023-07-21T22:40:20.190 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:19 smithi093 ceph-mon[96579]: pgmap v761: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 608 op/s 2023-07-21T22:40:20.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 357690 28.10 MB/sec execute 536 sec latency 93.675 ms 2023-07-21T22:40:21.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 358311 28.11 MB/sec execute 537 sec latency 72.985 ms 2023-07-21T22:40:22.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:21 smithi194 ceph-mon[98288]: pgmap v762: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-21T22:40:22.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:21 smithi134 ceph-mon[97873]: pgmap v762: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-21T22:40:22.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:21 smithi093 ceph-mon[96579]: pgmap v762: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-21T22:40:22.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 358888 28.13 MB/sec execute 538 sec latency 151.556 ms 2023-07-21T22:40:23.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 359439 28.14 MB/sec execute 539 sec latency 191.609 ms 2023-07-21T22:40:23.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:23 smithi194 ceph-mon[98288]: pgmap v763: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-07-21T22:40:23.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:23 smithi194 ceph-mon[98288]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T22:40:23.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:23 smithi134 ceph-mon[97873]: pgmap v763: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-07-21T22:40:23.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:23 smithi134 ceph-mon[97873]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T22:40:23.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:23 smithi093 ceph-mon[96579]: pgmap v763: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 514 op/s 2023-07-21T22:40:23.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:23 smithi093 ceph-mon[96579]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T22:40:24.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 359802 28.13 MB/sec execute 540 sec latency 101.936 ms 2023-07-21T22:40:25.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 360220 28.13 MB/sec execute 541 sec latency 111.154 ms 2023-07-21T22:40:25.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:25 smithi194 ceph-mon[98288]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:25.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:25 smithi134 ceph-mon[97873]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:25.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:25 smithi093 ceph-mon[96579]: pgmap v765: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:26.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 360628 28.09 MB/sec execute 542 sec latency 93.395 ms 2023-07-21T22:40:27.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 361189 28.10 MB/sec execute 543 sec latency 112.987 ms 2023-07-21T22:40:27.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:27 smithi194 ceph-mon[98288]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:27.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:27 smithi134 ceph-mon[97873]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:27.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:27 smithi093 ceph-mon[96579]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 515 op/s 2023-07-21T22:40:28.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 361775 28.10 MB/sec execute 544 sec latency 149.660 ms 2023-07-21T22:40:29.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 362393 28.10 MB/sec execute 545 sec latency 100.157 ms 2023-07-21T22:40:29.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:29 smithi194 ceph-mon[98288]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 494 op/s 2023-07-21T22:40:29.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:29 smithi134 ceph-mon[97873]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 494 op/s 2023-07-21T22:40:29.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:29 smithi093 ceph-mon[96579]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 494 op/s 2023-07-21T22:40:30.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 363051 28.13 MB/sec execute 546 sec latency 62.138 ms 2023-07-21T22:40:31.571 INFO:tasks.workunit.client.0.smithi093.stdout: 10 363527 28.15 MB/sec execute 547 sec latency 89.961 ms 2023-07-21T22:40:31.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:31 smithi194 ceph-mon[98288]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 492 op/s 2023-07-21T22:40:31.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:31 smithi134 ceph-mon[97873]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 492 op/s 2023-07-21T22:40:31.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:31 smithi093 ceph-mon[96579]: pgmap v768: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 492 op/s 2023-07-21T22:40:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 -- 172.21.15.93:0/751662019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2410103430 msgr2=0x7f241010b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 --2- 172.21.15.93:0/751662019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2410103430 0x7f241010b000 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f240800c240 tx=0x7f240800c550 comp rx=0 tx=0).stop 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 -- 172.21.15.93:0/751662019 shutdown_connections 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 --2- 172.21.15.93:0/751662019 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2410103430 0x7f241010b000 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 --2- 172.21.15.93:0/751662019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24100fe3f0 0x7f24100fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 --2- 172.21.15.93:0/751662019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 -- 172.21.15.93:0/751662019 >> 172.21.15.93:0/751662019 conn(0x7f2410075f90 msgr2=0x7f2410076390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 -- 172.21.15.93:0/751662019 shutdown_connections 2023-07-21T22:40:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.065+0000 7f2414cc8700 1 -- 172.21.15.93:0/751662019 wait complete. 2023-07-21T22:40:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 Processor -- start 2023-07-21T22:40:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 -- start start 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2410103430 0x7f24101938b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410193f20 con 0x7f24100fe3f0 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410194090 con 0x7f24100fed90 2023-07-21T22:40:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f2414cc8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410194250 con 0x7f2410103430 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50154/0 (socket says 172.21.15.93:50154) 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f240e59c700 1 -- 172.21.15.93:0/1105164363 learned_addr learned my addr 172.21.15.93:0/1105164363 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f240dd9b700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.066+0000 7f240ed9d700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2410103430 0x7f24101938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 -- 172.21.15.93:0/1105164363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2410103430 msgr2=0x7f24101938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2410103430 0x7f24101938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 -- 172.21.15.93:0/1105164363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 msgr2=0x7f2410199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 -- 172.21.15.93:0/1105164363 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240800ae10 con 0x7f24100fe3f0 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240dd9b700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f240e59c700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f240000da00 tx=0x7f240000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f23ff7fe700 1 -- 172.21.15.93:0/1105164363 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000eba0 con 0x7f24100fe3f0 2023-07-21T22:40:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f23ff7fe700 1 -- 172.21.15.93:0/1105164363 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240000ed00 con 0x7f24100fe3f0 2023-07-21T22:40:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2410194530 con 0x7f24100fe3f0 2023-07-21T22:40:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f23ff7fe700 1 -- 172.21.15.93:0/1105164363 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400014690 con 0x7f24100fe3f0 2023-07-21T22:40:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.067+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241006a110 con 0x7f24100fe3f0 2023-07-21T22:40:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.070+0000 7f23ff7fe700 1 -- 172.21.15.93:0/1105164363 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2400016440 con 0x7f24100fe3f0 2023-07-21T22:40:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.071+0000 7f23ff7fe700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 0x7f23f80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.071+0000 7f23ff7fe700 1 -- 172.21.15.93:0/1105164363 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f24000ce3f0 con 0x7f24100fe3f0 2023-07-21T22:40:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f240dd9b700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 0x7f23f80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f240dd9b700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 0x7f23f80a37b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24101048f0 tx=0x7f240400b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 Processor -- start 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 -- start start 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b85f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c660 0x563a26b864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26a8bc50 con 0x563a269e26b0 2023-07-21T22:40:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26ade9e0 con 0x563a26b8c210 2023-07-21T22:40:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.072+0000 7f2420e4db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26ad7e70 con 0x563a26b8c660 2023-07-21T22:40:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b85f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b85f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33094/0 (socket says 172.21.15.93:33094) 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240dd9b700 1 -- 172.21.15.93:0/4233163565 learned_addr learned my addr 172.21.15.93:0/4233163565 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50162/0 (socket says 172.21.15.93:50162) 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240ed9d700 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c660 0x563a26b864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 -- 172.21.15.93:0/4233163565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c660 msgr2=0x563a26b864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c660 0x563a26b864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 -- 172.21.15.93:0/4233163565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 msgr2=0x563a26b85f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b85f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 -- 172.21.15.93:0/4233163565 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a26905d80 con 0x563a269e26b0 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.073+0000 7f240e59c700 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f24000cbc50 tx=0x7f240009b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7f0e65d51bd033 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.074+0000 7f23fcff9700 1 -- 172.21.15.93:0/4233163565 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240009f600 con 0x563a269e26b0 2023-07-21T22:40:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.074+0000 7f23fcff9700 1 -- 172.21.15.93:0/4233163565 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24000a3a20 con 0x563a269e26b0 2023-07-21T22:40:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.074+0000 7f23fcff9700 1 -- 172.21.15.93:0/4233163565 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240009ba90 con 0x563a269e26b0 2023-07-21T22:40:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- 172.21.15.93:0/4233163565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 msgr2=0x563a26b8bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f24000cbc50 tx=0x7f240009b110 comp rx=0 tx=0).stop 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- 172.21.15.93:0/4233163565 shutdown_connections 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c660 0x563a26b864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b8bcd0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4233163565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b85f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- 172.21.15.93:0/4233163565 >> 172.21.15.93:0/4233163565 conn(0x563a26b7bae0 msgr2=0x563a26b810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- 172.21.15.93:0/4233163565 shutdown_connections 2023-07-21T22:40:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- 172.21.15.93:0/4233163565 wait complete. 2023-07-21T22:40:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.075+0000 7f2420e4db80 1 -- start start 2023-07-21T22:40:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 Processor -- start 2023-07-21T22:40:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c660 0x563a26b8ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26a77a60 con 0x563a269e26b0 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26a77760 con 0x563a26b8c660 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a2695b8c0 con 0x563a26b8c210 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f2420e4db80 10 client.0 _ll_register_callbacks cb 0x563a26a75700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50170/0 (socket says 172.21.15.93:50170) 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.076+0000 7f240e59c700 1 -- 172.21.15.93:0/3035852959 learned_addr learned my addr 172.21.15.93:0/3035852959 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240ed9d700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c660 0x563a26b8ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 -- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 msgr2=0x563a26b8f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 -- 172.21.15.93:0/3035852959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c660 msgr2=0x563a26b8ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c660 0x563a26b8ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 -- 172.21.15.93:0/3035852959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241006a110 con 0x563a269e26b0 2023-07-21T22:40:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f240e59c700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f24000cbd00 tx=0x7f24000bb170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28548f87ede4c1fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f23f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400099a30 con 0x563a269e26b0 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f240009b830 con 0x563a269e26b0 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a26a31490 con 0x563a269e26b0 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.077+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f240009b9f0 con 0x563a269e26b0 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 10 client.17997 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x563a268eec00 con 0x563a269e26b0 2023-07-21T22:40:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a26a31490 con 0x563a269e26b0 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f240009a530 con 0x563a269e26b0 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 10 client.17997 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 10 client.17997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 20 client.17997 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b9f7e0 0x563a26ba1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 4 client.17997 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:40:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a26ad0e50 con 0x563a26b9f7e0 2023-07-21T22:40:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.079+0000 7f240dd9b700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b9f7e0 0x563a26ba1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.080+0000 7f23f5ffb700 10 client.17997 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.080+0000 7f240dd9b700 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b9f7e0 0x563a26ba1bc0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf14fecf9a812e8 server_cookie=12ae082c747d772d in_seq=0 out_seq=0 2023-07-21T22:40:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f23f5ffb700 1 -- 172.21.15.93:0/3035852959 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a26ad0e50 con 0x563a26b9f7e0 2023-07-21T22:40:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f23f5ffb700 10 client.17997 handle_command_reply: tid=0 2023-07-21T22:40:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 client.17997 shutdown 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 msgr2=0x563a26b925c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f24000cbd00 tx=0x7f24000bb170 comp rx=0 tx=0).stop 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 shutdown_connections 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b9f7e0 0x563a26ba1bc0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c660 0x563a26b8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26b925c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 --2- 172.21.15.93:0/3035852959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 >> 172.21.15.93:0/3035852959 conn(0x563a26b7bae0 msgr2=0x563a26b803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f23f5ffb700 0 client.17997 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 shutdown_connections 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.083+0000 7f2420e4db80 1 -- 172.21.15.93:0/3035852959 wait complete. 2023-07-21T22:40:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.084+0000 7f2420e4db80 20 client.17997 trim_cache size 0 max 16384 2023-07-21T22:40:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 Processor -- start 2023-07-21T22:40:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 -- start start 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a26b7bae0 0x563a26b96b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b907f0 con 0x563a26b7bae0 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b87160 con 0x563a269e26b0 2023-07-21T22:40:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f2420e4db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b90e20 con 0x563a26b8c210 2023-07-21T22:40:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f240dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f240dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33108/0 (socket says 172.21.15.93:33108) 2023-07-21T22:40:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.105+0000 7f240dd9b700 1 -- 172.21.15.93:0/4102298194 learned_addr learned my addr 172.21.15.93:0/4102298194 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 -- 172.21.15.93:0/4102298194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 msgr2=0x563a26b8b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 -- 172.21.15.93:0/4102298194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a26b7bae0 msgr2=0x563a26b96b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a26b7bae0 0x563a26b96b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 -- 172.21.15.93:0/4102298194 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a26ba67f0 con 0x563a269e26b0 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f240dd9b700 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x563a26b90df0 tx=0x7f2404013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc024f86a7603221 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f23fd7fa700 1 -- 172.21.15.93:0/4102298194 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404025070 con 0x563a269e26b0 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f23fd7fa700 1 -- 172.21.15.93:0/4102298194 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2404015070 con 0x563a269e26b0 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.106+0000 7f23fd7fa700 1 -- 172.21.15.93:0/4102298194 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404026ac0 con 0x563a269e26b0 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- 172.21.15.93:0/4102298194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 msgr2=0x563a26b8f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x563a26b90df0 tx=0x7f2404013000 comp rx=0 tx=0).stop 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- 172.21.15.93:0/4102298194 shutdown_connections 2023-07-21T22:40:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b8c210 0x563a26b8b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a26b7bae0 0x563a26b96b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 --2- 172.21.15.93:0/4102298194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a269e26b0 0x563a26b8f050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- 172.21.15.93:0/4102298194 >> 172.21.15.93:0/4102298194 conn(0x563a26ba1b50 msgr2=0x563a26b8a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- 172.21.15.93:0/4102298194 shutdown_connections 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- 172.21.15.93:0/4102298194 wait complete. 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 -- start start 2023-07-21T22:40:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.107+0000 7f2420e4db80 1 Processor -- start 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b7bae0 0x563a26bebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b966d0 con 0x563a269e26b0 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b96840 con 0x563a26b8c210 2023-07-21T22:40:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a26b96a00 con 0x563a26b7bae0 2023-07-21T22:40:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f2420e4db80 10 client.0 _ll_register_callbacks cb 0x7f240800c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50182/0 (socket says 172.21.15.93:50182) 2023-07-21T22:40:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 -- 172.21.15.93:0/656731381 learned_addr learned my addr 172.21.15.93:0/656731381 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 -- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b7bae0 msgr2=0x563a26bebd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:40:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b7bae0 0x563a26bebd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 -- 172.21.15.93:0/656731381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 msgr2=0x563a26b96190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:40:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b96190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.156 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.108+0000 7f240e59c700 1 -- 172.21.15.93:0/656731381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a2692c1f0 con 0x563a269e26b0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.109+0000 7f240e59c700 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f240009b2d0 tx=0x7f240009b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a655754ecfbdcbff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.111+0000 7f23f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.111+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400099a30 con 0x563a269e26b0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.111+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f24000c9030 con 0x563a269e26b0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.111+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24000a3a20 con 0x563a269e26b0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f240009f330 con 0x563a269e26b0 2023-07-21T22:40:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 10 client.18000 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x563a26904250 con 0x563a269e26b0 2023-07-21T22:40:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24000a3a20 con 0x563a269e26b0 2023-07-21T22:40:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2400080070 con 0x563a269e26b0 2023-07-21T22:40:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 10 client.18000 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 20 client.18000 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b8c660 0x563a26bf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 4 client.18000 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:40:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.112+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a26a92ff0 con 0x563a26b8c660 2023-07-21T22:40:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.113+0000 7f240dd9b700 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b8c660 0x563a26bf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.114+0000 7f23f5ffb700 10 client.18000 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.114+0000 7f240dd9b700 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b8c660 0x563a26bf61c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824621af8b84dc67 server_cookie=c94b7e49d1757cb6 in_seq=0 out_seq=0 2023-07-21T22:40:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.453+0000 7f23f5ffb700 1 -- 172.21.15.93:0/656731381 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a26a92ff0 con 0x563a26b8c660 2023-07-21T22:40:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.453+0000 7f23f5ffb700 10 client.18000 handle_command_reply: tid=0 2023-07-21T22:40:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.453+0000 7f2420e4db80 1 client.18000 shutdown 2023-07-21T22:40:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 msgr2=0x563a26beb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f240009b2d0 tx=0x7f240009b470 comp rx=0 tx=0).stop 2023-07-21T22:40:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 shutdown_connections 2023-07-21T22:40:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x563a26b8c660 0x563a26bf61c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a26b8c210 0x563a26b96190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f23f5ffb700 0 client.18000 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563a269e26b0 0x563a26beb810 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 --2- 172.21.15.93:0/656731381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a26b7bae0 0x563a26bebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 >> 172.21.15.93:0/656731381 conn(0x563a26ba1b50 msgr2=0x563a26b8b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 shutdown_connections 2023-07-21T22:40:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 1 -- 172.21.15.93:0/656731381 wait complete. 2023-07-21T22:40:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2420e4db80 20 client.18000 trim_cache size 0 max 16384 2023-07-21T22:40:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:40:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:40:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:40:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:40:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 msgr2=0x7f23f80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 0x7f23f80a37b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24101048f0 tx=0x7f240400b410 comp rx=0 tx=0).stop 2023-07-21T22:40:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 msgr2=0x7f2410199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f240000da00 tx=0x7f240000dd10 comp rx=0 tx=0).stop 2023-07-21T22:40:32.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 shutdown_connections 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2410103430 0x7f24101938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f23f80a1300 0x7f23f80a37b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.454+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24100fe3f0 0x7f2410199a50 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.455+0000 7f2414cc8700 1 --2- 172.21.15.93:0/1105164363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24100fed90 0x7f2410199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.455+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 >> 172.21.15.93:0/1105164363 conn(0x7f2410075f90 msgr2=0x7f24100fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.455+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 shutdown_connections 2023-07-21T22:40:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:32.455+0000 7f2414cc8700 1 -- 172.21.15.93:0/1105164363 wait complete. 2023-07-21T22:40:32.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 363760 28.11 MB/sec execute 548 sec latency 302.097 ms 2023-07-21T22:40:33.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 364096 28.08 MB/sec execute 549 sec latency 134.480 ms 2023-07-21T22:40:33.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:33 smithi194 ceph-mon[98288]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 495 op/s 2023-07-21T22:40:33.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:33 smithi134 ceph-mon[97873]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 495 op/s 2023-07-21T22:40:33.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:33 smithi093 ceph-mon[96579]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 495 op/s 2023-07-21T22:40:34.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 364654 28.08 MB/sec execute 550 sec latency 82.982 ms 2023-07-21T22:40:35.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 365238 28.08 MB/sec execute 551 sec latency 109.478 ms 2023-07-21T22:40:35.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:35 smithi194 ceph-mon[98288]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 613 op/s 2023-07-21T22:40:35.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:35 smithi134 ceph-mon[97873]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 613 op/s 2023-07-21T22:40:35.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:35 smithi093 ceph-mon[96579]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 613 op/s 2023-07-21T22:40:36.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 365920 28.09 MB/sec execute 552 sec latency 82.512 ms 2023-07-21T22:40:37.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 366420 28.10 MB/sec execute 553 sec latency 98.506 ms 2023-07-21T22:40:37.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:37 smithi194 ceph-mon[98288]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2023-07-21T22:40:37.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:37 smithi134 ceph-mon[97873]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2023-07-21T22:40:37.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:37 smithi093 ceph-mon[96579]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2023-07-21T22:40:38.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 366992 28.12 MB/sec execute 554 sec latency 78.361 ms 2023-07-21T22:40:39.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 367405 28.10 MB/sec execute 555 sec latency 103.162 ms 2023-07-21T22:40:39.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:39 smithi194 ceph-mon[98288]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 594 op/s 2023-07-21T22:40:39.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:39 smithi134 ceph-mon[97873]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 594 op/s 2023-07-21T22:40:39.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:39 smithi093 ceph-mon[96579]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 594 op/s 2023-07-21T22:40:40.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 367890 28.10 MB/sec execute 556 sec latency 91.616 ms 2023-07-21T22:40:41.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 368408 28.09 MB/sec execute 557 sec latency 88.460 ms 2023-07-21T22:40:41.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:41 smithi194 ceph-mon[98288]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2023-07-21T22:40:41.878 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:41 smithi134 ceph-mon[97873]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2023-07-21T22:40:41.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:41 smithi093 ceph-mon[96579]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.3 MiB/s wr, 431 op/s 2023-07-21T22:40:42.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 368981 28.10 MB/sec execute 558 sec latency 89.080 ms 2023-07-21T22:40:43.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 369141 28.07 MB/sec execute 559 sec latency 722.276 ms 2023-07-21T22:40:43.840 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:43 smithi194 ceph-mon[98288]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 427 op/s 2023-07-21T22:40:43.879 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:43 smithi134 ceph-mon[97873]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 427 op/s 2023-07-21T22:40:43.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:43 smithi093 ceph-mon[96579]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 427 op/s 2023-07-21T22:40:44.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 369166 28.02 MB/sec execute 560 sec latency 1722.315 ms 2023-07-21T22:40:45.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 369375 27.98 MB/sec execute 561 sec latency 1801.831 ms 2023-07-21T22:40:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.639+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2418946454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 msgr2=0x7fb7a4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.639+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2418946454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a4104ea0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009990 tx=0x7fb78c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2418946454 shutdown_connections 2023-07-21T22:40:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2418946454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7a40ff470 0x7fb7a410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2418946454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a4104ea0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2418946454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7a40feac0 0x7fb7a40fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2418946454 >> 172.21.15.93:0/2418946454 conn(0x7fb7a4075bd0 msgr2=0x7fb7a4075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2418946454 shutdown_connections 2023-07-21T22:40:45.682 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.640+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2418946454 wait complete. 2023-07-21T22:40:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 Processor -- start 2023-07-21T22:40:45.683 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 -- start start 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7a40feac0 0x7fb7a41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7a40ff470 0x7fb7a4198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a419d080 con 0x7fb7a4104ad0 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a419d1f0 con 0x7fb7a40feac0 2023-07-21T22:40:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a8ebb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a419d3b0 con 0x7fb7a40ff470 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.641+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50196/0 (socket says 172.21.15.93:50196) 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2606124287 learned_addr learned my addr 172.21.15.93:0/2606124287 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2606124287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7a40ff470 msgr2=0x7fb7a4198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7a40ff470 0x7fb7a4198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2606124287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7a40feac0 msgr2=0x7fb7a41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7a40feac0 0x7fb7a41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2606124287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0095f0 con 0x7fb7a4104ad0 2023-07-21T22:40:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b780 tx=0x7fb79c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb79bfff700 1 -- 172.21.15.93:0/2606124287 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00f840 con 0x7fb7a4104ad0 2023-07-21T22:40:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb79bfff700 1 -- 172.21.15.93:0/2606124287 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c00fe60 con 0x7fb7a4104ad0 2023-07-21T22:40:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.642+0000 7fb79bfff700 1 -- 172.21.15.93:0/2606124287 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00d3d0 con 0x7fb7a4104ad0 2023-07-21T22:40:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.646+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a419d690 con 0x7fb7a4104ad0 2023-07-21T22:40:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.646+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a406a110 con 0x7fb7a4104ad0 2023-07-21T22:40:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.649+0000 7fb79bfff700 1 -- 172.21.15.93:0/2606124287 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb79c017690 con 0x7fb7a4104ad0 2023-07-21T22:40:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.650+0000 7fb79bfff700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 0x7fb7900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.650+0000 7fb79bfff700 1 -- 172.21.15.93:0/2606124287 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb79c0cd000 con 0x7fb7a4104ad0 2023-07-21T22:40:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.650+0000 7fb7b5040b80 1 Processor -- start 2023-07-21T22:40:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.650+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 0x7fb7900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.650+0000 7fb7b5040b80 1 -- start start 2023-07-21T22:40:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e90acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90b210 0x55843e904f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e80ac50 con 0x55843e90b660 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e85d9e0 con 0x55843e90b210 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e856e70 con 0x55843e7616b0 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50204/0 (socket says 172.21.15.93:50204) 2023-07-21T22:40:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/1856482267 learned_addr learned my addr 172.21.15.93:0/1856482267 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7a259c700 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e90acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.651+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 0x7fb7900a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098e0 tx=0x7fb78c005fb0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.652+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/1856482267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 msgr2=0x55843e90acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.652+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e90acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.652+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/1856482267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90b210 msgr2=0x55843e904f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.652+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90b210 0x55843e904f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.652+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/1856482267 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55843e684d80 con 0x55843e90b660 2023-07-21T22:40:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.653+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0a1f90 tx=0x7fb79c09a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6290a1e080a659be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.654+0000 7fb798d99700 1 -- 172.21.15.93:0/1856482267 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c098410 con 0x55843e90b660 2023-07-21T22:40:45.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.654+0000 7fb798d99700 1 -- 172.21.15.93:0/1856482267 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c0ca970 con 0x55843e90b660 2023-07-21T22:40:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.654+0000 7fb798d99700 1 -- 172.21.15.93:0/1856482267 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c099560 con 0x55843e90b660 2023-07-21T22:40:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 -- 172.21.15.93:0/1856482267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 msgr2=0x55843e9054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0a1f90 tx=0x7fb79c09a140 comp rx=0 tx=0).stop 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 -- 172.21.15.93:0/1856482267 shutdown_connections 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e9054a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e90acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/1856482267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90b210 0x55843e904f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.655+0000 7fb7b5040b80 1 -- 172.21.15.93:0/1856482267 >> 172.21.15.93:0/1856482267 conn(0x55843e8faae0 msgr2=0x55843e9000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.656+0000 7fb7b5040b80 1 -- 172.21.15.93:0/1856482267 shutdown_connections 2023-07-21T22:40:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.656+0000 7fb7b5040b80 1 -- 172.21.15.93:0/1856482267 wait complete. 2023-07-21T22:40:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 -- start start 2023-07-21T22:40:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 Processor -- start 2023-07-21T22:40:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e9115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90b660 0x55843e90e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e7f6a60 con 0x55843e90eef0 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e7f6760 con 0x55843e7616b0 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e6da8c0 con 0x55843e90b660 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.657+0000 7fb7b5040b80 10 client.0 _ll_register_callbacks cb 0x55843e7f4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50212/0 (socket says 172.21.15.93:50212) 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2516466163 learned_addr learned my addr 172.21.15.93:0/2516466163 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e9115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90b660 msgr2=0x55843e90e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90b660 0x55843e90e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2516466163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 msgr2=0x55843e9115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e9115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 -- 172.21.15.93:0/2516466163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a406a110 con 0x55843e90eef0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009fd0 tx=0x7fb79c0ca690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc0b1fac8ca598f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb789ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0c8070 con 0x55843e90eef0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb79c0cae40 con 0x55843e90eef0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.658+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55843e7b0490 con 0x55843e90eef0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79c0ca2b0 con 0x55843e90eef0 2023-07-21T22:40:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 10 client.18012 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55843e66dc00 con 0x55843e90eef0 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55843e7b0490 con 0x55843e90eef0 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb79c0b8290 con 0x55843e90eef0 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 10 client.18012 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 20 client.18012 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e91ea90 0x55843e920ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 4 client.18012 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:40:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.659+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55843e84fe50 con 0x55843e91ea90 2023-07-21T22:40:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.665+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e91ea90 0x55843e920ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.666+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e91ea90 0x55843e920ea0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a07a6fff0aca8f2 server_cookie=ec11c70055fc8bee in_seq=0 out_seq=0 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb789ffb700 10 client.18012 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb789ffb700 1 -- 172.21.15.93:0/2516466163 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55843e84fe50 con 0x55843e91ea90 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb789ffb700 10 client.18012 handle_command_reply: tid=0 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 client.18012 shutdown 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 msgr2=0x55843e91acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009fd0 tx=0x7fb79c0ca690 comp rx=0 tx=0).stop 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 shutdown_connections 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e91ea90 0x55843e920ea0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90eef0 0x55843e91acf0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e9115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2516466163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90b660 0x55843e90e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 >> 172.21.15.93:0/2516466163 conn(0x55843e8faae0 msgr2=0x55843e8ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb789ffb700 0 client.18012 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 shutdown_connections 2023-07-21T22:40:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2516466163 wait complete. 2023-07-21T22:40:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.669+0000 7fb7b5040b80 20 client.18012 trim_cache size 0 max 16384 2023-07-21T22:40:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.689+0000 7fb7b5040b80 1 Processor -- start 2023-07-21T22:40:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.689+0000 7fb7b5040b80 1 -- start start 2023-07-21T22:40:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e90a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90eef0 0x55843e904f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e905f80 con 0x55843e90b660 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e84fe50 con 0x55843e7616b0 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e7b0490 con 0x55843e90eef0 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb799d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e90a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33146/0 (socket says 172.21.15.93:33146) 2023-07-21T22:40:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb799d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e90a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50218/0 (socket says 172.21.15.93:50218) 2023-07-21T22:40:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 -- 172.21.15.93:0/2857142777 learned_addr learned my addr 172.21.15.93:0/2857142777 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90eef0 0x55843e904f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 -- 172.21.15.93:0/2857142777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90eef0 msgr2=0x55843e904f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90eef0 0x55843e904f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 -- 172.21.15.93:0/2857142777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 msgr2=0x55843e90a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e90a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.690+0000 7fb7a259c700 1 -- 172.21.15.93:0/2857142777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55843e6ab1f0 con 0x55843e7616b0 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.691+0000 7fb7a259c700 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40ffc00 tx=0x7fb78c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff4ac5150e9e344 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.691+0000 7fb79959a700 1 -- 172.21.15.93:0/2857142777 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c02b040 con 0x55843e7616b0 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.691+0000 7fb79959a700 1 -- 172.21.15.93:0/2857142777 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78c030e10 con 0x55843e7616b0 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.691+0000 7fb79959a700 1 -- 172.21.15.93:0/2857142777 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01d070 con 0x55843e7616b0 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2857142777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 msgr2=0x55843e900290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40ffc00 tx=0x7fb78c020000 comp rx=0 tx=0).stop 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2857142777 shutdown_connections 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e90eef0 0x55843e904f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e7616b0 0x55843e900290 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/2857142777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e90a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2857142777 >> 172.21.15.93:0/2857142777 conn(0x55843e8faae0 msgr2=0x55843e8feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2857142777 shutdown_connections 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- 172.21.15.93:0/2857142777 wait complete. 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 -- start start 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.692+0000 7fb7b5040b80 1 Processor -- start 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e913c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90eef0 0x55843e9146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e971950 con 0x55843e90b660 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e971a90 con 0x55843e90eef0 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55843e971bd0 con 0x55843e7616b0 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7b5040b80 10 client.0 _ll_register_callbacks cb 0x7fb78c005c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50224/0 (socket says 172.21.15.93:50224) 2023-07-21T22:40:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 -- 172.21.15.93:0/607866118 learned_addr learned my addr 172.21.15.93:0/607866118 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7a2d9d700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90eef0 0x55843e9146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb7a259c700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e913c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 -- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 msgr2=0x55843e913c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e913c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 -- 172.21.15.93:0/607866118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90eef0 msgr2=0x55843e9146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90eef0 0x55843e9146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.693+0000 7fb799d9b700 1 -- 172.21.15.93:0/607866118 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a4068830 con 0x55843e90b660 2023-07-21T22:40:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.694+0000 7fb799d9b700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55843e901b80 tx=0x7fb794011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c75ab4e70ae15a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.694+0000 7fb789ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.694+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794004020 con 0x55843e90b660 2023-07-21T22:40:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.694+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb794004180 con 0x55843e90b660 2023-07-21T22:40:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb79c0ca970 con 0x55843e90b660 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb794004340 con 0x55843e90b660 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 10 client.18018 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55843e683250 con 0x55843e90b660 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79c0ca970 con 0x55843e90b660 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb79400e700 con 0x55843e90b660 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 10 client.18018 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 10 client.18018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 20 client.18018 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e973420 0x55843e975830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 4 client.18018 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.696+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55843e811ff0 con 0x55843e973420 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.697+0000 7fb7a259c700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e973420 0x55843e975830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.697+0000 7fb789ffb700 10 client.18018 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:45.697+0000 7fb7a259c700 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e973420 0x55843e975830 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b8437bc5ce6d6d server_cookie=8473d18a0db6834c in_seq=0 out_seq=0 2023-07-21T22:40:45.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:45 smithi194 ceph-mon[98288]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 656 op/s 2023-07-21T22:40:45.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:45 smithi134 ceph-mon[97873]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 656 op/s 2023-07-21T22:40:45.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:45 smithi093 ceph-mon[96579]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 16 MiB/s wr, 656 op/s 2023-07-21T22:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.030+0000 7fb789ffb700 1 -- 172.21.15.93:0/607866118 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55843e811ff0 con 0x55843e973420 2023-07-21T22:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.030+0000 7fb789ffb700 10 client.18018 handle_command_reply: tid=0 2023-07-21T22:40:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.033+0000 7fb7b5040b80 1 client.18018 shutdown 2023-07-21T22:40:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.036+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 msgr2=0x55843e914160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.036+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55843e901b80 tx=0x7fb794011a00 comp rx=0 tx=0).stop 2023-07-21T22:40:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 shutdown_connections 2023-07-21T22:40:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55843e973420 0x55843e975830 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55843e90eef0 0x55843e9146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55843e7616b0 0x55843e913c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 --2- 172.21.15.93:0/607866118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55843e90b660 0x55843e914160 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 >> 172.21.15.93:0/607866118 conn(0x55843e8faae0 msgr2=0x55843e8ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.041+0000 7fb789ffb700 0 client.18018 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.043+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 shutdown_connections 2023-07-21T22:40:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.044+0000 7fb7b5040b80 1 -- 172.21.15.93:0/607866118 wait complete. 2023-07-21T22:40:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:40:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:40:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:40:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:40:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.045+0000 7fb7b5040b80 20 client.18018 trim_cache size 0 max 16384 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.045+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 msgr2=0x7fb7900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.045+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 0x7fb7900a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0098e0 tx=0x7fb78c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.045+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 msgr2=0x7fb7a419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.045+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b780 tx=0x7fb79c00bb40 comp rx=0 tx=0).stop 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 shutdown_connections 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fb7900a1490 0x7fb7900a3940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7a4104ad0 0x7fb7a419ca40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7a40feac0 0x7fb7a41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 --2- 172.21.15.93:0/2606124287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7a40ff470 0x7fb7a4198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.047+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 >> 172.21.15.93:0/2606124287 conn(0x7fb7a4075bd0 msgr2=0x7fb7a40fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.048+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 shutdown_connections 2023-07-21T22:40:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:46.048+0000 7fb7a8ebb700 1 -- 172.21.15.93:0/2606124287 wait complete. 2023-07-21T22:40:46.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 369586 27.94 MB/sec execute 562 sec latency 293.214 ms 2023-07-21T22:40:47.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 370034 27.94 MB/sec execute 563 sec latency 128.357 ms 2023-07-21T22:40:47.838 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:47 smithi194 ceph-mon[98288]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 458 op/s 2023-07-21T22:40:47.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:47 smithi134 ceph-mon[97873]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 458 op/s 2023-07-21T22:40:47.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:47 smithi093 ceph-mon[96579]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 458 op/s 2023-07-21T22:40:48.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 370662 27.96 MB/sec execute 564 sec latency 82.040 ms 2023-07-21T22:40:49.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 371171 27.96 MB/sec execute 565 sec latency 144.194 ms 2023-07-21T22:40:49.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:49 smithi194 ceph-mon[98288]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 535 op/s 2023-07-21T22:40:49.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:49 smithi134 ceph-mon[97873]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 535 op/s 2023-07-21T22:40:49.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:49 smithi093 ceph-mon[96579]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 535 op/s 2023-07-21T22:40:50.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 371659 27.96 MB/sec execute 566 sec latency 94.997 ms 2023-07-21T22:40:51.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 372159 27.96 MB/sec execute 567 sec latency 78.926 ms 2023-07-21T22:40:51.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:51 smithi194 ceph-mon[98288]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 678 op/s 2023-07-21T22:40:51.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:51 smithi134 ceph-mon[97873]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 678 op/s 2023-07-21T22:40:51.939 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:51 smithi093 ceph-mon[96579]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 678 op/s 2023-07-21T22:40:52.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 372595 27.95 MB/sec execute 568 sec latency 83.704 ms 2023-07-21T22:40:53.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 373109 27.94 MB/sec execute 569 sec latency 74.126 ms 2023-07-21T22:40:53.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:53 smithi194 ceph-mon[98288]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 710 op/s 2023-07-21T22:40:53.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:53 smithi134 ceph-mon[97873]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 710 op/s 2023-07-21T22:40:53.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:53 smithi093 ceph-mon[96579]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 710 op/s 2023-07-21T22:40:54.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 373667 27.95 MB/sec execute 570 sec latency 89.900 ms 2023-07-21T22:40:55.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 374204 27.94 MB/sec execute 571 sec latency 244.064 ms 2023-07-21T22:40:55.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:55 smithi194 ceph-mon[98288]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 851 op/s 2023-07-21T22:40:55.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:55 smithi134 ceph-mon[97873]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 851 op/s 2023-07-21T22:40:55.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:55 smithi093 ceph-mon[96579]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 22 MiB/s wr, 851 op/s 2023-07-21T22:40:56.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 374822 27.96 MB/sec execute 572 sec latency 69.785 ms 2023-07-21T22:40:57.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 375353 27.97 MB/sec execute 573 sec latency 68.357 ms 2023-07-21T22:40:57.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:57 smithi194 ceph-mon[98288]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-07-21T22:40:57.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:57 smithi134 ceph-mon[97873]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-07-21T22:40:57.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:57 smithi093 ceph-mon[96579]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 598 op/s 2023-07-21T22:40:58.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 375776 27.94 MB/sec execute 574 sec latency 108.744 ms 2023-07-21T22:40:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 -- 172.21.15.93:0/3934197381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 msgr2=0x7f77681089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 --2- 172.21.15.93:0/3934197381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681089f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7758009990 tx=0x7f7758009ca0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 -- 172.21.15.93:0/3934197381 shutdown_connections 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 --2- 172.21.15.93:0/3934197381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7768111bd0 0x7f7768113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 --2- 172.21.15.93:0/3934197381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681089f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 --2- 172.21.15.93:0/3934197381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7768108f30 0x7f7768111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 -- 172.21.15.93:0/3934197381 >> 172.21.15.93:0/3934197381 conn(0x7f776806c270 msgr2=0x7f776806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 -- 172.21.15.93:0/3934197381 shutdown_connections 2023-07-21T22:40:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.209+0000 7f776dfce700 1 -- 172.21.15.93:0/3934197381 wait complete. 2023-07-21T22:40:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 Processor -- start 2023-07-21T22:40:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 -- start start 2023-07-21T22:40:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7768108f30 0x7f77681aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7768111bd0 0x7f77681a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77681a4e60 con 0x7f7768108620 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77681a4fd0 con 0x7f7768108f30 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.210+0000 7f776dfce700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77681a5190 con 0x7f7768111bd0 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50236/0 (socket says 172.21.15.93:50236) 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 -- 172.21.15.93:0/2968992629 learned_addr learned my addr 172.21.15.93:0/2968992629 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 -- 172.21.15.93:0/2968992629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7768111bd0 msgr2=0x7f77681a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7768111bd0 0x7f77681a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 -- 172.21.15.93:0/2968992629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7768108f30 msgr2=0x7f77681aaed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7768108f30 0x7f77681aaed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 -- 172.21.15.93:0/2968992629 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580095f0 con 0x7f7768108620 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f77677fe700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f77580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f7764ff9700 1 -- 172.21.15.93:0/2968992629 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775801d070 con 0x7f7768108620 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77681a5410 con 0x7f7768108620 2023-07-21T22:40:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f7764ff9700 1 -- 172.21.15.93:0/2968992629 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7758021410 con 0x7f7768108620 2023-07-21T22:40:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.211+0000 7f7764ff9700 1 -- 172.21.15.93:0/2968992629 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758017650 con 0x7f7768108620 2023-07-21T22:40:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.212+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77681b1980 con 0x7f7768108620 2023-07-21T22:40:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.214+0000 7f7764ff9700 1 -- 172.21.15.93:0/2968992629 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7758021580 con 0x7f7768108620 2023-07-21T22:40:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.215+0000 7f7764ff9700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.215+0000 7f7764ff9700 1 -- 172.21.15.93:0/2968992629 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f77580ce040 con 0x7f7768108620 2023-07-21T22:40:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.215+0000 7f7766ffd700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.216+0000 7f7766ffd700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 0x7f77500a3770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f77681138d0 tx=0x7f775c00d040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.216+0000 7f777a153b80 1 Processor -- start 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.216+0000 7f777a153b80 1 -- start start 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698ab5c50 con 0x559698bb6660 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698b089e0 con 0x559698a0c6b0 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f777a153b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698b01e70 con 0x559698bb6210 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f77677fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f77677fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33172/0 (socket says 172.21.15.93:33172) 2023-07-21T22:40:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f77677fe700 1 -- 172.21.15.93:0/3258181080 learned_addr learned my addr 172.21.15.93:0/3258181080 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 -- 172.21.15.93:0/3258181080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 msgr2=0x559698baff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 -- 172.21.15.93:0/3258181080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 msgr2=0x559698bb5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 -- 172.21.15.93:0/3258181080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55969892fd80 con 0x559698bb6660 2023-07-21T22:40:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f7767fff700 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb04a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7768069880 tx=0x7f7760007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a265b9b0ff6ff3ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.217+0000 7f774dffb700 1 -- 172.21.15.93:0/3258181080 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000b5e0 con 0x559698bb6660 2023-07-21T22:40:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f774dffb700 1 -- 172.21.15.93:0/3258181080 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7760004bb0 con 0x559698bb6660 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f774dffb700 1 -- 172.21.15.93:0/3258181080 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000d0d0 con 0x559698bb6660 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 -- 172.21.15.93:0/3258181080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 msgr2=0x559698bb04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb04a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f7768069880 tx=0x7f7760007bb0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 -- 172.21.15.93:0/3258181080 shutdown_connections 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb04a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 --2- 172.21.15.93:0/3258181080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.218+0000 7f777a153b80 1 -- 172.21.15.93:0/3258181080 >> 172.21.15.93:0/3258181080 conn(0x559698ba5ae0 msgr2=0x559698bab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- 172.21.15.93:0/3258181080 shutdown_connections 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- 172.21.15.93:0/3258181080 wait complete. 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- start start 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 Processor -- start 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698a0c6b0 0x559698bbc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 0x559698bb9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698aa1a60 con 0x559698bb6660 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698aa1760 con 0x559698bb6210 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.219+0000 7f777a153b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596989858c0 con 0x559698a0c6b0 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f777a153b80 10 client.0 _ll_register_callbacks cb 0x559698a9f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 0x559698bb9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50252/0 (socket says 172.21.15.93:50252) 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 -- 172.21.15.93:0/2554494935 learned_addr learned my addr 172.21.15.93:0/2554494935 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f77677fe700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698a0c6b0 0x559698bbc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 -- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698a0c6b0 msgr2=0x559698bbc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698a0c6b0 0x559698bbc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 -- 172.21.15.93:0/2554494935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 msgr2=0x559698bb9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 0x559698bb9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 -- 172.21.15.93:0/2554494935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77600075b0 con 0x559698bb6660 2023-07-21T22:40:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7766ffd700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 0x559698bb9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7767fff700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f776000b080 tx=0x7f776000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896a2a5b0c5f27cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760003c00 con 0x559698bb6660 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.220+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f776001d030 con 0x559698bb6660 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.221+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559698a5b490 con 0x559698bb6660 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.221+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7760003d60 con 0x559698bb6660 2023-07-21T22:40:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.221+0000 7f777a153b80 10 client.18030 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.221+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x559698918c00 con 0x559698bb6660 2023-07-21T22:40:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.226+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7760027030 con 0x559698bb6660 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.226+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f77600042d0 con 0x559698bb6660 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 10 client.18030 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 20 client.18030 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698bc9bc0 0x559698bcbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 4 client.18030 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:40:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559698afae50 con 0x559698bc9bc0 2023-07-21T22:40:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.227+0000 7f77677fe700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698bc9bc0 0x559698bcbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.228+0000 7f7746ffd700 10 client.18030 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.228+0000 7f77677fe700 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698bc9bc0 0x559698bcbfd0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86de262329bd4500 server_cookie=1733208d5e702ada in_seq=0 out_seq=0 2023-07-21T22:40:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.230+0000 7f7746ffd700 1 -- 172.21.15.93:0/2554494935 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559698afae50 con 0x559698bc9bc0 2023-07-21T22:40:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.230+0000 7f7746ffd700 10 client.18030 handle_command_reply: tid=0 2023-07-21T22:40:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.230+0000 7f777a153b80 1 client.18030 shutdown 2023-07-21T22:40:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.230+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 msgr2=0x559698bb9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.230+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f776000b080 tx=0x7f776000da60 comp rx=0 tx=0).stop 2023-07-21T22:40:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 shutdown_connections 2023-07-21T22:40:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698bc9bc0 0x559698bcbfd0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb9f70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698a0c6b0 0x559698bbc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 --2- 172.21.15.93:0/2554494935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698bb6210 0x559698bb9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 >> 172.21.15.93:0/2554494935 conn(0x559698ba5ae0 msgr2=0x559698baa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.231+0000 7f7746ffd700 0 client.18030 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.232+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 shutdown_connections 2023-07-21T22:40:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.232+0000 7f777a153b80 1 -- 172.21.15.93:0/2554494935 wait complete. 2023-07-21T22:40:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.233+0000 7f777a153b80 20 client.18030 trim_cache size 0 max 16384 2023-07-21T22:40:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.259+0000 7f777a153b80 1 Processor -- start 2023-07-21T22:40:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 -- start start 2023-07-21T22:40:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 0x559698bc0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698bac760 con 0x559698bb6210 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698ba7c00 con 0x559698a0c6b0 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f777a153b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698bb0a70 con 0x559698bb6660 2023-07-21T22:40:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50258/0 (socket says 172.21.15.93:50258) 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7766ffd700 1 -- 172.21.15.93:0/1873944759 learned_addr learned my addr 172.21.15.93:0/1873944759 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7767fff700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 0x559698bc0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 -- 172.21.15.93:0/1873944759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 msgr2=0x559698bc0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 0x559698bc0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 -- 172.21.15.93:0/1873944759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 msgr2=0x559698baff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f77677fe700 1 -- 172.21.15.93:0/1873944759 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596989561f0 con 0x559698a0c6b0 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7766ffd700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.260+0000 7f7767fff700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 0x559698bc0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.261+0000 7f77677fe700 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb9e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7758097c10 tx=0x7f77580a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42796859577eff82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.261+0000 7f774e7fc700 1 -- 172.21.15.93:0/1873944759 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775809b1a0 con 0x559698a0c6b0 2023-07-21T22:40:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.261+0000 7f774e7fc700 1 -- 172.21.15.93:0/1873944759 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f775809f1d0 con 0x559698a0c6b0 2023-07-21T22:40:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.261+0000 7f774e7fc700 1 -- 172.21.15.93:0/1873944759 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775809b660 con 0x559698a0c6b0 2023-07-21T22:40:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 -- 172.21.15.93:0/1873944759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 msgr2=0x559698bb9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb9e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7758097c10 tx=0x7f77580a1040 comp rx=0 tx=0).stop 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 -- 172.21.15.93:0/1873944759 shutdown_connections 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6660 0x559698bc0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bb9e80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 --2- 172.21.15.93:0/1873944759 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6210 0x559698baff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 -- 172.21.15.93:0/1873944759 >> 172.21.15.93:0/1873944759 conn(0x559698ba5ae0 msgr2=0x559698bb7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 -- 172.21.15.93:0/1873944759 shutdown_connections 2023-07-21T22:40:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.262+0000 7f777a153b80 1 -- 172.21.15.93:0/1873944759 wait complete. 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 -- start start 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 Processor -- start 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bc0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698bb1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698bb2990 con 0x559698bb6660 2023-07-21T22:40:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698c1ca30 con 0x559698a0c6b0 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x559698c1cb70 con 0x559698bb6210 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f777a153b80 10 client.0 _ll_register_callbacks cb 0x7f776000ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f77677fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bc0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f77677fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bc0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33192/0 (socket says 172.21.15.93:33192) 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f77677fe700 1 -- 172.21.15.93:0/285771940 learned_addr learned my addr 172.21.15.93:0/285771940 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.263+0000 7f7767fff700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7766ffd700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698bb1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 -- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 msgr2=0x559698bb1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698bb1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 -- 172.21.15.93:0/285771940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 msgr2=0x559698bc0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bc0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 -- 172.21.15.93:0/285771940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c007ad0 con 0x559698bb6660 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7767fff700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb22c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77600078a0 tx=0x7f7760017490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55327eaf77a98b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:40:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776000d4e0 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f776001d040 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.264+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7760004bb0 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7760004950 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 10 client.18036 fetch_fsmap learned FSMap version 91 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x7f77681b1980 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7760039020 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7760017d10 con 0x559698bb6660 2023-07-21T22:40:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 10 client.18036 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 20 client.18036 populate_metadata read hostname 'smithi093' 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698c1de90 0x559698c202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 4 client.18036 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559698abcff0 con 0x559698c1de90 2023-07-21T22:40:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.265+0000 7f77677fe700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698c1de90 0x559698c202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:40:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.266+0000 7f7746ffd700 10 client.18036 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.266+0000 7f77677fe700 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698c1de90 0x559698c202a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4667233920d7a84a server_cookie=1b3fa33470e96847 in_seq=0 out_seq=0 2023-07-21T22:40:59.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 376171 27.94 MB/sec execute 575 sec latency 266.962 ms 2023-07-21T22:40:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.664+0000 7f7746ffd700 1 -- 172.21.15.93:0/285771940 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559698abcff0 con 0x559698c1de90 2023-07-21T22:40:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.664+0000 7f7746ffd700 10 client.18036 handle_command_reply: tid=0 2023-07-21T22:40:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.664+0000 7f777a153b80 1 client.18036 shutdown 2023-07-21T22:40:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.666+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 msgr2=0x559698bb22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.666+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb22c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f77600078a0 tx=0x7f7760017490 comp rx=0 tx=0).stop 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.667+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 shutdown_connections 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.667+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x559698c1de90 0x559698c202a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.667+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559698bb6660 0x559698bb22c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.667+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559698a0c6b0 0x559698bc0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.667+0000 7f777a153b80 1 --2- 172.21.15.93:0/285771940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559698bb6210 0x559698bb1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.668+0000 7f7746ffd700 0 client.18036 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:40:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.668+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 >> 172.21.15.93:0/285771940 conn(0x559698ba5ae0 msgr2=0x559698bb48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.668+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 shutdown_connections 2023-07-21T22:40:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.668+0000 7f777a153b80 1 -- 172.21.15.93:0/285771940 wait complete. 2023-07-21T22:40:59.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.668+0000 7f777a153b80 20 client.18036 trim_cache size 0 max 16384 2023-07-21T22:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 msgr2=0x7f77500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 0x7f77500a3770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f77681138d0 tx=0x7f775c00d040 comp rx=0 tx=0).stop 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 msgr2=0x7f77681aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f7758000c00 tx=0x7f77580047c0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 shutdown_connections 2023-07-21T22:40:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7768111bd0 0x7f77681a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f77500a12c0 0x7f77500a3770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7768108620 0x7f77681aa990 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 --2- 172.21.15.93:0/2968992629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7768108f30 0x7f77681aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 >> 172.21.15.93:0/2968992629 conn(0x7f776806c270 msgr2=0x7f776806eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 shutdown_connections 2023-07-21T22:40:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:40:59.669+0000 7f776dfce700 1 -- 172.21.15.93:0/2968992629 wait complete. 2023-07-21T22:40:59.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:40:59 smithi134 ceph-mon[97873]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 618 op/s 2023-07-21T22:40:59.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:40:59 smithi093 ceph-mon[96579]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 618 op/s 2023-07-21T22:41:00.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:40:59 smithi194 ceph-mon[98288]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 13 MiB/s wr, 618 op/s 2023-07-21T22:41:00.573 INFO:tasks.workunit.client.0.smithi093.stdout: 10 376689 27.94 MB/sec execute 576 sec latency 297.045 ms 2023-07-21T22:41:01.573 INFO:tasks.workunit.client.0.smithi093.stdout: 10 376764 27.90 MB/sec execute 577 sec latency 820.585 ms 2023-07-21T22:41:01.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:01 smithi134 ceph-mon[97873]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-21T22:41:01.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:01 smithi093 ceph-mon[96579]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-21T22:41:02.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:01 smithi194 ceph-mon[98288]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 687 op/s 2023-07-21T22:41:02.572 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377046 27.87 MB/sec execute 578 sec latency 1236.619 ms 2023-07-21T22:41:03.573 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377221 27.83 MB/sec execute 579 sec latency 199.459 ms 2023-07-21T22:41:03.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:03 smithi134 ceph-mon[97873]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 402 op/s 2023-07-21T22:41:03.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:03 smithi093 ceph-mon[96579]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 402 op/s 2023-07-21T22:41:04.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:03 smithi194 ceph-mon[98288]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 402 op/s 2023-07-21T22:41:04.573 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377310 27.79 MB/sec execute 580 sec latency 240.212 ms 2023-07-21T22:41:05.573 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377422 27.76 MB/sec execute 581 sec latency 695.187 ms 2023-07-21T22:41:05.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:05 smithi134 ceph-mon[97873]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 620 op/s 2023-07-21T22:41:05.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:05 smithi134 ceph-mon[97873]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T22:41:05.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:05 smithi093 ceph-mon[96579]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 620 op/s 2023-07-21T22:41:05.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:05 smithi093 ceph-mon[96579]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T22:41:06.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:05 smithi194 ceph-mon[98288]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 620 op/s 2023-07-21T22:41:06.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:05 smithi194 ceph-mon[98288]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T22:41:06.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377576 27.73 MB/sec execute 582 sec latency 1087.107 ms 2023-07-21T22:41:07.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 377914 27.71 MB/sec execute 583 sec latency 581.022 ms 2023-07-21T22:41:07.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:07 smithi134 ceph-mon[97873]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 564 op/s 2023-07-21T22:41:07.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:07 smithi093 ceph-mon[96579]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 564 op/s 2023-07-21T22:41:08.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:07 smithi194 ceph-mon[98288]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 564 op/s 2023-07-21T22:41:08.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 378451 27.73 MB/sec execute 584 sec latency 69.509 ms 2023-07-21T22:41:09.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 378888 27.72 MB/sec execute 585 sec latency 80.628 ms 2023-07-21T22:41:09.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:09 smithi134 ceph-mon[97873]: pgmap v788: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 618 op/s 2023-07-21T22:41:09.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:09 smithi093 ceph-mon[96579]: pgmap v788: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 618 op/s 2023-07-21T22:41:10.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:09 smithi194 ceph-mon[98288]: pgmap v788: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.5 MiB/s wr, 618 op/s 2023-07-21T22:41:10.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 379286 27.71 MB/sec execute 586 sec latency 86.324 ms 2023-07-21T22:41:11.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 379828 27.70 MB/sec execute 587 sec latency 116.272 ms 2023-07-21T22:41:11.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:11 smithi093 ceph-mon[96579]: pgmap v789: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 601 op/s 2023-07-21T22:41:12.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:11 smithi194 ceph-mon[98288]: pgmap v789: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 601 op/s 2023-07-21T22:41:12.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:11 smithi134 ceph-mon[97873]: pgmap v789: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.5 MiB/s wr, 601 op/s 2023-07-21T22:41:12.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 380296 27.70 MB/sec execute 588 sec latency 117.300 ms 2023-07-21T22:41:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 -- 172.21.15.93:0/3517855598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 msgr2=0x7f54e410b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 --2- 172.21.15.93:0/3517855598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e410b850 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00b0a0 tx=0x7f54dc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 -- 172.21.15.93:0/3517855598 shutdown_connections 2023-07-21T22:41:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 --2- 172.21.15.93:0/3517855598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e410b850 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 --2- 172.21.15.93:0/3517855598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 0x7f54e4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 --2- 172.21.15.93:0/3517855598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 0x7f54e40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 -- 172.21.15.93:0/3517855598 >> 172.21.15.93:0/3517855598 conn(0x7f54e40fa4d0 msgr2=0x7f54e40fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.811+0000 7f54ea008700 1 -- 172.21.15.93:0/3517855598 shutdown_connections 2023-07-21T22:41:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.812+0000 7f54ea008700 1 -- 172.21.15.93:0/3517855598 wait complete. 2023-07-21T22:41:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 Processor -- start 2023-07-21T22:41:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 -- start start 2023-07-21T22:41:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 0x7f54e4196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 0x7f54e41a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41a2120 con 0x7f54e40ff3c0 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41a2290 con 0x7f54e4104a20 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54ea008700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41a2450 con 0x7f54e40fea10 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50276/0 (socket says 172.21.15.93:50276) 2023-07-21T22:41:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3210761691 learned_addr learned my addr 172.21.15.93:0/3210761691 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54e3fff700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 0x7f54e41a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.813+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 0x7f54e4196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3210761691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 msgr2=0x7f54e4196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 0x7f54e4196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3210761691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 msgr2=0x7f54e41a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 0x7f54e41a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3210761691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009d00 con 0x7f54e40ff3c0 2023-07-21T22:41:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f54d800d980 tx=0x7f54d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e0ff9700 1 -- 172.21.15.93:0/3210761691 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8004220 con 0x7f54e40ff3c0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e0ff9700 1 -- 172.21.15.93:0/3210761691 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d8007640 con 0x7f54e40ff3c0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54e0ff9700 1 -- 172.21.15.93:0/3210761691 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8005610 con 0x7f54e40ff3c0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e41a2730 con 0x7f54e40ff3c0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.814+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e41a2bf0 con 0x7f54e40ff3c0 2023-07-21T22:41:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.816+0000 7f54e0ff9700 1 -- 172.21.15.93:0/3210761691 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f54d8004750 con 0x7f54e40ff3c0 2023-07-21T22:41:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.817+0000 7f54e0ff9700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 0x7f54cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.817+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 0x7f54cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.817+0000 7f54e0ff9700 1 -- 172.21.15.93:0/3210761691 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f54d80ccbe0 con 0x7f54e40ff3c0 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.818+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 0x7f54cc0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54e40ffb50 tx=0x7f54d400d4b0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.818+0000 7f54f618db80 1 Processor -- start 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.818+0000 7f54f618db80 1 -- start start 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.818+0000 7f54f618db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.818+0000 7f54f618db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cb9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54f618db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54f618db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2bbfc50 con 0x55afd2b166b0 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54f618db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2c129e0 con 0x55afd2cc0210 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54f618db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2c0be70 con 0x55afd2cc0660 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50284/0 (socket says 172.21.15.93:50284) 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 -- 172.21.15.93:0/3990555016 learned_addr learned my addr 172.21.15.93:0/3990555016 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e3fff700 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cb9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 -- 172.21.15.93:0/3990555016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 msgr2=0x55afd2cba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 -- 172.21.15.93:0/3990555016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 msgr2=0x55afd2cb9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cb9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 -- 172.21.15.93:0/3990555016 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd2a39d80 con 0x55afd2b166b0 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f54d400fc30 tx=0x7f54d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47caee6618a86bb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54c9ffb700 1 -- 172.21.15.93:0/3990555016 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4011ad0 con 0x55afd2b166b0 2023-07-21T22:41:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54c9ffb700 1 -- 172.21.15.93:0/3990555016 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d4011c30 con 0x55afd2b166b0 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.819+0000 7f54c9ffb700 1 -- 172.21.15.93:0/3990555016 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d401d630 con 0x55afd2b166b0 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 -- 172.21.15.93:0/3990555016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 msgr2=0x55afd2cbfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f54d400fc30 tx=0x7f54d4012000 comp rx=0 tx=0).stop 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 -- 172.21.15.93:0/3990555016 shutdown_connections 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbfcd0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 --2- 172.21.15.93:0/3990555016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cb9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 -- 172.21.15.93:0/3990555016 >> 172.21.15.93:0/3990555016 conn(0x55afd2cafae0 msgr2=0x55afd2cb50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 -- 172.21.15.93:0/3990555016 shutdown_connections 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.820+0000 7f54f618db80 1 -- 172.21.15.93:0/3990555016 wait complete. 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 -- start start 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 Processor -- start 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0210 0x55afd2cc3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2baba60 con 0x55afd2b166b0 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2bab760 con 0x55afd2cc0660 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2a8f8c0 con 0x55afd2cc0210 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54f618db80 10 client.0 _ll_register_callbacks cb 0x55afd2ba9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50290/0 (socket says 172.21.15.93:50290) 2023-07-21T22:41:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 -- 172.21.15.93:0/2818462174 learned_addr learned my addr 172.21.15.93:0/2818462174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e3fff700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 -- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0210 msgr2=0x55afd2cc3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0210 0x55afd2cc3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 -- 172.21.15.93:0/2818462174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 msgr2=0x55afd2cc4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.821+0000 7f54e37fe700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54e37fe700 1 -- 172.21.15.93:0/2818462174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d400f910 con 0x55afd2b166b0 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54e37fe700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005950 tx=0x7f54d401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb70d98cee2bb80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d402d070 con 0x55afd2b166b0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f54d4005260 con 0x55afd2b166b0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afd2b65490 con 0x55afd2b166b0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54d4005420 con 0x55afd2b166b0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54f618db80 10 client.18048 fetch_fsmap learned FSMap version 91 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55afd2a22c00 con 0x55afd2b166b0 2023-07-21T22:41:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d403e020 con 0x55afd2b166b0 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.822+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f54d4004990 con 0x55afd2b166b0 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 10 client.18048 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 20 client.18048 populate_metadata read hostname 'smithi093' 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2cd39c0 0x55afd2cd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 4 client.18048 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afd2c04e50 con 0x55afd2cd39c0 2023-07-21T22:41:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2cd39c0 0x55afd2cd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54c2ffd700 10 client.18048 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.823+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2cd39c0 0x55afd2cd5da0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d0c61603b7d292 server_cookie=7a12c81865d4260b in_seq=0 out_seq=0 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.824+0000 7f54c2ffd700 1 -- 172.21.15.93:0/2818462174 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55afd2c04e50 con 0x55afd2cd39c0 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.824+0000 7f54c2ffd700 10 client.18048 handle_command_reply: tid=0 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.824+0000 7f54f618db80 1 client.18048 shutdown 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 msgr2=0x55afd2cc38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f54d4005950 tx=0x7f54d401ec50 comp rx=0 tx=0).stop 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 shutdown_connections 2023-07-21T22:41:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2cd39c0 0x55afd2cd5da0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54c2ffd700 0 client.18048 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cc38f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 --2- 172.21.15.93:0/2818462174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0210 0x55afd2cc3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 >> 172.21.15.93:0/2818462174 conn(0x55afd2cafae0 msgr2=0x55afd2cb43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 shutdown_connections 2023-07-21T22:41:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 1 -- 172.21.15.93:0/2818462174 wait complete. 2023-07-21T22:41:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.825+0000 7f54f618db80 20 client.18048 trim_cache size 0 max 16384 2023-07-21T22:41:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 Processor -- start 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 -- start start 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cc41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cb4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2cc4e60 con 0x55afd2b166b0 2023-07-21T22:41:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2cbf900 con 0x55afd2cc0210 2023-07-21T22:41:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54f618db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2cb14f0 con 0x55afd2cc0660 2023-07-21T22:41:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cb4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cb4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37380/0 (socket says 172.21.15.93:37380) 2023-07-21T22:41:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54e3fff700 1 -- 172.21.15.93:0/3797020091 learned_addr learned my addr 172.21.15.93:0/3797020091 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.843+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cc41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e37fe700 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3797020091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 msgr2=0x55afd2cb4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cb4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3797020091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 msgr2=0x55afd2cbf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 -- 172.21.15.93:0/3797020091 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afd2a601f0 con 0x55afd2cc0210 2023-07-21T22:41:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cc41a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ca300 tx=0x7f54d80960f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a49b2e31001e72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54ca7fc700 1 -- 172.21.15.93:0/3797020091 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80a20d0 con 0x55afd2cc0210 2023-07-21T22:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54ca7fc700 1 -- 172.21.15.93:0/3797020091 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d8098020 con 0x55afd2cc0210 2023-07-21T22:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.844+0000 7f54ca7fc700 1 -- 172.21.15.93:0/3797020091 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8099170 con 0x55afd2cc0210 2023-07-21T22:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 -- 172.21.15.93:0/3797020091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 msgr2=0x55afd2cc41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cc41a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ca300 tx=0x7f54d80960f0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 -- 172.21.15.93:0/3797020091 shutdown_connections 2023-07-21T22:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2cc0660 0x55afd2cb4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2b166b0 0x55afd2cbf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 --2- 172.21.15.93:0/3797020091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0210 0x55afd2cc41a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 -- 172.21.15.93:0/3797020091 >> 172.21.15.93:0/3797020091 conn(0x55afd2cafae0 msgr2=0x55afd2cc2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 -- 172.21.15.93:0/3797020091 shutdown_connections 2023-07-21T22:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.845+0000 7f54f618db80 1 -- 172.21.15.93:0/3797020091 wait complete. 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 -- start start 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 Processor -- start 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 0x55afd2cc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2d27860 con 0x55afd2cc0210 2023-07-21T22:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2d279a0 con 0x55afd2cc0660 2023-07-21T22:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afd2d27b30 con 0x55afd2b166b0 2023-07-21T22:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54f618db80 10 client.0 _ll_register_callbacks cb 0x7f54d8095fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50306/0 (socket says 172.21.15.93:50306) 2023-07-21T22:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54e2ffd700 1 -- 172.21.15.93:0/620761979 learned_addr learned my addr 172.21.15.93:0/620761979 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.846+0000 7f54e37fe700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 0x55afd2cc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 -- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 msgr2=0x55afd2cc9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 0x55afd2cc9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 -- 172.21.15.93:0/620761979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 msgr2=0x55afd2cc9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 -- 172.21.15.93:0/620761979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e41a2bf0 con 0x55afd2cc0210 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e3fff700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:41:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e37fe700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 0x55afd2cc9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54e2ffd700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ca3b0 tx=0x7f54d8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a2cf3fe7201ceb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8099170 con 0x55afd2cc0210 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f54d80b8070 con 0x55afd2cc0210 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54d4011c30 con 0x55afd2cc0210 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54d80992d0 con 0x55afd2cc0210 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54f618db80 10 client.18054 fetch_fsmap learned FSMap version 91 2023-07-21T22:41:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.847+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x55afd2a38250 con 0x55afd2cc0210 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d4011c30 con 0x55afd2cc0210 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f54d80a1c40 con 0x55afd2cc0210 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 10 client.18054 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 20 client.18054 populate_metadata read hostname 'smithi093' 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2d28eb0 0x55afd2d2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 4 client.18054 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:41:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afd2bc6ff0 con 0x55afd2d28eb0 2023-07-21T22:41:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.848+0000 7f54e37fe700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2d28eb0 0x55afd2d2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.851+0000 7f54c2ffd700 10 client.18054 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:12.851+0000 7f54e37fe700 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2d28eb0 0x55afd2d2b2c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8164065d9a0323 server_cookie=bb1cff1a6cef1a48 in_seq=0 out_seq=0 2023-07-21T22:41:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54c2ffd700 1 -- 172.21.15.93:0/620761979 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afd2bc6ff0 con 0x55afd2d28eb0 2023-07-21T22:41:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54c2ffd700 10 client.18054 handle_command_reply: tid=0 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 client.18054 shutdown 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 msgr2=0x55afd2cc97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ca3b0 tx=0x7f54d8005770 comp rx=0 tx=0).stop 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 shutdown_connections 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x55afd2d28eb0 0x55afd2d2b2c0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afd2cc0660 0x55afd2cc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55afd2b166b0 0x55afd2cc9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 --2- 172.21.15.93:0/620761979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55afd2cc0210 0x55afd2cc97a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 >> 172.21.15.93:0/620761979 conn(0x55afd2cafae0 msgr2=0x55afd2cc0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54c2ffd700 0 client.18054 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 shutdown_connections 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 1 -- 172.21.15.93:0/620761979 wait complete. 2023-07-21T22:41:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.168+0000 7f54f618db80 20 client.18054 trim_cache size 0 max 16384 2023-07-21T22:41:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:41:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:41:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:41:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 msgr2=0x7f54cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 0x7f54cc0a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54e40ffb50 tx=0x7f54d400d4b0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 msgr2=0x7f54e41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f54d800d980 tx=0x7f54d800dc90 comp rx=0 tx=0).stop 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 shutdown_connections 2023-07-21T22:41:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e4104a20 0x7f54e41a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54e40fea10 0x7f54e4196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f54cc0a12c0 0x7f54cc0a3770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 --2- 172.21.15.93:0/3210761691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ff3c0 0x7f54e41971c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 >> 172.21.15.93:0/3210761691 conn(0x7f54e40fa4d0 msgr2=0x7f54e4105890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 shutdown_connections 2023-07-21T22:41:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:13.169+0000 7f54ea008700 1 -- 172.21.15.93:0/3210761691 wait complete. 2023-07-21T22:41:13.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 380792 27.69 MB/sec execute 589 sec latency 330.514 ms 2023-07-21T22:41:13.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:13 smithi093 ceph-mon[96579]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.9 MiB/s wr, 579 op/s 2023-07-21T22:41:14.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:13 smithi194 ceph-mon[98288]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.9 MiB/s wr, 579 op/s 2023-07-21T22:41:14.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:13 smithi134 ceph-mon[97873]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.9 MiB/s wr, 579 op/s 2023-07-21T22:41:14.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 381370 27.70 MB/sec execute 590 sec latency 94.936 ms 2023-07-21T22:41:14.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:14 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:41:15.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:14 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:41:15.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:14 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:41:15.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 381875 27.71 MB/sec execute 591 sec latency 103.595 ms 2023-07-21T22:41:15.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:15 smithi093 ceph-mon[96579]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 502 op/s 2023-07-21T22:41:15.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:15 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:41:15.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:15 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:41:15.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:15 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:41:16.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:15 smithi194 ceph-mon[98288]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 502 op/s 2023-07-21T22:41:16.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:15 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:41:16.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:15 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:41:16.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:15 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:41:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:15 smithi134 ceph-mon[97873]: pgmap v791: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 502 op/s 2023-07-21T22:41:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:15 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:41:16.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:15 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:41:16.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:15 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' 2023-07-21T22:41:16.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 382345 27.71 MB/sec execute 592 sec latency 93.891 ms 2023-07-21T22:41:17.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 382717 27.69 MB/sec execute 593 sec latency 110.544 ms 2023-07-21T22:41:17.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:17 smithi093 ceph-mon[96579]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 425 op/s 2023-07-21T22:41:17.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:17 smithi093 ceph-mon[96579]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T22:41:18.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:17 smithi194 ceph-mon[98288]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 425 op/s 2023-07-21T22:41:18.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:17 smithi194 ceph-mon[98288]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T22:41:18.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:17 smithi134 ceph-mon[97873]: pgmap v792: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.9 MiB/s wr, 425 op/s 2023-07-21T22:41:18.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:17 smithi134 ceph-mon[97873]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T22:41:18.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 383163 27.69 MB/sec execute 594 sec latency 79.178 ms 2023-07-21T22:41:19.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 383453 27.65 MB/sec execute 595 sec latency 262.729 ms 2023-07-21T22:41:19.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:19 smithi093 ceph-mon[96579]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 442 op/s 2023-07-21T22:41:20.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:19 smithi194 ceph-mon[98288]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 442 op/s 2023-07-21T22:41:20.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:19 smithi134 ceph-mon[97873]: pgmap v794: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.8 MiB/s wr, 442 op/s 2023-07-21T22:41:20.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 383994 27.67 MB/sec execute 596 sec latency 160.113 ms 2023-07-21T22:41:21.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 384625 27.67 MB/sec execute 597 sec latency 91.556 ms 2023-07-21T22:41:21.938 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:21 smithi093 ceph-mon[96579]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 481 op/s 2023-07-21T22:41:22.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:21 smithi194 ceph-mon[98288]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 481 op/s 2023-07-21T22:41:22.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:21 smithi134 ceph-mon[97873]: pgmap v795: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 481 op/s 2023-07-21T22:41:22.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 385199 27.67 MB/sec execute 598 sec latency 91.674 ms 2023-07-21T22:41:23.574 INFO:tasks.workunit.client.0.smithi093.stdout: 10 385641 27.67 MB/sec execute 599 sec latency 92.614 ms 2023-07-21T22:41:23.942 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:23 smithi093 ceph-mon[96579]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 471 op/s 2023-07-21T22:41:24.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:23 smithi194 ceph-mon[98288]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 471 op/s 2023-07-21T22:41:24.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:23 smithi134 ceph-mon[97873]: pgmap v796: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 471 op/s 2023-07-21T22:41:24.577 INFO:tasks.workunit.client.0.smithi093.stdout: 10 cleanup 600 sec 2023-07-21T22:41:25.578 INFO:tasks.workunit.client.0.smithi093.stdout: 10 cleanup 601 sec 2023-07-21T22:41:26.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:25 smithi194 ceph-mon[98288]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 375 op/s 2023-07-21T22:41:26.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:25 smithi134 ceph-mon[97873]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 375 op/s 2023-07-21T22:41:26.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:25 smithi093 ceph-mon[96579]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 9.3 MiB/s wr, 375 op/s 2023-07-21T22:41:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.357+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/2057196199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e410e2a0 msgr2=0x7fe4e410e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.357+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/2057196199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e410e2a0 0x7fe4e410e670 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b0a0 tx=0x7fe4dc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/2057196199 shutdown_connections 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/2057196199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e406d3a0 0x7fe4e411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/2057196199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e410e2a0 0x7fe4e410e670 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/2057196199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 0x7fe4e406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/2057196199 >> 172.21.15.93:0/2057196199 conn(0x7fe4e406bd70 msgr2=0x7fe4e4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/2057196199 shutdown_connections 2023-07-21T22:41:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.361+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/2057196199 wait complete. 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.362+0000 7fe4ebf3b700 1 Processor -- start 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.362+0000 7fe4ebf3b700 1 -- start start 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 0x7fe4e41aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e406d3a0 0x7fe4e41ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a5140 con 0x7fe4e406d3a0 2023-07-21T22:41:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a52b0 con 0x7fe4e410e2a0 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ebf3b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41a5470 con 0x7fe4e406c910 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33250/0 (socket says 172.21.15.93:33250) 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 -- 172.21.15.93:0/970950690 learned_addr learned my addr 172.21.15.93:0/970950690 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 0x7fe4e41aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e406d3a0 0x7fe4e41ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 -- 172.21.15.93:0/970950690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 msgr2=0x7fe4e41aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 0x7fe4e41aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 -- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e406d3a0 msgr2=0x7fe4e41ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e406d3a0 0x7fe4e41ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.363+0000 7fe4ea4d8700 1 -- 172.21.15.93:0/970950690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009d00 con 0x7fe4e410e2a0 2023-07-21T22:41:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.364+0000 7fe4ea4d8700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000d6d0 tx=0x7fe4e000d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.364+0000 7fe4daffd700 1 -- 172.21.15.93:0/970950690 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004a30 con 0x7fe4e410e2a0 2023-07-21T22:41:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.364+0000 7fe4daffd700 1 -- 172.21.15.93:0/970950690 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e00054e0 con 0x7fe4e410e2a0 2023-07-21T22:41:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.364+0000 7fe4daffd700 1 -- 172.21.15.93:0/970950690 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000fbf0 con 0x7fe4e410e2a0 2023-07-21T22:41:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.368+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e41a5750 con 0x7fe4e410e2a0 2023-07-21T22:41:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.368+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e41b1b20 con 0x7fe4e410e2a0 2023-07-21T22:41:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.372+0000 7fe4f80c0b80 1 Processor -- start 2023-07-21T22:41:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.374+0000 7fe4daffd700 1 -- 172.21.15.93:0/970950690 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe4e000fd50 con 0x7fe4e410e2a0 2023-07-21T22:41:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.374+0000 7fe4f80c0b80 1 -- start start 2023-07-21T22:41:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.374+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dac0bc50 con 0x5600dab626b0 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dac5e9e0 con 0x5600dad0c660 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dac57e70 con 0x5600dad0c210 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50326/0 (socket says 172.21.15.93:50326) 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 -- 172.21.15.93:0/3649816815 learned_addr learned my addr 172.21.15.93:0/3649816815 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4daffd700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 0x7fe4d00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4ea4d8700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 0x7fe4d00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4daffd700 1 -- 172.21.15.93:0/970950690 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe4e00ccaa0 con 0x7fe4e410e2a0 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 -- 172.21.15.93:0/3649816815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dad0c210 msgr2=0x5600dad05f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 -- 172.21.15.93:0/3649816815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 msgr2=0x5600dad064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 -- 172.21.15.93:0/3649816815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600daa85d80 con 0x5600dab626b0 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.375+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4007f70 tx=0x7fe4d40071a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c549b405be7034f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.376+0000 7fe4ea4d8700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 0x7fe4d00a3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0099870 tx=0x7fe4e006bbe0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.376+0000 7fe4cffff700 1 -- 172.21.15.93:0/3649816815 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4018620 con 0x5600dab626b0 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.376+0000 7fe4cffff700 1 -- 172.21.15.93:0/3649816815 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d4016e30 con 0x5600dab626b0 2023-07-21T22:41:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.376+0000 7fe4cffff700 1 -- 172.21.15.93:0/3649816815 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4017940 con 0x5600dab626b0 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/3649816815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 msgr2=0x5600dad0bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4007f70 tx=0x7fe4d40071a0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/3649816815 shutdown_connections 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dab626b0 0x5600dad0bcd0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/3649816815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/3649816815 >> 172.21.15.93:0/3649816815 conn(0x5600dacfbae0 msgr2=0x5600dad010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/3649816815 shutdown_connections 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/3649816815 wait complete. 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 -- start start 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.377+0000 7fe4f80c0b80 1 Processor -- start 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dabf7a60 con 0x5600dad0c210 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dabf7760 con 0x5600dad0c660 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600daadb8c0 con 0x5600dab626b0 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4f80c0b80 10 client.0 _ll_register_callbacks cb 0x5600dabf5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50334/0 (socket says 172.21.15.93:50334) 2023-07-21T22:41:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 -- 172.21.15.93:0/823905974 learned_addr learned my addr 172.21.15.93:0/823905974 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 -- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 msgr2=0x5600dad125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 -- 172.21.15.93:0/823905974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 msgr2=0x5600dad0ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad0ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.378+0000 7fe4e9cd7700 1 -- 172.21.15.93:0/823905974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d4016310 con 0x5600dad0c210 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b0a0 tx=0x7fe4dc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f7ee9d90c5e61c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00e040 con 0x5600dad0c210 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe4dc007970 con 0x5600dad0c210 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600dabb1490 con 0x5600dad0c210 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4dc005040 con 0x5600dad0c210 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4f80c0b80 10 client.18066 fetch_fsmap learned FSMap version 91 2023-07-21T22:41:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.379+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x5600daa6ec00 con 0x5600dad0c210 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600dabb1490 con 0x5600dad0c210 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe4dc00b0d0 con 0x5600dad0c210 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 10 client.18066 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 20 client.18066 populate_metadata read hostname 'smithi093' 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad1fb80 0x5600dad21f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 4 client.18066 mds_command: new command op to 24484 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:41:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.380+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600dac50e50 con 0x5600dad1fb80 2023-07-21T22:41:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.388+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad1fb80 0x5600dad21f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.390+0000 7fe4cd7fa700 10 client.18066 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.390+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad1fb80 0x5600dad21f90 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd53e3f4e7e9b756 server_cookie=d19eefda6cb18c2b in_seq=0 out_seq=0 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/823905974 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5600dac50e50 con 0x5600dad1fb80 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4cd7fa700 10 client.18066 handle_command_reply: tid=0 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 client.18066 shutdown 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 msgr2=0x5600dad0f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b0a0 tx=0x7fe4dc004ec0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 shutdown_connections 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad1fb80 0x5600dad21f90 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad0ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/823905974 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad0f8a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 >> 172.21.15.93:0/823905974 conn(0x5600dacfbae0 msgr2=0x5600dad003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4cd7fa700 0 client.18066 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.394+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 shutdown_connections 2023-07-21T22:41:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.395+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/823905974 wait complete. 2023-07-21T22:41:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.395+0000 7fe4f80c0b80 20 client.18066 trim_cache size 0 max 16384 2023-07-21T22:41:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.419+0000 7fe4f80c0b80 1 Processor -- start 2023-07-21T22:41:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.419+0000 7fe4f80c0b80 1 -- start start 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.419+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad11850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.419+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.420+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad06480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.420+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dad02510 con 0x5600dad0c210 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.420+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dac50e50 con 0x5600dad0c660 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.420+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dabb1490 con 0x5600dab626b0 2023-07-21T22:41:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.423+0000 7fe4e9cd7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.423+0000 7fe4e9cd7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50340/0 (socket says 172.21.15.93:50340) 2023-07-21T22:41:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.423+0000 7fe4e9cd7700 1 -- 172.21.15.93:0/926579131 learned_addr learned my addr 172.21.15.93:0/926579131 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.423+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad11850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad06480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 -- 172.21.15.93:0/926579131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 msgr2=0x5600dad06480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad06480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 -- 172.21.15.93:0/926579131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 msgr2=0x5600dad05f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 -- 172.21.15.93:0/926579131 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600dad26b80 con 0x5600dab626b0 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.424+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad11850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5600dad22830 tx=0x7fe4d4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c766c9433cc73cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.425+0000 7fe4d8ff9700 1 -- 172.21.15.93:0/926579131 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d402c050 con 0x5600dab626b0 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.425+0000 7fe4d8ff9700 1 -- 172.21.15.93:0/926579131 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d402d040 con 0x5600dab626b0 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.425+0000 7fe4d8ff9700 1 -- 172.21.15.93:0/926579131 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d402fd70 con 0x5600dab626b0 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.426+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/926579131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 msgr2=0x5600dad11850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.426+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad11850 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5600dad22830 tx=0x7fe4d4000bc0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.427+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/926579131 shutdown_connections 2023-07-21T22:41:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.427+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad06480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.427+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad11850 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.427+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/926579131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad05f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.427+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/926579131 >> 172.21.15.93:0/926579131 conn(0x5600dacfbae0 msgr2=0x5600dad11140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.432+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/926579131 shutdown_connections 2023-07-21T22:41:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.432+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/926579131 wait complete. 2023-07-21T22:41:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.433+0000 7fe4f80c0b80 1 -- start start 2023-07-21T22:41:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.433+0000 7fe4f80c0b80 1 Processor -- start 2023-07-21T22:41:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dad83f50 con 0x5600dad0c210 2023-07-21T22:41:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dad84090 con 0x5600dad0c660 2023-07-21T22:41:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600dad841d0 con 0x5600dab626b0 2023-07-21T22:41:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4f80c0b80 10 client.0 _ll_register_callbacks cb 0x7fe4dc004a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:41:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4e94d6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4e94d6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4e94d6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33278/0 (socket says 172.21.15.93:33278) 2023-07-21T22:41:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4e94d6700 1 -- 172.21.15.93:0/515341082 learned_addr learned my addr 172.21.15.93:0/515341082 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:26.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.434+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 -- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 msgr2=0x5600dad14c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad14c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 -- 172.21.15.93:0/515341082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 msgr2=0x5600dad151d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad151d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 -- 172.21.15.93:0/515341082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600daa85d80 con 0x5600dad0c660 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4e94d6700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4032040 tx=0x7fe4d4017480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726d9910a9036de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-21T22:41:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4007ea0 con 0x5600dad0c660 2023-07-21T22:41:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.435+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d4016e30 con 0x5600dad0c660 2023-07-21T22:41:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.438+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mon.2 v2:172.21.15.134:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe4d402f230 con 0x5600dad0c660 2023-07-21T22:41:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.438+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d402c030 con 0x5600dad0c660 2023-07-21T22:41:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.438+0000 7fe4f80c0b80 10 client.27350 fetch_fsmap learned FSMap version 91 2023-07-21T22:41:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.438+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x5600daaac1f0 con 0x5600dad0c660 2023-07-21T22:41:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.438+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mon.2 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d4016e30 con 0x5600dad0c660 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mon.2 v2:172.21.15.134:3300/0 5 ==== fsmap(e 91) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe4d40177a0 con 0x5600dad0c660 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 10 client.27350 fetch_fsmap finished waiting for FSMap version 91 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 10 client.27350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 20 client.27350 populate_metadata read hostname 'smithi093' 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad85550 0x5600dad87930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 4 client.27350 mds_command: new command op to 24484 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:41:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.439+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 --> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600dac12ff0 con 0x5600dad85550 2023-07-21T22:41:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.440+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad85550 0x5600dad87930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.441+0000 7fe4cd7fa700 10 client.27350 ms_handle_connect on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.441+0000 7fe4e9cd7700 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad85550 0x5600dad87930 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e12cf8eb1cd356 server_cookie=e00bfea166c7b57d in_seq=0 out_seq=0 2023-07-21T22:41:26.579 INFO:tasks.workunit.client.0.smithi093.stdout: 5 cleanup 602 sec 2023-07-21T22:41:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.966+0000 7fe4cd7fa700 1 -- 172.21.15.93:0/515341082 <== mds.0 v2:172.21.15.194:6838/1819166614 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600dac12ff0 con 0x5600dad85550 2023-07-21T22:41:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.966+0000 7fe4cd7fa700 10 client.27350 handle_command_reply: tid=0 2023-07-21T22:41:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.966+0000 7fe4f80c0b80 1 client.27350 shutdown 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 msgr2=0x5600dad83a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4032040 tx=0x7fe4d4017480 comp rx=0 tx=0).stop 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 shutdown_connections 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:6838/1819166614,v1:172.21.15.194:6839/1819166614] conn(0x5600dad85550 0x5600dad87930 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600dad0c660 0x5600dad83a10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5600dab626b0 0x5600dad14c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 --2- 172.21.15.93:0/515341082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5600dad0c210 0x5600dad151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 >> 172.21.15.93:0/515341082 conn(0x5600dacfbae0 msgr2=0x5600dad0d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4cd7fa700 0 client.27350 ms_handle_reset on v2:172.21.15.194:6838/1819166614 2023-07-21T22:41:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 shutdown_connections 2023-07-21T22:41:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.975+0000 7fe4f80c0b80 1 -- 172.21.15.93:0/515341082 wait complete. 2023-07-21T22:41:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.976+0000 7fe4f80c0b80 20 client.27350 trim_cache size 0 max 16384 2023-07-21T22:41:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-07-21T22:41:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-07-21T22:41:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-07-21T22:41:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-07-21T22:41:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.976+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 msgr2=0x7fe4d00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.976+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 0x7fe4d00a3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0099870 tx=0x7fe4e006bbe0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.976+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 msgr2=0x7fe4e41a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.976+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000d6d0 tx=0x7fe4e000d9e0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 shutdown_connections 2023-07-21T22:41:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e410e2a0 0x7fe4e41a4970 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4e406c910 0x7fe4e41aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe4d00a1380 0x7fe4d00a3830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 --2- 172.21.15.93:0/970950690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4e406d3a0 0x7fe4e41ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.980+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 >> 172.21.15.93:0/970950690 conn(0x7fe4e406bd70 msgr2=0x7fe4e40707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.983+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 shutdown_connections 2023-07-21T22:41:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-07-21T22:41:26.983+0000 7fe4ebf3b700 1 -- 172.21.15.93:0/970950690 wait complete. 2023-07-21T22:41:27.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:26 smithi134 ceph-mon[97873]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 530 op/s 2023-07-21T22:41:27.188 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:26 smithi093 ceph-mon[96579]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 530 op/s 2023-07-21T22:41:27.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:26 smithi194 ceph-mon[98288]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 14 MiB/s wr, 530 op/s 2023-07-21T22:41:27.410 INFO:tasks.workunit.client.0.smithi093.stdout: 0 cleanup 603 sec 2023-07-21T22:41:27.410 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:41:27.410 INFO:tasks.workunit.client.0.smithi093.stdout: Operation Count AvgLat MaxLat 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: ---------------------------------------- 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: NTCreateX 532145 4.386 7467.064 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: Close 390770 0.002 0.176 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: Rename 22556 19.984 3837.234 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: Unlink 107592 5.236 10879.395 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: Qpathinfo 482750 3.747 8479.069 2023-07-21T22:41:27.411 INFO:tasks.workunit.client.0.smithi093.stdout: Qfileinfo 84163 0.001 0.172 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: Qfsinfo 88487 0.538 66.287 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: Sfileinfo 43413 0.168 181.188 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: Find 186546 0.522 9190.224 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: WriteX 262958 0.846 1246.369 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: ReadX 834843 0.040 1308.692 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: LockX 1736 2.515 45.139 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: UnlockX 1736 2.089 61.674 2023-07-21T22:41:27.412 INFO:tasks.workunit.client.0.smithi093.stdout: Flush 37338 11.405 1260.545 2023-07-21T22:41:27.413 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-07-21T22:41:27.413 INFO:tasks.workunit.client.0.smithi093.stdout:Throughput 27.6716 MB/sec 10 clients 10 procs max_latency=10879.401 ms 2023-07-21T22:41:27.421 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-21T22:41:27.421 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-21T22:41:29.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:29 smithi194 ceph-mon[98288]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 530 op/s 2023-07-21T22:41:29.847 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:29 smithi093 ceph-mon[96579]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 530 op/s 2023-07-21T22:41:29.848 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-07-21T22:41:29.849 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-21T22:41:29.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:29 smithi134 ceph-mon[97873]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 16 MiB/s wr, 530 op/s 2023-07-21T22:41:30.201 DEBUG:teuthology.parallel:result is None 2023-07-21T22:41:30.202 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T22:41:30.247 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T22:41:30.248 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-21T22:41:30.258 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:41:30.266 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-21T22:41:30.275 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-21T22:41:30.275 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T22:41:30.276 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:41:30.276 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-21T22:41:30.375 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-21T22:41:30.405 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T22:41:30.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.527+0000 7fa35cabb700 1 -- 172.21.15.93:0/1599066789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3581004e0 msgr2=0x7fa35810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.527+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1599066789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3581004e0 0x7fa35810c920 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b0a0 tx=0x7fa35000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 -- 172.21.15.93:0/1599066789 shutdown_connections 2023-07-21T22:41:30.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1599066789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3581004e0 0x7fa35810c920 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1599066789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa358105b60 0x7fa358105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1599066789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3580ffb30 0x7fa3580fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 -- 172.21.15.93:0/1599066789 >> 172.21.15.93:0/1599066789 conn(0x7fa3580fb5f0 msgr2=0x7fa3580fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:30.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 -- 172.21.15.93:0/1599066789 shutdown_connections 2023-07-21T22:41:30.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 -- 172.21.15.93:0/1599066789 wait complete. 2023-07-21T22:41:30.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 Processor -- start 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.528+0000 7fa35cabb700 1 -- start start 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ffb30 0x7fa3580729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3581004e0 0x7fa358072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a42e0 con 0x7fa358105b60 2023-07-21T22:41:30.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a4420 con 0x7fa3581004e0 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a4590 con 0x7fa3580ffb30 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50364/0 (socket says 172.21.15.93:50364) 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 -- 172.21.15.93:0/1054098857 learned_addr learned my addr 172.21.15.93:0/1054098857 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35659c700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ffb30 0x7fa3580729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa355d9b700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3581004e0 0x7fa358072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 -- 172.21.15.93:0/1054098857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ffb30 msgr2=0x7fa3580729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ffb30 0x7fa3580729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 -- 172.21.15.93:0/1054098857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3581004e0 msgr2=0x7fa358072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3581004e0 0x7fa358072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 -- 172.21.15.93:0/1054098857 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350009d00 con 0x7fa358105b60 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa356d9d700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b830 tx=0x7fa3500079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000e040 con 0x7fa358105b60 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3581a4810 con 0x7fa358105b60 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa350004d60 con 0x7fa358105b60 2023-07-21T22:41:30.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35001d480 con 0x7fa358105b60 2023-07-21T22:41:30.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.529+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a4c90 con 0x7fa358105b60 2023-07-21T22:41:30.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.530+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3580ff250 con 0x7fa358105b60 2023-07-21T22:41:30.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.532+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa350003a00 con 0x7fa358105b60 2023-07-21T22:41:30.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.532+0000 7fa3477fe700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 0x7fa3400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.532+0000 7fa35659c700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 0x7fa3400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.533+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa3500cea70 con 0x7fa358105b60 2023-07-21T22:41:30.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.533+0000 7fa35659c700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 0x7fa3400a3770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa348007be0 tx=0x7fa3480073d0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:30.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.536+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa350097f10 con 0x7fa358105b60 2023-07-21T22:41:30.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.752+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fa358106fe0 con 0x7fa3400a12c0 2023-07-21T22:41:30.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.767+0000 7fa3477fe700 1 -- 172.21.15.93:0/1054098857 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa358106fe0 con 0x7fa3400a12c0 2023-07-21T22:41:30.808 INFO:teuthology.orchestra.run.smithi093.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T22:16:26", "first": "2023-07-21T22:17:00", "last": "2023-07-21T22:41:00", "last_pruned": "2023-07-21T22:41:00", "created_count": 25, "pruned_count": 19, "active": true} 2023-07-21T22:41:30.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 msgr2=0x7fa3400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 0x7fa3400a3770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa348007be0 tx=0x7fa3480073d0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 msgr2=0x7fa3581a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa35000b830 tx=0x7fa3500079b0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 shutdown_connections 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa3400a12c0 0x7fa3400a3770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa358105b60 0x7fa3581a3da0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3580ffb30 0x7fa3580729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 --2- 172.21.15.93:0/1054098857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3581004e0 0x7fa358072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 >> 172.21.15.93:0/1054098857 conn(0x7fa3580fb5f0 msgr2=0x7fa3581068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:30.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 shutdown_connections 2023-07-21T22:41:30.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.769+0000 7fa35cabb700 1 -- 172.21.15.93:0/1054098857 wait complete. 2023-07-21T22:41:30.821 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:30 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:30.822 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T22:41:30.836 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:30 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:30.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:30 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:30.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 -- 172.21.15.93:0/2624579134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0101610 msgr2=0x7f5df0109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 --2- 172.21.15.93:0/2624579134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0101610 0x7f5df0109c50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009990 tx=0x7f5de4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 -- 172.21.15.93:0/2624579134 shutdown_connections 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 --2- 172.21.15.93:0/2624579134 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 0x7f5df010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 --2- 172.21.15.93:0/2624579134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0100c70 0x7f5df0101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 --2- 172.21.15.93:0/2624579134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0101610 0x7f5df0109c50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 -- 172.21.15.93:0/2624579134 >> 172.21.15.93:0/2624579134 conn(0x7f5df0075bd0 msgr2=0x7f5df0075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:30.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 -- 172.21.15.93:0/2624579134 shutdown_connections 2023-07-21T22:41:30.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.903+0000 7f5df65b2700 1 -- 172.21.15.93:0/2624579134 wait complete. 2023-07-21T22:41:30.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 Processor -- start 2023-07-21T22:41:30.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 -- start start 2023-07-21T22:41:30.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0101610 0x7f5df019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 0x7f5df0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0196130 con 0x7f5df0100c70 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01962a0 con 0x7f5df0101610 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5df65b2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0196460 con 0x7f5df010a190 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.904+0000 7f5deffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50374/0 (socket says 172.21.15.93:50374) 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 -- 172.21.15.93:0/1275512478 learned_addr learned my addr 172.21.15.93:0/1275512478 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5df4b4f700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 0x7f5df0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5def7fe700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0101610 0x7f5df019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 -- 172.21.15.93:0/1275512478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 msgr2=0x7f5df0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 0x7f5df0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 -- 172.21.15.93:0/1275512478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0101610 msgr2=0x7f5df019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0101610 0x7f5df019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:30.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 -- 172.21.15.93:0/1275512478 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40095f0 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5deffff700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e9a0 tx=0x7f5de000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0009790 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de000cbd0 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df0196740 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0018680 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.905+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df01a2c60 con 0x7f5df0100c70 2023-07-21T22:41:30.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.906+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df00fe360 con 0x7f5df0100c70 2023-07-21T22:41:30.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.907+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5de000cd40 con 0x7f5df0100c70 2023-07-21T22:41:30.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.908+0000 7f5ded7fa700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 0x7f5dd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:30.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.908+0000 7f5def7fe700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 0x7f5dd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:30.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.908+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5de0014070 con 0x7f5df0100c70 2023-07-21T22:41:30.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.908+0000 7f5def7fe700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 0x7f5dd80a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5de4006010 tx=0x7f5de400b260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:30.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:30.911+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5de0096920 con 0x7f5df0100c70 2023-07-21T22:41:31.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.124+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 --> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5df0197260 con 0x7f5dd80a12c0 2023-07-21T22:41:31.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.139+0000 7f5ded7fa700 1 -- 172.21.15.93:0/1275512478 <== mgr.25079 v2:172.21.15.93:6800/904524118 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f5df0197260 con 0x7f5dd80a12c0 2023-07-21T22:41:31.180 INFO:teuthology.orchestra.run.smithi093.stdout:/ 1M 6M3h 2023-07-21T22:41:31.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 msgr2=0x7f5dd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 0x7f5dd80a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5de4006010 tx=0x7f5de400b260 comp rx=0 tx=0).stop 2023-07-21T22:41:31.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 msgr2=0x7f5df019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e9a0 tx=0x7f5de000ed60 comp rx=0 tx=0).stop 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 shutdown_connections 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5df010a190 0x7f5df0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f5dd80a12c0 0x7f5dd80a3770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5df0100c70 0x7f5df019bc70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 --2- 172.21.15.93:0/1275512478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5df0101610 0x7f5df019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 >> 172.21.15.93:0/1275512478 conn(0x7f5df0075bd0 msgr2=0x7f5df010ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:31.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 shutdown_connections 2023-07-21T22:41:31.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.142+0000 7f5df65b2700 1 -- 172.21.15.93:0/1275512478 wait complete. 2023-07-21T22:41:31.194 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-21T22:41:31.209 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-21T22:41:31.210 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-21T22:41:31.210 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-21T22:41:31.227 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:41:31.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/2897757024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 msgr2=0x7fcd48104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/2897757024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48104db0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009990 tx=0x7fcd3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/2897757024 shutdown_connections 2023-07-21T22:41:31.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/2897757024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 0x7fcd4810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/2897757024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48104db0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/2897757024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 0x7fcd480fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/2897757024 >> 172.21.15.93:0/2897757024 conn(0x7fcd480fa490 msgr2=0x7fcd480fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/2897757024 shutdown_connections 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/2897757024 wait complete. 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 Processor -- start 2023-07-21T22:41:31.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.309+0000 7fcd4ee0e700 1 -- start start 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 0x7fcd4819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 0x7fcd4819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48198a00 con 0x7fcd481049e0 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48198b70 con 0x7fcd480fe9d0 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48198d30 con 0x7fcd480ff380 2023-07-21T22:41:31.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50392/0 (socket says 172.21.15.93:50392) 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 -- 172.21.15.93:0/3237867042 learned_addr learned my addr 172.21.15.93:0/3237867042 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd47fff700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 0x7fcd4819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4cbaa700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 0x7fcd4819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 -- 172.21.15.93:0/3237867042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 msgr2=0x7fcd4819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 0x7fcd4819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 -- 172.21.15.93:0/3237867042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 msgr2=0x7fcd4819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 0x7fcd4819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 -- 172.21.15.93:0/3237867042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0095f0 con 0x7fcd481049e0 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4d3ab700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcd40007ee0 tx=0x7fcd4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40017070 con 0x7fcd481049e0 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4000f040 con 0x7fcd481049e0 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd48199010 con 0x7fcd481049e0 2023-07-21T22:41:31.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.310+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40005580 con 0x7fcd481049e0 2023-07-21T22:41:31.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.311+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd481a4ad0 con 0x7fcd481049e0 2023-07-21T22:41:31.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.312+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd48067300 con 0x7fcd481049e0 2023-07-21T22:41:31.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.312+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcd4000db20 con 0x7fcd481049e0 2023-07-21T22:41:31.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.313+0000 7fcd45ffb700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 0x7fcd380a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.313+0000 7fcd4cbaa700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 0x7fcd380a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.313+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fcd400cc3e0 con 0x7fcd481049e0 2023-07-21T22:41:31.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.313+0000 7fcd4cbaa700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 0x7fcd380a37b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b5c0 tx=0x7fcd3c005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:31.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.317+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcd400957f0 con 0x7fcd481049e0 2023-07-21T22:41:31.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.512+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd481a4ea0 con 0x7fcd481049e0 2023-07-21T22:41:31.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.514+0000 7fcd45ffb700 1 -- 172.21.15.93:0/3237867042 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+22948 (secure 0 0 0) 0x7fcd40098e10 con 0x7fcd481049e0 2023-07-21T22:41:31.555 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:41:31.555 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":130,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:41:17.425830+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/2266779369":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1882208449":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/503504133":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6801/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3365813988":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2153177670":"2023-07-22T22:16:25.946546+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:41:31.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 msgr2=0x7fcd380a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 0x7fcd380a37b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b5c0 tx=0x7fcd3c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 msgr2=0x7fcd48198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcd40007ee0 tx=0x7fcd4000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 shutdown_connections 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fcd380a1300 0x7fcd380a37b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd481049e0 0x7fcd48198260 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd480fe9d0 0x7fcd4819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 --2- 172.21.15.93:0/3237867042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd480ff380 0x7fcd4819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 >> 172.21.15.93:0/3237867042 conn(0x7fcd480fa490 msgr2=0x7fcd480fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 shutdown_connections 2023-07-21T22:41:31.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.516+0000 7fcd4ee0e700 1 -- 172.21.15.93:0/3237867042 wait complete. 2023-07-21T22:41:31.570 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:41:31.570 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:41:31.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 -- 172.21.15.93:0/2390968555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 msgr2=0x7fa30410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 --2- 172.21.15.93:0/2390968555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa30410c9c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00b0a0 tx=0x7fa2fc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 -- 172.21.15.93:0/2390968555 shutdown_connections 2023-07-21T22:41:31.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 --2- 172.21.15.93:0/2390968555 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa30410c9c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 --2- 172.21.15.93:0/2390968555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 0x7fa304105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 --2- 172.21.15.93:0/2390968555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 0x7fa3040fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 -- 172.21.15.93:0/2390968555 >> 172.21.15.93:0/2390968555 conn(0x7fa304076200 msgr2=0x7fa304076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:31.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 -- 172.21.15.93:0/2390968555 shutdown_connections 2023-07-21T22:41:31.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.655+0000 7fa308f65700 1 -- 172.21.15.93:0/2390968555 wait complete. 2023-07-21T22:41:31.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 Processor -- start 2023-07-21T22:41:31.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 -- start start 2023-07-21T22:41:31.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 0x7fa304195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 0x7fa30419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419ad70 con 0x7fa3040feb90 2023-07-21T22:41:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419aee0 con 0x7fa3040fe200 2023-07-21T22:41:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa308f65700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419b0a0 con 0x7fa304105c00 2023-07-21T22:41:31.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50406/0 (socket says 172.21.15.93:50406) 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 -- 172.21.15.93:0/101596346 learned_addr learned my addr 172.21.15.93:0/101596346 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa302d9d700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 0x7fa30419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa30259c700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 0x7fa304195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 -- 172.21.15.93:0/101596346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 msgr2=0x7fa30419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 0x7fa30419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 -- 172.21.15.93:0/101596346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 msgr2=0x7fa304195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 0x7fa304195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 -- 172.21.15.93:0/101596346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc009d00 con 0x7fa3040feb90 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.656+0000 7fa301d9b700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400d950 tx=0x7fa2f400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.657+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4009b40 con 0x7fa3040feb90 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.657+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2f4009ca0 con 0x7fa3040feb90 2023-07-21T22:41:31.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.657+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30419b380 con 0x7fa3040feb90 2023-07-21T22:41:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.657+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400f620 con 0x7fa3040feb90 2023-07-21T22:41:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.657+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041a2bc0 con 0x7fa3040feb90 2023-07-21T22:41:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.658+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa304066a00 con 0x7fa3040feb90 2023-07-21T22:41:31.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.659+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa2f4010460 con 0x7fa3040feb90 2023-07-21T22:41:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.659+0000 7fa2f37fe700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 0x7fa2ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.659+0000 7fa30259c700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 0x7fa2ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.659+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa2f40cd0a0 con 0x7fa3040feb90 2023-07-21T22:41:31.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.660+0000 7fa30259c700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 0x7fa2ec0a3770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005fd0 tx=0x7fa2f8005ee0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:31.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.663+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa2f40964b0 con 0x7fa3040feb90 2023-07-21T22:41:31.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:31 smithi194 ceph-mon[98288]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 849 op/s 2023-07-21T22:41:31.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:31 smithi134 ceph-mon[97873]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 849 op/s 2023-07-21T22:41:31.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.852+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3041a2e50 con 0x7fa3040feb90 2023-07-21T22:41:31.893 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.852+0000 7fa2f37fe700 1 -- 172.21.15.93:0/101596346 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2f4016020 con 0x7fa3040feb90 2023-07-21T22:41:31.894 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:41:31.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.854+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 msgr2=0x7fa2ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.854+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 0x7fa2ec0a3770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005fd0 tx=0x7fa2f8005ee0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.854+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 msgr2=0x7fa304196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:31.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.854+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400d950 tx=0x7fa2f400dd10 comp rx=0 tx=0).stop 2023-07-21T22:41:31.896 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 shutdown_connections 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa304105c00 0x7fa30419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3040fe200 0x7fa304195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fa2ec0a12c0 0x7fa2ec0a3770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 --2- 172.21.15.93:0/101596346 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3040feb90 0x7fa304196340 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 >> 172.21.15.93:0/101596346 conn(0x7fa304076200 msgr2=0x7fa3040fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:31.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 shutdown_connections 2023-07-21T22:41:31.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.855+0000 7fa308f65700 1 -- 172.21.15.93:0/101596346 wait complete. 2023-07-21T22:41:31.906 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:31 smithi093 ceph-mon[96579]: pgmap v800: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 849 op/s 2023-07-21T22:41:31.908 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:41:32.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 -- 172.21.15.93:0/2459167116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01580ff380 msgr2=0x7f015810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 --2- 172.21.15.93:0/2459167116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01580ff380 0x7f015810b810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f015000b0a0 tx=0x7f015000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 -- 172.21.15.93:0/2459167116 shutdown_connections 2023-07-21T22:41:32.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 --2- 172.21.15.93:0/2459167116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01580ff380 0x7f015810b810 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 --2- 172.21.15.93:0/2459167116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01581049e0 0x7f0158104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 --2- 172.21.15.93:0/2459167116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f01580fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 -- 172.21.15.93:0/2459167116 >> 172.21.15.93:0/2459167116 conn(0x7f01580fa4b0 msgr2=0x7f01580fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 -- 172.21.15.93:0/2459167116 shutdown_connections 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.992+0000 7f016002d700 1 -- 172.21.15.93:0/2459167116 wait complete. 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.993+0000 7f016002d700 1 Processor -- start 2023-07-21T22:41:32.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.993+0000 7f016002d700 1 -- start start 2023-07-21T22:41:32.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f015819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 0x7f015819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01581950a0 con 0x7f01581049e0 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158195210 con 0x7f01580fe9d0 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01581953d0 con 0x7f01580ff380 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50420/0 (socket says 172.21.15.93:50420) 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 -- 172.21.15.93:0/1610171833 learned_addr learned my addr 172.21.15.93:0/1610171833 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015ddc9700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f015819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015d5c8700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 0x7f015819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 -- 172.21.15.93:0/1610171833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 msgr2=0x7f015819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 0x7f015819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 -- 172.21.15.93:0/1610171833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 msgr2=0x7f015819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f015819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 -- 172.21.15.93:0/1610171833 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009d00 con 0x7f01581049e0 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015d5c8700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 0x7f015819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015ddc9700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f015819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f015e5ca700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0150009fd0 tx=0x7f015000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015000e040 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0158195650 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0150004020 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015001b690 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.994+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01581a19b0 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.995+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158067290 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.996+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0150004190 con 0x7f01581049e0 2023-07-21T22:41:32.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.997+0000 7f014effd700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 0x7f01440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.997+0000 7f015ddc9700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 0x7f01440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.997+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f01500ce3d0 con 0x7f01581049e0 2023-07-21T22:41:32.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:31.997+0000 7f015ddc9700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 0x7f01440a3770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0148005e10 tx=0x7f0148005d20 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.001+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0150097810 con 0x7f01581049e0 2023-07-21T22:41:32.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.191+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01581a1c00 con 0x7f01581049e0 2023-07-21T22:41:32.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.191+0000 7f014effd700 1 -- 172.21.15.93:0/1610171833 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f015009ae30 con 0x7f01581049e0 2023-07-21T22:41:32.232 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:41:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 msgr2=0x7f01440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 0x7f01440a3770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0148005e10 tx=0x7f0148005d20 comp rx=0 tx=0).stop 2023-07-21T22:41:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 msgr2=0x7f0158194a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0150009fd0 tx=0x7f015000cf40 comp rx=0 tx=0).stop 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 shutdown_connections 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f01440a12c0 0x7f01440a3770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01581049e0 0x7f0158194a30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01580fe9d0 0x7f015819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 --2- 172.21.15.93:0/1610171833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01580ff380 0x7f015819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 >> 172.21.15.93:0/1610171833 conn(0x7f01580fa4b0 msgr2=0x7f0158105830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:32.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 shutdown_connections 2023-07-21T22:41:32.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.193+0000 7f016002d700 1 -- 172.21.15.93:0/1610171833 wait complete. 2023-07-21T22:41:32.246 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:41:32.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 -- 172.21.15.93:0/3634270216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33380ffe00 msgr2=0x7f33381001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 --2- 172.21.15.93:0/3634270216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33380ffe00 0x7f33381001d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f33280099f0 tx=0x7f3328009d00 comp rx=0 tx=0).stop 2023-07-21T22:41:32.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 -- 172.21.15.93:0/3634270216 shutdown_connections 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 --2- 172.21.15.93:0/3634270216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 0x7f333810b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 --2- 172.21.15.93:0/3634270216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33380ffe00 0x7f33381001d0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 --2- 172.21.15.93:0/3634270216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3338100710 0x7f3338108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 -- 172.21.15.93:0/3634270216 >> 172.21.15.93:0/3634270216 conn(0x7f33380742e0 msgr2=0x7f33380746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 -- 172.21.15.93:0/3634270216 shutdown_connections 2023-07-21T22:41:32.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.329+0000 7f333d78f700 1 -- 172.21.15.93:0/3634270216 wait complete. 2023-07-21T22:41:32.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.330+0000 7f333d78f700 1 Processor -- start 2023-07-21T22:41:32.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.330+0000 7f333d78f700 1 -- start start 2023-07-21T22:41:32.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33380ffe00 0x7f333819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 0x7f3338194ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338195120 con 0x7f3338100710 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338195290 con 0x7f33380ffe00 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338195450 con 0x7f33381093b0 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50432/0 (socket says 172.21.15.93:50432) 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 -- 172.21.15.93:0/4280541276 learned_addr learned my addr 172.21.15.93:0/4280541276 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f3336ffd700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33380ffe00 0x7f333819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33377fe700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 0x7f3338194ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 -- 172.21.15.93:0/4280541276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 msgr2=0x7f3338194ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 0x7f3338194ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 -- 172.21.15.93:0/4280541276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33380ffe00 msgr2=0x7f333819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.375 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33380ffe00 0x7f333819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 -- 172.21.15.93:0/4280541276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33280095f0 con 0x7f3338100710 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f33367fc700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b530 tx=0x7f332c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c011830 con 0x7f3338100710 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338195730 con 0x7f3338100710 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f332c011e50 con 0x7f3338100710 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00f2e0 con 0x7f3338100710 2023-07-21T22:41:32.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.331+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33381a1a30 con 0x7f3338100710 2023-07-21T22:41:32.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.332+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33380676e0 con 0x7f3338100710 2023-07-21T22:41:32.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.334+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f332c010380 con 0x7f3338100710 2023-07-21T22:41:32.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.334+0000 7f331ffff700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 0x7f33200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.334+0000 7f3336ffd700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 0x7f33200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.334+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f332c0ccce0 con 0x7f3338100710 2023-07-21T22:41:32.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.335+0000 7f3336ffd700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 0x7f33200a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3328005f10 tx=0x7f3328005ea0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.338+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f332c096200 con 0x7f3338100710 2023-07-21T22:41:32.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.537+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33381a1c80 con 0x7f3338100710 2023-07-21T22:41:32.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.537+0000 7f331ffff700 1 -- 172.21.15.93:0/4280541276 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7f332c0150a0 con 0x7f3338100710 2023-07-21T22:41:32.578 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:41:32.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.539+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 msgr2=0x7f33200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.580 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.539+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 0x7f33200a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3328005f10 tx=0x7f3328005ea0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.539+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 msgr2=0x7f333819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f332c00b530 tx=0x7f332c00b840 comp rx=0 tx=0).stop 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 shutdown_connections 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33381093b0 0x7f3338194ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33380ffe00 0x7f333819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f33200a12c0 0x7f33200a3770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 --2- 172.21.15.93:0/4280541276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3338100710 0x7f333819af80 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 >> 172.21.15.93:0/4280541276 conn(0x7f33380742e0 msgr2=0x7f33380fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:32.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 shutdown_connections 2023-07-21T22:41:32.582 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.540+0000 7f333d78f700 1 -- 172.21.15.93:0/4280541276 wait complete. 2023-07-21T22:41:32.591 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:32 smithi093 ceph-mon[96579]: from='client.18078 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:32 smithi093 ceph-mon[96579]: from='client.18084 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:32 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3237867042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:41:32.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:32 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/101596346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.592 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:32 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1610171833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.593 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-21T22:41:32.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- 172.21.15.93:0/450108164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 msgr2=0x7fe59c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 --2- 172.21.15.93:0/450108164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c101070 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009990 tx=0x7fe58c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- 172.21.15.93:0/450108164 shutdown_connections 2023-07-21T22:41:32.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 --2- 172.21.15.93:0/450108164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 0x7fe59c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 --2- 172.21.15.93:0/450108164 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c101070 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 --2- 172.21.15.93:0/450108164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 0x7fe59c109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- 172.21.15.93:0/450108164 >> 172.21.15.93:0/450108164 conn(0x7fe59c0fa490 msgr2=0x7fe59c0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- 172.21.15.93:0/450108164 shutdown_connections 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- 172.21.15.93:0/450108164 wait complete. 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 Processor -- start 2023-07-21T22:41:32.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 -- start start 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.674+0000 7fe5a348f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 0x7fe59c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 0x7fe59c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c196020 con 0x7fe59c100ca0 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c196190 con 0x7fe59c10a070 2023-07-21T22:41:32.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c196350 con 0x7fe59c1015b0 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50444/0 (socket says 172.21.15.93:50444) 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 -- 172.21.15.93:0/3277552968 learned_addr learned my addr 172.21.15.93:0/3277552968 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a1a2c700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 0x7fe59c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a0a2a700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 0x7fe59c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 -- 172.21.15.93:0/3277552968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 msgr2=0x7fe59c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 0x7fe59c19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 -- 172.21.15.93:0/3277552968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 msgr2=0x7fe59c1959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 0x7fe59c1959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 -- 172.21.15.93:0/3277552968 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c0095f0 con 0x7fe59c100ca0 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a122b700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c00f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c01d070 con 0x7fe59c100ca0 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58c00fce0 con 0x7fe59c100ca0 2023-07-21T22:41:32.720 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c1965d0 con 0x7fe59c100ca0 2023-07-21T22:41:32.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c017770 con 0x7fe59c100ca0 2023-07-21T22:41:32.721 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.675+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c1a2930 con 0x7fe59c100ca0 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.676+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c104900 con 0x7fe59c100ca0 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.677+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe58c0178d0 con 0x7fe59c100ca0 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.678+0000 7fe5927fc700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 0x7fe5880a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.678+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe58c0ce630 con 0x7fe59c100ca0 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.678+0000 7fe5a0a2a700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 0x7fe5880a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:41:32.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.678+0000 7fe5a0a2a700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 0x7fe5880a3830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe598006fd0 tx=0x7fe598008040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:41:32.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.681+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe58c097a70 con 0x7fe59c100ca0 2023-07-21T22:41:32.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:32 smithi194 ceph-mon[98288]: from='client.18078 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:32 smithi194 ceph-mon[98288]: from='client.18084 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:32 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3237867042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:41:32.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:32 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/101596346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:32 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1610171833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:32 smithi134 ceph-mon[97873]: from='client.18078 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:32 smithi134 ceph-mon[97873]: from='client.18084 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:41:32.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:32 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3237867042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:41:32.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:32 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/101596346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:32 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1610171833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:41:32.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.930+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fe59c067300 con 0x7fe59c100ca0 2023-07-21T22:41:32.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.937+0000 7fe5927fc700 1 -- 172.21.15.93:0/3277552968 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+733772 (secure 0 0 0) 0x7fe58c09b090 con 0x7fe59c100ca0 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout: "epoch": 24, 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout: "active_gid": 25079, 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout: "active_name": "x", 2023-07-21T22:41:32.985 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addrs": { 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6800", 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 904524118 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:32.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6801", 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 904524118 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addr": "172.21.15.93:6801/904524118", 2023-07-21T22:41:32.987 INFO:teuthology.orchestra.run.smithi093.stdout: "active_change": "2023-07-21T22:16:25.946751+0000", 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "active_mgr_features": 4540138322906710015, 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "available": true, 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "standbys": [ 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 24982, 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "z", 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:41:32.988 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:32.989 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:32.990 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:32.991 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:32.992 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:32.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:32.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:32.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:32.996 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:32.997 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:32.998 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:32.999 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.000 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.001 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.002 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.005 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.006 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-07-21T22:41:33.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.008 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.010 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.012 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.013 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.014 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.015 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.016 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.017 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-07-21T22:41:33.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:41:33.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.021 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.022 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.023 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.024 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-07-21T22:41:33.025 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.026 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.027 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-07-21T22:41:33.028 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:41:33.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.030 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.031 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.035 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.036 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.038 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:41:33.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-07-21T22:41:33.042 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-07-21T22:41:33.043 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:41:33.044 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.045 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.046 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.049 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-07-21T22:41:33.051 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.052 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.053 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.054 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.055 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.056 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.057 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.058 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.059 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-07-21T22:41:33.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.061 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-07-21T22:41:33.062 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:41:33.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.067 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.068 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-07-21T22:41:33.069 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.070 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.071 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:41:33.075 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.076 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.077 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-07-21T22:41:33.078 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.079 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:41:33.080 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:41:33.081 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.082 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.083 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:41:33.084 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.085 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-07-21T22:41:33.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.088 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-07-21T22:41:33.089 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.090 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.091 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.092 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.093 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.094 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.095 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.096 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.097 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:41:33.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.099 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:41:33.100 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:41:33.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.102 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.103 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.104 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.105 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.106 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-07-21T22:41:33.107 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.108 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-07-21T22:41:33.109 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:41:33.110 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.111 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.114 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.115 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.116 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.117 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.118 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.120 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.121 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.123 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.124 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-07-21T22:41:33.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.126 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-07-21T22:41:33.127 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:41:33.128 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.129 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-07-21T22:41:33.131 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:41:33.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.133 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.134 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.135 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.136 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.137 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-07-21T22:41:33.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.139 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.140 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:41:33.141 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.142 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.143 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.144 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.145 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.146 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-07-21T22:41:33.147 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.148 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-07-21T22:41:33.149 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:41:33.150 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.151 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.152 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.153 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.154 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.155 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-07-21T22:41:33.156 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.157 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.158 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.160 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.161 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.162 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.163 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.164 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.165 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.166 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.167 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.168 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.169 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.170 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.171 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.172 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.173 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:41:33.174 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.177 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.178 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.181 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.182 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:41:33.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.184 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.185 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.186 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.187 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.188 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.189 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:41:33.190 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:41:33.192 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.198 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:41:33.199 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.200 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:41:33.201 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.205 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.206 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.207 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:41:33.208 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.209 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.213 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.214 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.215 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.216 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:41:33.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.218 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.219 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.222 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.223 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.224 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.225 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:41:33.226 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.227 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.228 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.231 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.232 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:41:33.233 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.234 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:41:33.235 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.239 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.240 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.241 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.242 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.243 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:41:33.244 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.245 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.246 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.247 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.248 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.249 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.250 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-07-21T22:41:33.251 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.252 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-07-21T22:41:33.253 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.254 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.255 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.256 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-07-21T22:41:33.257 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.258 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.259 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.260 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.261 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.262 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.263 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.264 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.265 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.266 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.267 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.268 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-07-21T22:41:33.269 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:41:33.271 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.272 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.273 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.274 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-07-21T22:41:33.275 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.276 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.277 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-07-21T22:41:33.278 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.279 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-07-21T22:41:33.281 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:33.282 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.283 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.284 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.285 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.286 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.287 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.288 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.289 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.290 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.291 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.292 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.293 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.294 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.295 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.296 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.297 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.298 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-07-21T22:41:33.299 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.300 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.301 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.302 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.303 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.304 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.305 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.306 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.307 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-07-21T22:41:33.308 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:41:33.309 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.310 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.311 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.312 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.313 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.314 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.315 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.316 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.317 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.318 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.319 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.320 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.321 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.322 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.323 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.324 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.325 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-07-21T22:41:33.326 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:41:33.327 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.328 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.329 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.330 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.331 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-07-21T22:41:33.332 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.333 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.334 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.335 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.336 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.337 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.338 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.339 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.340 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.341 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.342 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.343 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-07-21T22:41:33.344 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.345 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:33.346 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:33.347 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.348 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.349 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.350 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.351 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.352 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.353 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.354 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.355 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.356 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.357 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.358 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.359 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.360 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.361 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.362 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.363 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.364 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.365 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.366 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.367 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.368 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.369 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.370 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.371 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.372 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.373 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.374 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.375 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.377 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.378 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.379 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.380 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.381 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.382 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.383 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.384 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.385 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.386 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.387 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.388 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.389 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.390 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.391 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.392 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.393 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.394 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.395 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.396 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.397 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.398 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.399 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-07-21T22:41:33.400 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.401 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.402 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.403 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.404 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.405 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.409 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.410 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.411 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.412 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.413 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.414 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.415 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.416 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.417 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.418 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.420 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.421 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.422 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.423 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.424 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.425 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.426 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.427 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.428 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.429 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.430 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.431 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.432 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.433 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.434 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.435 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.436 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.437 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.438 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.439 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.440 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.441 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.442 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.443 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.444 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.445 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.446 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.447 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.448 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.449 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-07-21T22:41:33.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-07-21T22:41:33.451 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.452 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.453 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.455 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.456 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.457 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.458 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.459 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.460 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-07-21T22:41:33.461 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.462 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.463 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.464 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.465 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.466 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.467 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.468 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.469 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.470 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.471 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.472 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.473 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.475 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.476 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.477 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.478 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.479 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.480 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.481 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.482 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.483 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-07-21T22:41:33.484 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.485 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.487 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.488 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.489 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.490 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.491 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.492 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.493 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.494 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.495 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.496 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.497 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.498 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.499 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.500 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.501 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.502 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.503 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.504 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.505 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.506 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.507 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.508 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.509 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.513 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.514 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.515 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.516 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.517 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:41:33.518 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.519 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-07-21T22:41:33.520 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.521 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.522 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.523 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-07-21T22:41:33.524 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.525 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:33.527 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.529 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:33.530 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.531 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.532 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.533 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.534 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.535 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.536 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.537 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.538 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.539 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.541 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.542 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.543 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.544 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.545 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.546 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-07-21T22:41:33.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.548 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.549 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.550 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.552 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.553 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.554 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.555 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.556 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.557 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.558 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.559 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.560 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.561 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.562 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.563 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.564 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.565 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.566 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.567 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.568 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.569 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.570 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.571 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.572 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.573 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-07-21T22:41:33.574 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.575 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.576 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.577 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.578 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.580 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.581 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.582 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.584 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-07-21T22:41:33.585 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:41:33.586 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-07-21T22:41:33.587 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.588 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.589 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.593 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.594 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.595 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.596 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.597 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.598 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.599 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.600 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.601 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-07-21T22:41:33.602 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.603 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.604 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.605 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.606 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.607 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.608 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.609 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.610 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-07-21T22:41:33.611 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.612 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.613 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.614 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.615 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.616 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.617 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.618 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-07-21T22:41:33.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:41:33.620 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.621 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.622 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.623 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.624 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.625 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.626 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.627 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.628 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.629 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.630 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.631 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.632 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.633 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.634 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.635 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.636 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.637 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.638 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.639 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.640 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.641 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.642 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.643 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.644 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.645 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.646 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.647 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.648 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.649 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.650 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.651 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.652 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.653 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:33.654 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.655 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.657 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.658 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.659 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.660 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.661 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.662 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.663 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.664 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-07-21T22:41:33.665 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.666 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.667 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.668 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.669 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.670 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-07-21T22:41:33.671 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.672 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.673 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.674 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.675 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.676 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.677 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.678 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.679 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:33.680 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.681 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.682 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.683 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.684 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.685 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.686 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-07-21T22:41:33.687 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.688 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.689 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.690 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.691 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.692 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.693 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.694 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.695 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.696 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.697 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.698 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.699 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.700 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.701 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:41:33.702 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.703 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.704 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.705 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.706 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.707 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.708 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.709 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.710 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.711 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.712 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.713 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.714 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.715 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.716 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.717 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.718 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.719 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.720 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.721 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.722 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-07-21T22:41:33.723 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.724 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.725 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.726 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.727 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.728 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.729 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.730 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.731 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.732 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.733 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.734 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.735 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.736 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.737 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.738 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.740 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.741 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-07-21T22:41:33.742 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.743 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-07-21T22:41:33.744 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.747 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.748 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.749 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.750 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 25082, 2023-07-21T22:41:33.751 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "y", 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.752 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.753 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.754 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.755 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-07-21T22:41:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-07-21T22:41:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-07-21T22:41:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-07-21T22:41:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-07-21T22:41:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-07-21T22:41:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-07-21T22:41:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-07-21T22:41:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-07-21T22:41:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-07-21T22:41:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:41:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:41:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-07-21T22:41:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-07-21T22:41:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:41:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-07-21T22:41:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-07-21T22:41:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:41:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:41:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-07-21T22:41:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:41:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:41:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:41:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-07-21T22:41:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:41:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-07-21T22:41:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:41:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-07-21T22:41:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-07-21T22:41:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-07-21T22:41:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-07-21T22:41:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-07-21T22:41:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:41:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-07-21T22:41:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:41:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-07-21T22:41:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:41:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-07-21T22:41:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:41:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:41:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:41:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:41:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:41:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:41:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:41:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:41:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:41:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:41:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:41:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-07-21T22:41:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:41:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:41:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:41:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-07-21T22:41:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:41:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:41:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-07-21T22:41:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-07-21T22:41:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-07-21T22:41:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-07-21T22:41:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-07-21T22:41:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-07-21T22:41:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-07-21T22:41:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:41:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-07-21T22:41:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-07-21T22:41:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-07-21T22:41:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-07-21T22:41:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-07-21T22:41:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:41:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-07-21T22:41:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-07-21T22:41:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-07-21T22:41:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-07-21T22:41:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-07-21T22:41:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:41:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-07-21T22:41:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-07-21T22:41:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-07-21T22:41:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-07-21T22:41:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.410 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.411 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.411 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.411 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.412 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-07-21T22:41:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-07-21T22:41:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:41:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-07-21T22:41:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-07-21T22:41:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-07-21T22:41:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-07-21T22:41:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-07-21T22:41:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-07-21T22:41:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-07-21T22:41:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:41:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "modules": [ 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-07-21T22:41:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard", 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "iostat", 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "nfs", 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "restful", 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "snap_schedule" 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:41:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-07-21T22:41:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-07-21T22:41:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-07-21T22:41:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:41:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-07-21T22:41:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-07-21T22:41:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:41:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-07-21T22:41:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:41:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:41:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-07-21T22:41:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-07-21T22:41:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-07-21T22:41:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:41:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-07-21T22:41:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-07-21T22:41:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:41:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:41:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-07-21T22:41:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:41:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-07-21T22:41:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-07-21T22:41:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-07-21T22:41:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:41:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-07-21T22:41:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-07-21T22:41:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:41:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-07-21T22:41:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:41:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:41:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:41:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:41:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-07-21T22:41:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-07-21T22:41:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-07-21T22:41:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:41:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:41:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-07-21T22:41:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-07-21T22:41:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:41:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:41:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:41:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:41:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:41:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:41:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:41:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:41:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:41:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:41:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:41:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:41:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-07-21T22:41:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-07-21T22:41:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-07-21T22:41:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-07-21T22:41:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-07-21T22:41:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-07-21T22:41:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-07-21T22:41:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:41:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:41:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-07-21T22:41:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-07-21T22:41:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:41:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-07-21T22:41:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-07-21T22:41:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-07-21T22:41:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-07-21T22:41:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-07-21T22:41:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-07-21T22:41:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-07-21T22:41:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-07-21T22:41:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-07-21T22:41:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-07-21T22:41:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-07-21T22:41:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-07-21T22:41:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-07-21T22:41:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:41:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-07-21T22:41:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:41:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-07-21T22:41:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-07-21T22:41:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-07-21T22:41:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-07-21T22:41:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-07-21T22:41:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-07-21T22:41:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-07-21T22:41:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-07-21T22:41:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-07-21T22:41:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-07-21T22:41:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-07-21T22:41:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-07-21T22:41:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-07-21T22:41:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:41:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-07-21T22:41:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-07-21T22:41:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-07-21T22:41:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-07-21T22:41:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:41:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-07-21T22:41:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-07-21T22:41:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-07-21T22:41:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-07-21T22:41:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-07-21T22:41:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-07-21T22:41:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-07-21T22:41:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-07-21T22:41:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "services": { 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard": "https://172.21.15.93:8443/" 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "always_on_modules": { 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "octopus": [ 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-07-21T22:41:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "pacific": [ 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-07-21T22:41:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "quincy": [ 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-07-21T22:41:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-07-21T22:41:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "reef": [ 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-07-21T22:41:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "last_failure_osd_epoch": 85, 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "active_clients": [ 2023-07-21T22:41:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 4108076586 2023-07-21T22:41:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "libcephsqlite", 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 2484323911 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-07-21T22:41:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1011682963 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 2866982852 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 883691992 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-07-21T22:41:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1571047802 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-07-21T22:41:35.303 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:41:35.305 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:33 smithi134 ceph-mon[97873]: pgmap v801: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 712 op/s 2023-07-21T22:41:35.305 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:33 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.305 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:33 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/4280541276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:33 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3277552968' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:34 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:33 smithi194 ceph-mon[98288]: pgmap v801: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 712 op/s 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:33 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:33 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/4280541276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:41:35.306 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:33 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3277552968' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:41:35.307 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:34 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.942+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 msgr2=0x7fe5880a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.942+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 0x7fe5880a3830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe598006fd0 tx=0x7fe598008040 comp rx=0 tx=0).stop 2023-07-21T22:41:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 msgr2=0x7fe59c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c00f830 comp rx=0 tx=0).stop 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 shutdown_connections 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe59c10a070 0x7fe59c1959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fe5880a1380 0x7fe5880a3830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe59c100ca0 0x7fe59c19b940 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 --2- 172.21.15.93:0/3277552968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe59c1015b0 0x7fe59c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 >> 172.21.15.93:0/3277552968 conn(0x7fe59c0fa490 msgr2=0x7fe59c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:41:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 shutdown_connections 2023-07-21T22:41:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:41:32.943+0000 7fe5a348f700 1 -- 172.21.15.93:0/3277552968 wait complete. 2023-07-21T22:41:35.316 DEBUG:tasks.check_counter:Getting stats from a 2023-07-21T22:41:35.317 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-21T22:41:35.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:33 smithi093 ceph-mon[96579]: pgmap v801: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 712 op/s 2023-07-21T22:41:35.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:33 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:33 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/4280541276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:41:35.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:33 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3277552968' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:41:35.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:34 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:35 smithi093 ceph-mon[96579]: pgmap v802: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 707 op/s 2023-07-21T22:41:35.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:35 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:35 smithi194 ceph-mon[98288]: pgmap v802: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 707 op/s 2023-07-21T22:41:35.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:35 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:35 smithi134 ceph-mon[97873]: pgmap v802: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 19 MiB/s wr, 707 op/s 2023-07-21T22:41:35.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:35 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:35.882 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:41:37.405 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 447247, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 376313, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 141648468, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 408021227, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 22, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 8, 2023-07-21T22:41:37.406 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 23.124474703, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 9.421767119, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 15.819780773, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 1.222972279, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 376308, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 20.822871053, 2023-07-21T22:41:37.407 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000055334 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 731178, 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.107053419, 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000146 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.408 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 608576 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 574066, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 755745, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 169683011, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 773778395, 2023-07-21T22:41:37.409 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 9, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 33.277096970, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 18.190357591, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 20.649102248, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 1.112967567, 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:37.410 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 755745, 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 67.102663260, 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000088790 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 873453, 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.169836215, 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000194 2023-07-21T22:41:37.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1837264, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 262464 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 403595, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 425148, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 126037399, 2023-07-21T22:41:37.412 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 574770777, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 12, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 19.716458932, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 7.983045041, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 12.567094520, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 1.644403095, 2023-07-21T22:41:37.413 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 425148, 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 38.440591097, 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000090416 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 714031, 2023-07-21T22:41:37.414 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.107722130, 2023-07-21T22:41:37.415 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000150 2023-07-21T22:41:37.415 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.415 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 4976, 2023-07-21T22:41:37.415 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 144096 2023-07-21T22:41:37.415 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 1, 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: "finisher-MDSRank": { 2023-07-21T22:41:37.416 INFO:teuthology.orchestra.run.smithi093.stdout: "queue_len": 0, 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "complete_latency": { 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 103381, 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 98.662155818, 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000954354 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:37.417 INFO:teuthology.orchestra.run.smithi093.stdout: "queue_len": 0, 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "complete_latency": { 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 62473, 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 7.541383091, 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000120714 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "mds": { 2023-07-21T22:41:37.418 INFO:teuthology.orchestra.run.smithi093.stdout: "request": 239529, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "reply": 237250, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "reply_latency": { 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 237250, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 883.493770423, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.003723893 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "slow_reply": 0, 2023-07-21T22:41:37.419 INFO:teuthology.orchestra.run.smithi093.stdout: "forward": 2276, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_commit": 86, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_split": 5, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_merge": 2, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes": 1937, 2023-07-21T22:41:37.420 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_top": 791, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_bottom": 1136, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_pin_tail": 10, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_pinned": 803, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_expired": 457, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_with_caps": 3, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "caps": 3, 2023-07-21T22:41:37.421 INFO:teuthology.orchestra.run.smithi093.stdout: "subtrees": 8, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse": 395659, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_hit": 228659, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_forward": 2272, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_discover": 180, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:37.422 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_lock": 5316, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "load_cent": 3141, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "q": 0, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "exported": 6, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "exported_inodes": 185, 2023-07-21T22:41:37.423 INFO:teuthology.orchestra.run.smithi093.stdout: "imported": 6, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "imported_inodes": 415, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_backtrace_fetch": 3, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rfiles": 712, 2023-07-21T22:41:37.424 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rbytes": 213687301, 2023-07-21T22:41:37.425 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rsnaps": 6, 2023-07-21T22:41:37.425 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_backtrace_fetch": 3, 2023-07-21T22:41:37.425 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:37.425 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:37.425 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_file_inodes": 3, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_inode_file_caps": 14, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_revoke": 40654, 2023-07-21T22:41:37.426 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_grant": 127587, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_flushsnap_ack": 1406, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_caps": 92108, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_caps_dirty": 32266, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_cap_release": 5717, 2023-07-21T22:41:37.427 INFO:teuthology.orchestra.run.smithi093.stdout: "process_request_cap_release": 189412 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_cache": { 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_update": 493, 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_update_receipt": 2595, 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_try_discover": 1366, 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_send_discover": 2523, 2023-07-21T22:41:37.428 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_handle_discover": 996, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays": 21, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_created": 33464, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_enqueued": 33443, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_migrated": 0, 2023-07-21T22:41:37.429 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "recovery_started": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "recovery_completed": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_exportdir": 9, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_flush": 0, 2023-07-21T22:41:37.430 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_fragmentdir": 7, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_inodestats": 0 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_log": { 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "evadd": 172080, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "evex": 120145, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "evtrm": 120145, 2023-07-21T22:41:37.431 INFO:teuthology.orchestra.run.smithi093.stdout: "ev": 51935, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "evexg": 0, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "evexd": 503, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "segadd": 295, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "segex": 167, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "segtrm": 167, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "seg": 129, 2023-07-21T22:41:37.432 INFO:teuthology.orchestra.run.smithi093.stdout: "segexg": 0, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "segexd": 1, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "expos": 528499013, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "wrpos": 1061874216, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "rdpos": 0, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "jlat": { 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 55875, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 454.081140300, 2023-07-21T22:41:37.433 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.008126731 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: "replayed": 0 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_mem": { 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1403, 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: "ino+": 35150, 2023-07-21T22:41:37.434 INFO:teuthology.orchestra.run.smithi093.stdout: "ino-": 33747, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dir": 75, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dir+": 1500, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dir-": 1425, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dn": 1918, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dn+": 62758, 2023-07-21T22:41:37.435 INFO:teuthology.orchestra.run.smithi093.stdout: "dn-": 60840, 2023-07-21T22:41:37.436 INFO:teuthology.orchestra.run.smithi093.stdout: "cap": 3, 2023-07-21T22:41:37.436 INFO:teuthology.orchestra.run.smithi093.stdout: "cap+": 37013, 2023-07-21T22:41:37.436 INFO:teuthology.orchestra.run.smithi093.stdout: "cap-": 37010, 2023-07-21T22:41:37.436 INFO:teuthology.orchestra.run.smithi093.stdout: "rss": 80224, 2023-07-21T22:41:37.436 INFO:teuthology.orchestra.run.smithi093.stdout: "heap": 198964 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_server": { 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: "dispatch_client_request": 372696, 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: "dispatch_server_request": 12888, 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_request": 239529, 2023-07-21T22:41:37.437 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_session": 5877, 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_peer_request": 34254, 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "req_create_latency": { 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 33329, 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 170.512911340, 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.005116052 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.438 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getattr_latency": { 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 202, 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.350954257, 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.001737397 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.439 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: "req_link_latency": { 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.440 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookup_latency": { 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 68095, 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 128.000759308, 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.001879738 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.441 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupino_latency": { 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 1, 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.002970332, 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.002970332 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.442 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupname_latency": { 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.444 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lssnap_latency": { 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mkdir_latency": { 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.445 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.446 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.446 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.446 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mknod_latency": { 2023-07-21T22:41:37.446 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mksnap_latency": { 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.447 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: "req_open_latency": { 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.448 INFO:teuthology.orchestra.run.smithi093.stdout: "req_readdir_latency": { 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 92988, 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 422.390924878, 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.004542424 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rename_latency": { 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 7318, 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 38.855342667, 2023-07-21T22:41:37.449 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.005309557 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.450 INFO:teuthology.orchestra.run.smithi093.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmdir_latency": { 2023-07-21T22:41:37.451 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 20, 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.235720954, 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.011786047 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.452 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setattr_latency": { 2023-07-21T22:41:37.453 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 37, 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.699801080, 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.018913542 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 1816, 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 1.469851200, 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000809389 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setlayout_latency": { 2023-07-21T22:41:37.455 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setxattr_latency": { 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.456 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: "req_symlink_latency": { 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: "req_unlink_latency": { 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 33444, 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 120.974534407, 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.003617226 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.458 INFO:teuthology.orchestra.run.smithi093.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.459 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_sessions": { 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "session_count": 2, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "session_add": 2, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "session_remove": 0, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "sessions_open": 2, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "sessions_stale": 0, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "total_load": 21836, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "average_load": 10918, 2023-07-21T22:41:37.460 INFO:teuthology.orchestra.run.smithi093.stdout: "avg_session_uptime": 1488 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:37.461 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:37.462 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:37.463 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:37.464 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:37.465 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 502634, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 721, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 440, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 5, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-07-21T22:41:37.466 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-07-21T22:41:37.467 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:37.467 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:37.467 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:37.467 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:37.467 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 186, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 5025883, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 38752, 2023-07-21T22:41:37.468 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "objecter": { 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "op_active": 0, 2023-07-21T22:41:37.469 INFO:teuthology.orchestra.run.smithi093.stdout: "op_laggy": 0, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "op_send": 93701, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "op_send_bytes": 1064031508, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "op_resend": 33, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "op_reply": 93701, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "op_latency": { 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 93701, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 824.761952587, 2023-07-21T22:41:37.470 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.008802061 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "op_inflight": 0, 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "oplen_avg": { 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 93701, 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 94451 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "op": 93701, 2023-07-21T22:41:37.471 INFO:teuthology.orchestra.run.smithi093.stdout: "op_r": 1372, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "op_w": 92329, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "op_rmw": 0, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "op_pg": 0, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_stat": 33, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_create": 21, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_read": 1369, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_write": 57244, 2023-07-21T22:41:37.472 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_writefull": 834, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_writesame": 0, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_append": 0, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_zero": 0, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_truncate": 5, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_delete": 33848, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_mapext": 0, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:37.473 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_getxattr": 3, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_setxattr": 42, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_call": 0, 2023-07-21T22:41:37.474 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_watch": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_notify": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_pgls": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_other": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_active": 0, 2023-07-21T22:41:37.475 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_send": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_resend": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_ping": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_active": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_send": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_resend": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_active": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_send": 0, 2023-07-21T22:41:37.476 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_resend": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_active": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_send": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_resend": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "command_active": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "command_send": 0, 2023-07-21T22:41:37.477 INFO:teuthology.orchestra.run.smithi093.stdout: "command_resend": 0, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "map_epoch": 130, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "map_full": 0, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "map_inc": 48, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_sessions": 12, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_session_open": 12, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_session_close": 0, 2023-07-21T22:41:37.478 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_laggy": 0, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_wr": 735, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_rd": 0, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_del": 317 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "oft": { 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_objs": 1, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:37.479 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_updates": 0, 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_removes": 0 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "osd-slow-ops": { 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "slow_ops_count": 252420 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "purge_queue": { 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:37.480 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing": 0, 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_high_water": 6, 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executed": 33443, 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:37.481 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 1424904, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 314680032, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 1424904, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:37.482 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 1424904, 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 314680032, 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.483 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:37.484 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 93701, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 1067418259, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 93701, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1067418259, 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:37.485 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 1024, 2023-07-21T22:41:37.486 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 93701, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 93701, 2023-07-21T22:41:37.487 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 93701, 2023-07-21T22:41:37.488 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 93701, 2023-07-21T22:41:37.488 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:37.488 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.488 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.488 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 3758096384, 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 33443, 2023-07-21T22:41:37.489 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 3386751, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 33443, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 1369, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 3386751, 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:37.490 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-write_buf_throttle-0x556920e300a0": { 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 599, 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 3758096384, 2023-07-21T22:41:37.491 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 172080, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1057679912, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 172080, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:37.492 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 55875, 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1057679313, 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.493 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:37.494 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:41:37.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:37 smithi093 ceph-mon[96579]: pgmap v803: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 605 op/s 2023-07-21T22:41:37.743 INFO:tasks.check_counter:Daemon mds.a mds.exported=6 2023-07-21T22:41:37.744 INFO:tasks.check_counter:Daemon mds.a mds.imported=6 2023-07-21T22:41:37.744 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=493 2023-07-21T22:41:37.745 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=2595 2023-07-21T22:41:37.745 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2023-07-21T22:41:37.745 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:37.746 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:37.746 DEBUG:tasks.check_counter:Getting stats from d 2023-07-21T22:41:37.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-21T22:41:37.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:37 smithi194 ceph-mon[98288]: pgmap v803: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 605 op/s 2023-07-21T22:41:37.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:37 smithi134 ceph-mon[97873]: pgmap v803: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 605 op/s 2023-07-21T22:41:38.298 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:41:39.503 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:41:39.503 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:39.503 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 443, 2023-07-21T22:41:39.503 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 434, 2023-07-21T22:41:39.503 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1349904, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 248768, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.061170020, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.021964364, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.032137671, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.009615190, 2023-07-21T22:41:39.504 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 433, 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.032891278, 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000075961 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:39.505 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1349904, 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 248768 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 3, 2023-07-21T22:41:39.506 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 332, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 480, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 285808, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.020549606, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.015349921, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.002060319, 2023-07-21T22:41:39.507 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 329, 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.032230268, 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000097964 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:39.508 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 285808 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:39.509 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 4, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 4, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 2992, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 448, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.000763787, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000040633, 2023-07-21T22:41:39.510 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000667637, 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 3, 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000781766, 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000260588 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:39.511 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-07-21T22:41:39.512 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:39.513 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:39.514 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:39.515 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:39.516 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:39.517 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148731, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 31, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:39.518 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:39.519 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-07-21T22:41:39.520 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 446, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1287298, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:39.521 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 446, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 446, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1287298, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:39.522 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:39.523 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:39.523 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:39.523 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:41:39.636 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:39 smithi093 ceph-mon[96579]: pgmap v804: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 479 op/s 2023-07-21T22:41:39.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:39 smithi194 ceph-mon[98288]: pgmap v804: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 479 op/s 2023-07-21T22:41:39.849 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-21T22:41:39.850 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-21T22:41:39.851 DEBUG:tasks.check_counter:Getting stats from g 2023-07-21T22:41:39.851 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-21T22:41:39.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:39 smithi134 ceph-mon[97873]: pgmap v804: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 479 op/s 2023-07-21T22:41:40.403 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:41:41.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:41 smithi194 ceph-mon[98288]: pgmap v805: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2023-07-21T22:41:41.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:41 smithi134 ceph-mon[97873]: pgmap v805: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2023-07-21T22:41:41.915 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:41:41.915 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:41.915 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 2806, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2806, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 344790609, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 871871, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 4, 2023-07-21T22:41:41.916 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.396933827, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.078101733, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.218849713, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.079270087, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 2805, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.170015860, 2023-07-21T22:41:41.917 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000060611 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.918 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 2589, 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2511, 2023-07-21T22:41:41.919 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 296704339, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 886726, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.369102738, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.056690951, 2023-07-21T22:41:41.920 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.247024580, 2023-07-21T22:41:41.921 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.060007758, 2023-07-21T22:41:41.921 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:41.921 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 2510, 2023-07-21T22:41:41.921 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.105173225, 2023-07-21T22:41:41.921 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000041901 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.922 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1819312, 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 256272 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 1624, 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 1950, 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 419845956, 2023-07-21T22:41:41.923 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 1310910, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 9, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.357430452, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.043538129, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.257002574, 2023-07-21T22:41:41.924 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.042916836, 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 1947, 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.134740206, 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000069204 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:41.925 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.926 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.926 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.926 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.926 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T22:41:41.926 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 806768 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 1, 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.927 INFO:teuthology.orchestra.run.smithi093.stdout: "finisher-MDSRank": { 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: "queue_len": 0, 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: "complete_latency": { 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 10701, 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.431073135, 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000040283 2023-07-21T22:41:41.928 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "queue_len": 0, 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "complete_latency": { 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.929 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: "mds": { 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: "request": 0, 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: "reply": 0, 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: "reply_latency": { 2023-07-21T22:41:41.930 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "slow_reply": 0, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "forward": 0, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:41.931 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_commit": 0, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_split": 0, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_merge": 0, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes": 22457, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_top": 7218, 2023-07-21T22:41:41.932 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_bottom": 13828, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_pin_tail": 1411, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_pinned": 12145, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_expired": 12689, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "inodes_with_caps": 0, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "caps": 0, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "subtrees": 1, 2023-07-21T22:41:41.933 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_hit": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_forward": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_discover": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "traverse_lock": 0, 2023-07-21T22:41:41.934 INFO:teuthology.orchestra.run.smithi093.stdout: "load_cent": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "q": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "exported": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "exported_inodes": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "imported": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "imported_inodes": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:41.935 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:41:41.936 INFO:teuthology.orchestra.run.smithi093.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:41.936 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rfiles": 0, 2023-07-21T22:41:41.936 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rbytes": 0, 2023-07-21T22:41:41.936 INFO:teuthology.orchestra.run.smithi093.stdout: "root_rsnaps": 0, 2023-07-21T22:41:41.936 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:41.937 INFO:teuthology.orchestra.run.smithi093.stdout: "scrub_file_inodes": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:41.938 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_caps": 0, 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_cap_release": 0, 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "process_request_cap_release": 0 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_cache": { 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_update": 0, 2023-07-21T22:41:41.939 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_update_receipt": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_try_discover": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_send_discover": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "dir_handle_discover": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:41.940 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_created": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_enqueued": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "strays_migrated": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:41.941 INFO:teuthology.orchestra.run.smithi093.stdout: "recovery_started": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "recovery_completed": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_exportdir": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_flush": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:41.942 INFO:teuthology.orchestra.run.smithi093.stdout: "ireq_inodestats": 0 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_log": { 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "evadd": 0, 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "evex": 0, 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "evtrm": 0, 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "ev": 0, 2023-07-21T22:41:41.943 INFO:teuthology.orchestra.run.smithi093.stdout: "evexg": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "evexd": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "segadd": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "segex": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "segtrm": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "seg": 129, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "segexg": 0, 2023-07-21T22:41:41.944 INFO:teuthology.orchestra.run.smithi093.stdout: "segexd": 0, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "expos": 528499013, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "wrpos": 0, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "rdpos": 1061908914, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "jlat": { 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.945 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: "replayed": 172082 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_mem": { 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: "ino": 1380, 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: "ino+": 34983, 2023-07-21T22:41:41.946 INFO:teuthology.orchestra.run.smithi093.stdout: "ino-": 33603, 2023-07-21T22:41:41.947 INFO:teuthology.orchestra.run.smithi093.stdout: "dir": 52, 2023-07-21T22:41:41.947 INFO:teuthology.orchestra.run.smithi093.stdout: "dir+": 90, 2023-07-21T22:41:41.947 INFO:teuthology.orchestra.run.smithi093.stdout: "dir-": 38, 2023-07-21T22:41:41.948 INFO:teuthology.orchestra.run.smithi093.stdout: "dn": 22457, 2023-07-21T22:41:41.948 INFO:teuthology.orchestra.run.smithi093.stdout: "dn+": 35341, 2023-07-21T22:41:41.948 INFO:teuthology.orchestra.run.smithi093.stdout: "dn-": 12884, 2023-07-21T22:41:41.949 INFO:teuthology.orchestra.run.smithi093.stdout: "cap": 0, 2023-07-21T22:41:41.949 INFO:teuthology.orchestra.run.smithi093.stdout: "cap+": 0, 2023-07-21T22:41:41.949 INFO:teuthology.orchestra.run.smithi093.stdout: "cap-": 0, 2023-07-21T22:41:41.949 INFO:teuthology.orchestra.run.smithi093.stdout: "rss": 82812, 2023-07-21T22:41:41.949 INFO:teuthology.orchestra.run.smithi093.stdout: "heap": 207156 2023-07-21T22:41:41.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.950 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_server": { 2023-07-21T22:41:41.950 INFO:teuthology.orchestra.run.smithi093.stdout: "dispatch_client_request": 0, 2023-07-21T22:41:41.951 INFO:teuthology.orchestra.run.smithi093.stdout: "dispatch_server_request": 0, 2023-07-21T22:41:41.951 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_request": 0, 2023-07-21T22:41:41.951 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_client_session": 0, 2023-07-21T22:41:41.951 INFO:teuthology.orchestra.run.smithi093.stdout: "handle_peer_request": 0, 2023-07-21T22:41:41.952 INFO:teuthology.orchestra.run.smithi093.stdout: "req_create_latency": { 2023-07-21T22:41:41.952 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.952 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.952 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.953 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.953 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getattr_latency": { 2023-07-21T22:41:41.953 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.953 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.954 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: "req_link_latency": { 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.955 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookup_latency": { 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:41.956 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupino_latency": { 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.957 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupname_latency": { 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.958 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.959 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: "req_lssnap_latency": { 2023-07-21T22:41:41.960 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.961 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.961 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.961 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.961 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mkdir_latency": { 2023-07-21T22:41:41.961 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mknod_latency": { 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.962 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: "req_mksnap_latency": { 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.963 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: "req_open_latency": { 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: "req_readdir_latency": { 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.965 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rename_latency": { 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:41.966 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.967 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.967 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.967 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.967 INFO:teuthology.orchestra.run.smithi093.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmdir_latency": { 2023-07-21T22:41:41.968 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.969 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setattr_latency": { 2023-07-21T22:41:41.970 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.971 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.972 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.973 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setlayout_latency": { 2023-07-21T22:41:41.973 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.973 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.973 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: "req_setxattr_latency": { 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.974 INFO:teuthology.orchestra.run.smithi093.stdout: "req_symlink_latency": { 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "req_unlink_latency": { 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.975 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:41.976 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_sessions": { 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: "session_count": 2, 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: "session_add": 2, 2023-07-21T22:41:41.977 INFO:teuthology.orchestra.run.smithi093.stdout: "session_remove": 0, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "sessions_open": 2, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "sessions_stale": 0, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "total_load": 0, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "average_load": 0, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "avg_session_uptime": 3290 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.978 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:41.979 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:41.980 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:41.981 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:41.982 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:41.983 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 295898, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 1740, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-07-21T22:41:41.984 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 186, 2023-07-21T22:41:41.985 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 19452118, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 129284, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-07-21T22:41:41.986 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: "objecter": { 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: "op_active": 0, 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: "op_laggy": 0, 2023-07-21T22:41:41.987 INFO:teuthology.orchestra.run.smithi093.stdout: "op_send": 6450, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "op_send_bytes": 21, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "op_resend": 3, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "op_reply": 6450, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "op_latency": { 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 6450, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 21.856547961, 2023-07-21T22:41:41.988 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.003388612 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: "op_inflight": 0, 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: "oplen_avg": { 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 6450, 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 6451 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.989 INFO:teuthology.orchestra.run.smithi093.stdout: "op": 6450, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "op_r": 6450, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "op_w": 0, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "op_rmw": 0, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "op_pg": 0, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_stat": 3449, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_create": 0, 2023-07-21T22:41:41.990 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_read": 2999, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_write": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_writefull": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_writesame": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_append": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_zero": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_truncate": 0, 2023-07-21T22:41:41.991 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_delete": 0, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_mapext": 0, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_getxattr": 1, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_setxattr": 0, 2023-07-21T22:41:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_call": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_watch": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_notify": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_pgls": 0, 2023-07-21T22:41:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "osdop_other": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_active": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_send": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_resend": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "linger_ping": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_active": 0, 2023-07-21T22:41:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_send": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "poolop_resend": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_active": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_send": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "poolstat_resend": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_active": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_send": 0, 2023-07-21T22:41:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: "statfs_resend": 0, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "command_active": 0, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "command_send": 0, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "command_resend": 0, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "map_epoch": 130, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "map_full": 0, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "map_inc": 48, 2023-07-21T22:41:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_sessions": 11, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_session_open": 11, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_session_close": 0, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_laggy": 0, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_wr": 0, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_rd": 2, 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_del": 0 2023-07-21T22:41:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: "oft": { 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_objs": 0, 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_updates": 0, 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: "omap_total_removes": 0 2023-07-21T22:41:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: "osd-slow-ops": { 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: "slow_ops_count": 0 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: "purge_queue": { 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing": 0, 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executing_high_water": 0, 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_executed": 0, 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 7015, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1060678153, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 7015, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 7015, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1060678153, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 6450, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 1057747870, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 6450, 2023-07-21T22:41:42.004 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1057747870, 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:42.005 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 1024, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:42.006 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 6450, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 6450, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 6450, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 6450, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:42.007 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 3758096384, 2023-07-21T22:41:42.008 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:42.009 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:42.009 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:42.009 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:42.009 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:42.009 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 0, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 0, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:42.010 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-write_buf_throttle-0x55b2997c8460": { 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 3758096384, 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:42.011 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:42.012 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 0, 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 0, 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:42.013 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:41:42.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:41 smithi093 ceph-mon[96579]: pgmap v805: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.6 MiB/s wr, 427 op/s 2023-07-21T22:41:42.365 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-07-21T22:41:42.366 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-07-21T22:41:42.366 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-07-21T22:41:42.366 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-07-21T22:41:42.366 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-07-21T22:41:42.366 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:42.367 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:42.367 DEBUG:tasks.check_counter:Getting stats from j 2023-07-21T22:41:42.367 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-21T22:41:42.923 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:41:43.687 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:43 smithi093 ceph-mon[96579]: pgmap v806: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 275 KiB/s wr, 63 op/s 2023-07-21T22:41:43.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:43 smithi194 ceph-mon[98288]: pgmap v806: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 275 KiB/s wr, 63 op/s 2023-07-21T22:41:43.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:43 smithi134 ceph-mon[97873]: pgmap v806: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 275 KiB/s wr, 63 op/s 2023-07-21T22:41:44.136 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-07-21T22:41:44.136 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:44.136 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-07-21T22:41:44.136 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.001311261, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:41:44.137 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000287593, 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.138 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:41:44.139 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 3, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 333, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 480, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 285984, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:44.140 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.020745591, 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.015313377, 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001991597, 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:44.141 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 330, 2023-07-21T22:41:44.142 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.066158542, 2023-07-21T22:41:44.142 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000200480 2023-07-21T22:41:44.142 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.142 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:44.142 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 285984 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.143 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:44.144 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 448, 2023-07-21T22:41:44.144 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 437, 2023-07-21T22:41:44.144 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1353168, 2023-07-21T22:41:44.144 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 249808, 2023-07-21T22:41:44.144 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.060588930, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.022140157, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.032738588, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.009480736, 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:44.145 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 437, 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.024579709, 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000056246 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:44.146 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1353168, 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 249808 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-07-21T22:41:44.147 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:44.148 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:44.149 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:44.150 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:44.151 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:44.152 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:44.152 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:44.152 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:44.152 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:44.153 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:44.153 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:44.153 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-07-21T22:41:44.154 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:44.154 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:44.154 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:44.154 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:44.155 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148784, 2023-07-21T22:41:44.155 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 31, 2023-07-21T22:41:44.155 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:44.156 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:44.156 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-07-21T22:41:44.156 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-07-21T22:41:44.156 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:44.157 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:44.157 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:44.157 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:44.158 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-07-21T22:41:44.158 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-07-21T22:41:44.158 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:44.158 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-07-21T22:41:44.159 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-07-21T22:41:44.160 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 447, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1287428, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 447, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-07-21T22:41:44.161 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 447, 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1287428, 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:44.162 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-07-21T22:41:44.163 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-07-21T22:41:44.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-21T22:41:44.572 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-21T22:41:44.572 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-07-21T22:41:44.572 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-07-21T22:41:44.573 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-21T22:41:44.573 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-21T22:41:44.573 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-21T22:41:44.573 DEBUG:tasks.check_counter:Getting stats from b 2023-07-21T22:41:44.573 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-21T22:41:45.166 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:41:45.668 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:45 smithi134 ceph-mon[97873]: pgmap v807: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 139 KiB/s wr, 38 op/s 2023-07-21T22:41:45.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:45 smithi194 ceph-mon[98288]: pgmap v807: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 139 KiB/s wr, 38 op/s 2023-07-21T22:41:45.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:45 smithi093 ceph-mon[96579]: pgmap v807: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 139 KiB/s wr, 38 op/s 2023-07-21T22:41:46.626 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:46 smithi134 ceph-mon[97873]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:46.770 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 632267, 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 553818, 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 200109462, 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 630532849, 2023-07-21T22:41:46.771 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 12, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 28.165644652, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 10.978579670, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 19.637933517, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 1.647282880, 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:46.772 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 553816, 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 32.793283112, 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000059213 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1056277, 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.145000419, 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000137 2023-07-21T22:41:46.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 421657, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 424644, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 138023272, 2023-07-21T22:41:46.774 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 426730582, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 14, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 18.536770722, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 7.397951491, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 12.479460327, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.977640689, 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 424644, 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 33.249769521, 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000078300 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 767679, 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.104889879, 2023-07-21T22:41:46.776 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000136 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 739920 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 482606, 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 719658, 2023-07-21T22:41:46.777 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 143847927, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 794624035, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 19, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 23.257436482, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 11.309210351, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 16.553401057, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 1.467127507, 2023-07-21T22:41:46.778 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 719655, 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 61.957433301, 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000086093 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 584805, 2023-07-21T22:41:46.779 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.127522865, 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000218 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1862896, 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 269040 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-21T22:41:46.780 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 100497, 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 106.715071733, 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001061873 2023-07-21T22:41:46.781 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 73929, 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 8.731901506, 2023-07-21T22:41:46.782 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000118111 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 313885, 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 263079, 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 263079, 2023-07-21T22:41:46.783 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1134.621270610, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004312853 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 50805, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 92, 2023-07-21T22:41:46.784 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 5, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 4, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 1730, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 694, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 977, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 59, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 719, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 1309, 2023-07-21T22:41:46.785 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 2, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 2, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 6, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 495038, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 241490, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 50791, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 288, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:46.786 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 3054, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 2529, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 21, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 585, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 14, 2023-07-21T22:41:46.787 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 778, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 9, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 246, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 9, 2023-07-21T22:41:46.788 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 9, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 26, 2023-07-21T22:41:46.789 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 71630, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 176182, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 1503, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 115164, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 27852, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 4981, 2023-07-21T22:41:46.790 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 204922 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 1044, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 1382, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 767, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 1383, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 1662, 2023-07-21T22:41:46.791 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 21, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 39693, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 39672, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-21T22:41:46.792 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 23, 2023-07-21T22:41:46.793 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 9, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 187248, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 133442, 2023-07-21T22:41:46.794 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 133442, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 53806, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 1420, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 316, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 188, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 188, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 130, 2023-07-21T22:41:46.795 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 2, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 604004635, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 1137705438, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 54402, 2023-07-21T22:41:46.796 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 459.369862935, 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008443988 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 1277, 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 41749, 2023-07-21T22:41:46.797 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 40472, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 91, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 906, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 815, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 1752, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 77068, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 75316, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 2, 2023-07-21T22:41:46.798 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 42762, 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 42760, 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 80004, 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 198964 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-21T22:41:46.799 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 472475, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 12329, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 313885, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 4382, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 45541, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 39570, 2023-07-21T22:41:46.800 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 206.467432830, 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005217776 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8336, 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 141.343874450, 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.016955839 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.801 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.802 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 73745, 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 197.807700600, 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002682320 2023-07-21T22:41:46.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-21T22:41:46.804 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000097467, 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000097467 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.806 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.807 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.808 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.809 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000501268, 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000501268 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.810 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 94495, 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 411.624020810, 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004356040 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6227, 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 45.171085274, 2023-07-21T22:41:46.811 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007254068 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.812 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 20, 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.212455442, 2023-07-21T22:41:46.813 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.010622772 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:46.814 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 71, 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.130195113, 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.015918241 2023-07-21T22:41:46.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 940, 2023-07-21T22:41:46.816 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.290750242, 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001373138 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.817 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-21T22:41:46.818 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 39673, 2023-07-21T22:41:46.819 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 129.573157114, 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003266028 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:46.820 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2023-07-21T22:41:46.821 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 20791, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 10395, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 1500 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.822 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:46.823 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:46.824 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:46.825 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-21T22:41:46.827 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:46.828 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 469173, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 728, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 88, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 1, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:46.829 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 186, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:46.830 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 4702539, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 37463, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-21T22:41:46.831 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 98492, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 1140756324, 2023-07-21T22:41:46.832 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 98492, 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 98492, 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 870.957162786, 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.008842922 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.833 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 98492, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 99269 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 98492, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 1383, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 97109, 2023-07-21T22:41:46.834 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 37, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 26, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1374, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 55776, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 800, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-21T22:41:46.835 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 5, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 40115, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:46.836 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 9, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 52, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-21T22:41:46.837 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-21T22:41:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-21T22:41:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-21T22:41:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 130, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 48, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-21T22:41:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 755, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 320 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 325896 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 6, 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 39672, 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1536526, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 350880345, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1536526, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1536526, 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 350880345, 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 98492, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1144771796, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 98492, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1144771796, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 98492, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 98492, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 98492, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 98492, 2023-07-21T22:41:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 39672, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 4015472, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 39672, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1374, 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 4015472, 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55a6085ea0a0": { 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 187248, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1133511134, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 187248, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 54402, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1133511134, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:46.856 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-21T22:41:46.856 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:46 smithi194 ceph-mon[98288]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:46.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:46 smithi093 ceph-mon[96579]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:41:47.234 INFO:tasks.check_counter:Daemon mds.b mds.exported=21 2023-07-21T22:41:47.234 INFO:tasks.check_counter:Daemon mds.b mds.imported=14 2023-07-21T22:41:47.235 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1044 2023-07-21T22:41:47.235 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1382 2023-07-21T22:41:47.235 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-21T22:41:47.235 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:47.235 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:47.235 DEBUG:tasks.check_counter:Getting stats from e 2023-07-21T22:41:47.236 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-21T22:41:47.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:47 smithi134 ceph-mon[97873]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 72 KiB/s wr, 18 op/s 2023-07-21T22:41:47.792 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:41:47.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:47 smithi194 ceph-mon[98288]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 72 KiB/s wr, 18 op/s 2023-07-21T22:41:47.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:47 smithi093 ceph-mon[96579]: pgmap v808: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 72 KiB/s wr, 18 op/s 2023-07-21T22:41:49.027 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-21T22:41:49.027 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:49.027 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1634, 2023-07-21T22:41:49.028 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1636, 2023-07-21T22:41:49.028 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 386312592, 2023-07-21T22:41:49.028 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 508238, 2023-07-21T22:41:49.029 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:49.029 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-07-21T22:41:49.029 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.306061585, 2023-07-21T22:41:49.029 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.034747432, 2023-07-21T22:41:49.030 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.215668063, 2023-07-21T22:41:49.030 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.043760013, 2023-07-21T22:41:49.030 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1634, 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.117142268, 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000071690 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.032 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2748, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2664, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 463315185, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 935207, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:49.033 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.487309110, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.061612820, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.345113401, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.071011078, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2664, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.173570581, 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000065154 2023-07-21T22:41:49.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1845424, 2023-07-21T22:41:49.035 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 259088 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2630, 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2960, 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 218912492, 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1542940, 2023-07-21T22:41:49.036 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.340520510, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.069640736, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.166309737, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.081215797, 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:49.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2957, 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.184975206, 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000062555 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 725632 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-21T22:41:49.039 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10736, 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.461523415, 2023-07-21T22:41:49.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000042988 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.041 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-07-21T22:41:49.042 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-07-21T22:41:49.043 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 20042, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 5895, 2023-07-21T22:41:49.044 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 14139, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 8, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 11620, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 14022, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-07-21T22:41:49.045 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-07-21T22:41:49.046 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:41:49.047 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:49.048 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:41:49.049 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-07-21T22:41:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-07-21T22:41:49.051 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:49.052 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-07-21T22:41:49.053 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-07-21T22:41:49.054 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-21T22:41:49.055 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 131, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 532688085, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 0, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 1069009836, 2023-07-21T22:41:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 182515 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 1424, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 33984, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 32560, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 59, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 109, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 50, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 20042, 2023-07-21T22:41:49.058 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 34437, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 14395, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 78412, 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207156 2023-07-21T22:41:49.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-07-21T22:41:49.060 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-21T22:41:49.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-21T22:41:49.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-21T22:41:49.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.072 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:49.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.077 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.078 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.079 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.080 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.081 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-07-21T22:41:49.082 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 5060 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.083 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:49.085 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:49.086 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:49.087 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:49.088 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 300467, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 1812, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-21T22:41:49.089 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 186, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:49.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 17883822, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 120989, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-21T22:41:49.091 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 6436, 2023-07-21T22:41:49.092 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 6436, 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6436, 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 22.632484170, 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003516545 2023-07-21T22:41:49.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6436, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6437 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 6436, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 6436, 2023-07-21T22:41:49.094 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 3400, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 3034, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-07-21T22:41:49.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-21T22:41:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:49.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-21T22:41:49.098 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-21T22:41:49.099 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-21T22:41:49.100 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 130, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 48, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 11, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 11, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-21T22:41:49.101 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-07-21T22:41:49.102 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.103 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:49.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 7008, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1067877787, 2023-07-21T22:41:49.105 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 7008, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 7008, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1067877787, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:49.107 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6436, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1064927249, 2023-07-21T22:41:49.108 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6436, 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1064927249, 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:49.110 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6436, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 6436, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6436, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6436, 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:49.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:49.112 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:49.113 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55e061dba460": { 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:49.115 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:49.116 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:49.117 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:49.117 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:49.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:49.117 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-21T22:41:49.452 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-21T22:41:49.452 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-21T22:41:49.452 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-07-21T22:41:49.452 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-07-21T22:41:49.453 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-07-21T22:41:49.453 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:49.453 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:49.453 DEBUG:tasks.check_counter:Getting stats from h 2023-07-21T22:41:49.454 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-21T22:41:49.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:49 smithi194 ceph-mon[98288]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-07-21T22:41:49.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:49 smithi134 ceph-mon[97873]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-07-21T22:41:49.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:49 smithi093 ceph-mon[96579]: pgmap v809: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-07-21T22:41:50.010 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 836, 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 837, 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 87554696, 2023-07-21T22:41:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 259521, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.110360104, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.019599112, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.060497332, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.023690676, 2023-07-21T22:41:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 836, 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.128645637, 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000153882 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2656, 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 541, 2023-07-21T22:41:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 527, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 80075744, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 189318, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.085733569, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.011596800, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.066033518, 2023-07-21T22:41:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.013623882, 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 526, 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.052165477, 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000099173 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 281504, 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 66416 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 463, 2023-07-21T22:41:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 546, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 78172577, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 368604, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.083579848, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.009981213, 2023-07-21T22:41:51.266 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.057564071, 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.012501922, 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 544, 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.032323583, 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000059418 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:51.267 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 224912 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-21T22:41:51.268 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:51.269 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2805, 2023-07-21T22:41:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.244520746, 2023-07-21T22:41:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000087173 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-21T22:41:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 12920, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 8682, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 4234, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 4, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 516, 2023-07-21T22:41:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 22, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-07-21T22:41:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-07-21T22:41:51.428 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 246, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2023-07-21T22:41:51.429 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-07-21T22:41:51.430 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-07-21T22:41:51.431 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-21T22:41:51.432 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:51.433 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-07-21T22:41:51.434 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-21T22:41:51.435 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-21T22:41:51.436 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 6, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 847378235, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 0, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 849803679, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.437 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 55861 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 887, 2023-07-21T22:41:51.438 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 12198, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 11311, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 90, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 107, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 17, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 12920, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 13056, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 136, 2023-07-21T22:41:51.439 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 63120, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207132 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-07-21T22:41:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-21T22:41:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-21T22:41:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2023-07-21T22:41:51.461 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3291 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:51.462 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:51.463 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:51.464 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:51.465 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:51.466 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 265907, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 1136, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-21T22:41:51.467 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 186, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:51.468 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 11412946, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 78297, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:51.469 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 1690, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2023-07-21T22:41:51.470 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 1690, 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1690, 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 12.644714874, 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007482079 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-21T22:41:51.471 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1690, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1691 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 1690, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 1690, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-21T22:41:51.472 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 876, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 811, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-21T22:41:51.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-07-21T22:41:51.474 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-21T22:41:51.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-21T22:41:51.476 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-21T22:41:51.477 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 130, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 14, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 10, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 10, 2023-07-21T22:41:51.478 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-07-21T22:41:51.479 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-21T22:41:51.480 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.481 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1837, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 245627671, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1837, 2023-07-21T22:41:51.482 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1837, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 245627671, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.483 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:51.484 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1690, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 245049573, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1690, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 245049573, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.485 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-21T22:41:51.486 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1690, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1690, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1690, 2023-07-21T22:41:51.487 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1690, 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:51.488 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:51.489 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.490 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x562019ad4320": { 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:51.491 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:51.492 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:51.493 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:51.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:51.493 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-21T22:41:51.667 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:51 smithi134 ceph-mon[97873]: pgmap v810: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-07-21T22:41:51.669 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-07-21T22:41:51.669 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-07-21T22:41:51.669 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-07-21T22:41:51.670 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-07-21T22:41:51.670 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-07-21T22:41:51.670 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:51.670 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:51.670 DEBUG:tasks.check_counter:Getting stats from k 2023-07-21T22:41:51.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-21T22:41:51.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:51 smithi194 ceph-mon[98288]: pgmap v810: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-07-21T22:41:51.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:51 smithi093 ceph-mon[96579]: pgmap v810: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 14 op/s 2023-07-21T22:41:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:41:53.647 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3209, 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 3209, 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 319724461, 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 997204, 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:53.648 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.409448306, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.080040498, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.217207780, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.088997464, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3208, 2023-07-21T22:41:53.649 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.213104455, 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000066429 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.650 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2780, 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2698, 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 350549506, 2023-07-21T22:41:53.651 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 946861, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.430938925, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.058024131, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.295360723, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.071059306, 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:53.652 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2697, 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.167779227, 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000062209 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1843248, 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 260416 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1059, 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1392, 2023-07-21T22:41:53.654 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 222387713, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 1057991, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.202582990, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.030390687, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.135751881, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.027172276, 2023-07-21T22:41:53.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1389, 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.107407776, 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000077327 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 729264 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-21T22:41:53.657 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10687, 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.390950610, 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000036581 2023-07-21T22:41:53.658 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.659 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.660 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-07-21T22:41:53.661 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 20287, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 4654, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 10692, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 4941, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 13638, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 9939, 2023-07-21T22:41:53.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:53.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-07-21T22:41:53.664 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:41:53.665 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:41:53.666 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-07-21T22:41:53.667 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-07-21T22:41:53.668 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:53.669 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:41:53.670 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-21T22:41:53.671 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 131, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-21T22:41:53.672 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 360725264, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 0, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 893173692, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 144691 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 1451, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 30076, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 28625, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 59, 2023-07-21T22:41:53.674 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 99, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 40, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 20287, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 30448, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 10161, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-07-21T22:41:53.675 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 79684, 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207156 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-07-21T22:41:53.676 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-21T22:41:53.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.679 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.682 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.683 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.684 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.685 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.686 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.688 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.689 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.690 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.691 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-21T22:41:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 5069 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 299487, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 1824, 2023-07-21T22:41:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 186, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 18081635, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 122631, 2023-07-21T22:41:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-21T22:41:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 6469, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 6469, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6469, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 20.266537578, 2023-07-21T22:41:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003132870 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6469, 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6470 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 6469, 2023-07-21T22:41:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 6469, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 3499, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 2968, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-07-21T22:41:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-21T22:41:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-21T22:41:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-21T22:41:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 130, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-21T22:41:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 48, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 11, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 11, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-07-21T22:41:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-21T22:41:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-07-21T22:41:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 7044, 2023-07-21T22:41:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 891995802, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 7044, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 7044, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 891995802, 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-21T22:41:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6469, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 889039568, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6469, 2023-07-21T22:41:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 889039568, 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6469, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 6469, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6469, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6469, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:53.726 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5581fc6ee460": { 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-21T22:41:53.727 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-21T22:41:53.728 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-21T22:41:53.729 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-21T22:41:53.795 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:53 smithi134 ceph-mon[97873]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-07-21T22:41:53.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:53 smithi194 ceph-mon[98288]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-07-21T22:41:53.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:53 smithi093 ceph-mon[96579]: pgmap v811: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2023-07-21T22:41:54.070 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-07-21T22:41:54.070 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-07-21T22:41:54.070 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-07-21T22:41:54.071 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-07-21T22:41:54.071 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-07-21T22:41:54.071 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:54.071 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:54.071 DEBUG:tasks.check_counter:Getting stats from c 2023-07-21T22:41:54.072 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-21T22:41:54.748 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:41:55.768 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:55 smithi194 ceph-mon[98288]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-07-21T22:41:55.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:55 smithi134 ceph-mon[97873]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-07-21T22:41:55.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:55 smithi093 ceph-mon[96579]: pgmap v812: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 822954, 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 927122, 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 250410804, 2023-07-21T22:41:56.634 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 861694830, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 17, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 42.135121852, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 19.981674538, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 31.608075944, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.162258809, 2023-07-21T22:41:56.635 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 927120, 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 156.993376651, 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000169334 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1157789, 2023-07-21T22:41:56.636 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.250588966, 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000216 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 403042, 2023-07-21T22:41:56.637 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 392225, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 127871931, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 496304423, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 21.673725208, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 9.369334857, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 14.378164077, 2023-07-21T22:41:56.638 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.247875824, 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 392229, 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 88.974167167, 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000226842 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:56.639 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 700284, 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.115318668, 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000164 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1857632, 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 281712 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:56.640 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 227373, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 218180, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 78134848, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 402488242, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 20, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 8, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 12.385816722, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 4.652361208, 2023-07-21T22:41:56.641 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 7.938010843, 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.335577116, 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 218173, 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 58.381427824, 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000267592 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.642 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 366669, 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.065935669, 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000179 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1232, 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 728160 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.643 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 94916, 2023-07-21T22:41:56.644 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 116.335829679, 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001225671 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 56615, 2023-07-21T22:41:56.645 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 10.434724740, 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000184310 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 306022, 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 254788, 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2023-07-21T22:41:56.646 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 254788, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 895.354368304, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003514115 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 51233, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 6, 2023-07-21T22:41:56.647 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 99, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 6, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 2, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 1907, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 746, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 1130, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 31, 2023-07-21T22:41:56.648 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 800, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 737, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 1, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 1, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 6, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 477156, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 255928, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 51228, 2023-07-21T22:41:56.649 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 187, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 5296, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 11, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 355, 2023-07-21T22:41:56.650 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 11, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 513, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 9, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 246, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2023-07-21T22:41:56.651 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 9, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:56.652 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 9, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 12, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 39317, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 130417, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 1535, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 89558, 2023-07-21T22:41:56.653 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 30336, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 4605, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 206388 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 1089, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 1834, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 968, 2023-07-21T22:41:56.654 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 1865, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 1359, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 21, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 32278, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 32257, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:56.655 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 14, 2023-07-21T22:41:56.656 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 8, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 182516, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 130268, 2023-07-21T22:41:56.657 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 130268, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 52248, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 1902, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 307, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 179, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 179, 2023-07-21T22:41:56.658 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 131, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 3, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 532688085, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 1069103694, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 51428, 2023-07-21T22:41:56.659 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 423.265018899, 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.008230244 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 1452, 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 34133, 2023-07-21T22:41:56.660 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 32681, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 101, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1190, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 1089, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 1926, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 61459, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 59533, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2023-07-21T22:41:56.661 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 36212, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 36212, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 76984, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 198964 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 431205, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 9944, 2023-07-21T22:41:56.662 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 306022, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 4653, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 32101, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 32143, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 162.127208429, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.005043935 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.663 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 116, 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.828546414, 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.015763331 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.664 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.665 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 70010, 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 123.373890206, 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001762232 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.666 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2023-07-21T22:41:56.667 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.668 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2023-07-21T22:41:56.669 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.670 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2023-07-21T22:41:56.671 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.672 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 110743, 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 437.839132792, 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003953650 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 8575, 2023-07-21T22:41:56.673 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 45.260372415, 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.005278177 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.674 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 20, 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.126483288, 2023-07-21T22:41:56.675 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006324164 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.676 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 75, 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.390551292, 2023-07-21T22:41:56.677 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.018540683 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:56.678 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 848, 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.298354466, 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001531078 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.679 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.680 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 32258, 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 122.109829002, 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003785412 2023-07-21T22:41:56.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 2, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 1, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 21937, 2023-07-21T22:41:56.683 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 21937, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1470 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:56.684 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:56.685 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:56.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:56.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:56.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 460525, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 647, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 352, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 4, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2023-07-21T22:41:56.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 186, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:56.690 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 5205693, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 37986, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2023-07-21T22:41:56.691 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 88036, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 1071481656, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 27, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 88036, 2023-07-21T22:41:56.692 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 88036, 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 777.990656238, 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.008837187 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2023-07-21T22:41:56.693 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 88036, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 88810 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 88036, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 1360, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 86676, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2023-07-21T22:41:56.694 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 37, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 22, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1345, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 52773, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 821, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2023-07-21T22:41:56.695 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 6, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 32664, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 9, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 44, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:56.696 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:56.697 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2023-07-21T22:41:56.698 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2023-07-21T22:41:56.699 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 130, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 48, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2023-07-21T22:41:56.700 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 760, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 12, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 317 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:56.701 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 315678 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:56.702 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 6, 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 32257, 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:56.703 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1453365, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 332176159, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1453365, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:56.704 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1453365, 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 332176159, 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.705 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:41:56.706 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 88036, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1074748917, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 88036, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1074748917, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.707 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:56.708 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 88036, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 88036, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 88036, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 88036, 2023-07-21T22:41:56.709 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:56.710 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 32257, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 3267357, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 32257, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:56.711 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1345, 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3267357, 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.712 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55694d8020a0": { 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 30104, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 182516, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 1064909390, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:56.713 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 182516, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 51428, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1064879286, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:56.714 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:56.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.715 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:56.715 INFO:teuthology.orchestra.run.smithi194.stdout:} 2023-07-21T22:41:57.099 INFO:tasks.check_counter:Daemon mds.c mds.exported=11 2023-07-21T22:41:57.099 INFO:tasks.check_counter:Daemon mds.c mds.imported=11 2023-07-21T22:41:57.100 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1089 2023-07-21T22:41:57.100 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1834 2023-07-21T22:41:57.100 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-07-21T22:41:57.100 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:57.100 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:57.100 DEBUG:tasks.check_counter:Getting stats from f 2023-07-21T22:41:57.101 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-21T22:41:57.657 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:41:57.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:57 smithi194 ceph-mon[98288]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-07-21T22:41:57.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:57 smithi134 ceph-mon[97873]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-07-21T22:41:57.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:57 smithi093 ceph-mon[96579]: pgmap v813: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 63 KiB/s wr, 10 op/s 2023-07-21T22:41:58.951 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 206062, 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 200025, 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 66813874, 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 324748961, 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 17, 2023-07-21T22:41:58.952 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 11.500206825, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 4.436912572, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 7.379862550, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.081785892, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 200025, 2023-07-21T22:41:58.953 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 52.638475766, 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000263159 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 339621, 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.061249935, 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000180 2023-07-21T22:41:58.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1854224, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 282848 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 203709, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 201734, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 63979167, 2023-07-21T22:41:58.955 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 439146704, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 13, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 13.269482694, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 6.033243376, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 7.559644246, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.219539415, 2023-07-21T22:41:58.956 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 201731, 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 50.566011390, 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000250660 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 324873, 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.059609435, 2023-07-21T22:41:58.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000183 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2128, 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 733504 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 962108, 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1011102, 2023-07-21T22:41:58.958 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 301091728, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 781709165, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 15, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 46.760662030, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 21.353730004, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 34.591697079, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.021737623, 2023-07-21T22:41:58.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1011100, 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 209.558418841, 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000207257 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1432422, 2023-07-21T22:41:58.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.287565664, 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000200 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2023-07-21T22:41:58.961 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 76414, 2023-07-21T22:41:58.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 99.300781155, 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001299510 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:41:58.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 49911, 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 9.861682174, 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000197585 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 269034, 2023-07-21T22:41:58.964 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 217579, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 217579, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 852.040447110, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003916004 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 51454, 2023-07-21T22:41:58.965 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 85, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 5, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 1, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 1897, 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 772, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 1094, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 31, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 749, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 479, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2023-07-21T22:41:58.967 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 6, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 396063, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 207316, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 51452, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 201, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:41:58.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 4549, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 6, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 202, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 6, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 387, 2023-07-21T22:41:58.969 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 11, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 246, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 11, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2023-07-21T22:41:58.970 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 11, 2023-07-21T22:41:58.971 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 16, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 37976, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 118047, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 1131, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 77193, 2023-07-21T22:41:58.972 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 23784, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 3880, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 178319 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 382, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 3060, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 1595, 2023-07-21T22:41:58.973 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 3044, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 449, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 20, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 28454, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 28434, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2023-07-21T22:41:58.974 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 6, 2023-07-21T22:41:58.975 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 6, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 144693, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 87432, 2023-07-21T22:41:58.976 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 87432, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 57261, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 1750, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 248, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 120, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 120, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 131, 2023-07-21T22:41:58.977 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 3, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 360725264, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 893218778, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 41731, 2023-07-21T22:41:58.978 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 385.959104416, 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.009248738 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2023-07-21T22:41:58.979 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 1474, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 30228, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 28754, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 83, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1751, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 1668, 2023-07-21T22:41:58.980 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 1901, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 53300, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 51399, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 32158, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 32158, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 79536, 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 198964 2023-07-21T22:41:58.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 366375, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 10955, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 269034, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 3563, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 26312, 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2023-07-21T22:41:58.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 28303, 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 163.895288690, 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.005790739 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 162, 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.607867883, 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003752270 2023-07-21T22:41:58.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2023-07-21T22:41:58.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 57637, 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 132.172634320, 2023-07-21T22:41:58.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002293190 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2023-07-21T22:41:58.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2023-07-21T22:41:58.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.989 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.990 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.991 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.992 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 95967, 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 397.908090706, 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004146301 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6176, 2023-07-21T22:41:58.993 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 37.329035628, 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006044209 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.994 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 20, 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.139200923, 2023-07-21T22:41:58.995 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006960046 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2023-07-21T22:41:58.997 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 60, 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.193948897, 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.019899148 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:58.998 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 820, 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.248342742, 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001522369 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2023-07-21T22:41:58.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2023-07-21T22:41:59.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 28434, 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 117.546037321, 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004133995 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2023-07-21T22:41:59.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2023-07-21T22:41:59.003 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 2, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 1, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 21596, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 21596, 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1473 2023-07-21T22:41:59.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:41:59.005 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:41:59.006 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:41:59.007 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:41:59.008 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 546057, 2023-07-21T22:41:59.009 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 698, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 176, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 2, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2023-07-21T22:41:59.010 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8160, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 186, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:41:59.011 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 5127724, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 38737, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2023-07-21T22:41:59.012 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 74229, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 894707542, 2023-07-21T22:41:59.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 33, 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 74229, 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 74229, 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 724.168723059, 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.009755873 2023-07-21T22:41:59.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 74229, 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 74971 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 74229, 2023-07-21T22:41:59.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 1288, 2023-07-21T22:41:59.016 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 72941, 2023-07-21T22:41:59.016 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2023-07-21T22:41:59.016 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2023-07-21T22:41:59.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 33, 2023-07-21T22:41:59.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 21, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1277, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 43008, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 780, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2023-07-21T22:41:59.017 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 5, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 28750, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 11, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 42, 2023-07-21T22:41:59.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2023-07-21T22:41:59.019 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2023-07-21T22:41:59.020 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2023-07-21T22:41:59.021 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 130, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 48, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2023-07-21T22:41:59.022 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 733, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 311 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2023-07-21T22:41:59.023 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 279995 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.024 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 6, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 28434, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.025 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1371875, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 314171119, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1371875, 2023-07-21T22:41:59.026 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1371875, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 314171119, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:59.028 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 74229, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 897589592, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 74229, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 897589592, 2023-07-21T22:41:59.029 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2023-07-21T22:41:59.030 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 74229, 2023-07-21T22:41:59.031 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 74229, 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 74229, 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 74229, 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.032 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 28434, 2023-07-21T22:41:59.033 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2882050, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 28434, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1277, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2882050, 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:59.034 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55c4ff2880a0": { 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 599, 2023-07-21T22:41:59.035 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 144693, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 889024474, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 144693, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:41:59.036 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 41731, 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 889023875, 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:41:59.037 INFO:teuthology.orchestra.run.smithi194.stdout:} 2023-07-21T22:41:59.403 INFO:tasks.check_counter:Daemon mds.f mds.exported=6 2023-07-21T22:41:59.403 INFO:tasks.check_counter:Daemon mds.f mds.imported=6 2023-07-21T22:41:59.403 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=382 2023-07-21T22:41:59.403 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=3060 2023-07-21T22:41:59.404 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-07-21T22:41:59.404 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:41:59.404 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:41:59.404 DEBUG:tasks.check_counter:Getting stats from i 2023-07-21T22:41:59.405 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-21T22:41:59.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:41:59 smithi194 ceph-mon[98288]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-07-21T22:41:59.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:41:59 smithi134 ceph-mon[97873]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-07-21T22:41:59.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:41:59 smithi093 ceph-mon[96579]: pgmap v814: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 64 KiB/s wr, 10 op/s 2023-07-21T22:41:59.960 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:42:01.348 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2023-07-21T22:42:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:42:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 659159, 2023-07-21T22:42:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 827558, 2023-07-21T22:42:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 187799655, 2023-07-21T22:42:01.349 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 685671899, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 11, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 40.677441714, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 18.973198599, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 24.361693248, 2023-07-21T22:42:01.350 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 2.148908423, 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 827558, 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 144.508527500, 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000174620 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:01.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1316774, 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.215461783, 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000163 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 625392 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.352 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 561116, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 729288, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 161289514, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 792038119, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 30, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 27.477279877, 2023-07-21T22:42:01.353 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 12.834649095, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 19.466057912, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 2.370713773, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 729286, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 194.839780803, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000267165 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.354 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 558523, 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.155889185, 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000279 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 901376, 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 422032 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.355 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 697811, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 853345, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 202751529, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 400829575, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 254, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 11, 2023-07-21T22:42:01.356 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 35.964318840, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 13.913778383, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 25.585258816, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 1.057192482, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 853342, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 151.999332787, 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000178122 2023-07-21T22:42:01.357 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1467928, 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.260991857, 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000177 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.358 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1912048, 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 280080 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2023-07-21T22:42:01.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 151957, 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 146.258414165, 2023-07-21T22:42:01.360 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000962498 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 50094, 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 11.602725559, 2023-07-21T22:42:01.361 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000231619 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 351358, 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 291336, 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2023-07-21T22:42:01.362 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 291336, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2203.335922637, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.007562868 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 60022, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 30, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 1, 2023-07-21T22:42:01.363 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 4326, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 10, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 6, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 2193, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 1189, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 851, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 153, 2023-07-21T22:42:01.364 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 445, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 3155, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 2, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 2, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 6, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 552257, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 360614, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 59971, 2023-07-21T22:42:01.365 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 232, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 1, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 3958, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 31, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 28, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 1716, 2023-07-21T22:42:01.366 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 35, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 950, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 158, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 246, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2023-07-21T22:42:01.367 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 158, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 57, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 4, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 53, 2023-07-21T22:42:01.368 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 101, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 66, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 90454, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 126801, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 1027, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 128647, 2023-07-21T22:42:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 26443, 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 4197, 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 174562 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 1272, 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 244, 2023-07-21T22:42:01.370 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 139, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 404, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 4753, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 33, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 28727, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 28694, 2023-07-21T22:42:01.371 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2023-07-21T22:42:01.372 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 4, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 28, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 16, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2023-07-21T22:42:01.373 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 177492, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 176913, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 176913, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 579, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 2445, 2023-07-21T22:42:01.374 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 359, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 358, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 358, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 6, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 5, 2023-07-21T22:42:01.375 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 847378235, 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 849843262, 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 80722, 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 610.948950760, 2023-07-21T22:42:01.376 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.007568555 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 1739, 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 32327, 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 30588, 2023-07-21T22:42:01.377 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 116, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 531, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 415, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 2193, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 62298, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 60105, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 2, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 31638, 2023-07-21T22:42:01.378 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 31636, 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 85004, 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 207156 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 572195, 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 37058, 2023-07-21T22:42:01.379 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 351358, 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 4295, 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 109275, 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 29162, 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 208.374273309, 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.007145404 2023-07-21T22:42:01.380 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 81754, 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 893.739660749, 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.010932060 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.381 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.382 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 55695, 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 353.067369434, 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006339301 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2023-07-21T22:42:01.383 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2023-07-21T22:42:01.384 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.385 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2023-07-21T22:42:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.386 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2023-07-21T22:42:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.387 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2023-07-21T22:42:01.388 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 26, 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.017519449, 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000673824 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 119, 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.788154523, 2023-07-21T22:42:01.389 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006623147 2023-07-21T22:42:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2023-07-21T22:42:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.390 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 26, 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 18.786173459, 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.722545133 2023-07-21T22:42:01.391 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9, 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.014522111, 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001613567 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2023-07-21T22:42:01.392 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 89203, 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 443.994234765, 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004977346 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5688, 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 77.961689309, 2023-07-21T22:42:01.393 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.013706344 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2023-07-21T22:42:01.394 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 44, 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.144721559, 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.026016399 2023-07-21T22:42:01.395 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 20, 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 20.863614177, 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 1.043180708 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2023-07-21T22:42:01.396 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 90, 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.872811112, 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.020809012 2023-07-21T22:42:01.397 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 800, 2023-07-21T22:42:01.398 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2.323777536, 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002904721 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.399 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.400 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.400 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2023-07-21T22:42:01.400 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6, 2023-07-21T22:42:01.400 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.018918628, 2023-07-21T22:42:01.400 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003153104 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.401 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 28694, 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 180.368482517, 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006285930 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:42:01.402 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2023-07-21T22:42:01.403 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 5, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 3, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 17768, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 8884, 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1522 2023-07-21T22:42:01.404 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 360, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 360, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:42:01.405 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:42:01.406 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:42:01.407 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:42:01.408 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:42:01.409 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 655459, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 1267, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 176, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 2, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2023-07-21T22:42:01.410 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8160, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 186, 2023-07-21T22:42:01.411 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 6174455, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 51973, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2023-07-21T22:42:01.412 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 133705, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 881738742, 2023-07-21T22:42:01.413 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 37, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 133705, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 133705, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2705.005704479, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.020231148 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2023-07-21T22:42:01.414 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 133705, 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 174380 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 133705, 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 1337, 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 132368, 2023-07-21T22:42:01.415 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 4110, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 14985, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1078, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 81796, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 1071, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2023-07-21T22:42:01.416 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 10, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 29650, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 172, 2023-07-21T22:42:01.417 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 29970, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2023-07-21T22:42:01.418 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2023-07-21T22:42:01.419 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2023-07-21T22:42:01.420 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2023-07-21T22:42:01.421 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 130, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 49, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2023-07-21T22:42:01.422 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 8842, 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 128, 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 2568 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 388090 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.424 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 6, 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 28694, 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2023-07-21T22:42:01.425 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1918082, 2023-07-21T22:42:01.426 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 387132738, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1918082, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1918082, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 387132738, 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:01.427 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:01.428 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 133705, 2023-07-21T22:42:01.429 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 884644333, 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 133705, 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 884644333, 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.430 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:01.431 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 133705, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 133705, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 133705, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 133705, 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:01.432 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:01.433 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 28694, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2908014, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 28694, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:01.434 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1074, 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2908014, 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:01.435 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55f16581a0a0": { 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 84, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 177492, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 845648958, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:01.436 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 177492, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 80722, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 845648874, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:01.437 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:01.438 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:01.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.438 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:01.438 INFO:teuthology.orchestra.run.smithi194.stdout:} 2023-07-21T22:42:01.823 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:01 smithi194 ceph-mon[98288]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-07-21T22:42:01.824 INFO:tasks.check_counter:Daemon mds.i mds.exported=28 2023-07-21T22:42:01.825 INFO:tasks.check_counter:Daemon mds.i mds.imported=35 2023-07-21T22:42:01.825 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1272 2023-07-21T22:42:01.825 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=244 2023-07-21T22:42:01.825 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-07-21T22:42:01.826 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=26 2023-07-21T22:42:01.826 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=20 2023-07-21T22:42:01.826 DEBUG:tasks.check_counter:Getting stats from l 2023-07-21T22:42:01.827 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid de1cec7a-2812-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-21T22:42:01.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:01 smithi134 ceph-mon[97873]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-07-21T22:42:01.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:01 smithi093 ceph-mon[96579]: pgmap v815: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 15 op/s 2023-07-21T22:42:02.379 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:42:03.697 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2023-07-21T22:42:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:42:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2936, 2023-07-21T22:42:03.697 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2854, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 199709664, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 996152, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.380111120, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.075444283, 2023-07-21T22:42:03.698 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.206272559, 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.086634251, 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2853, 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.344833061, 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000120866 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.699 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1844064, 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 262192 2023-07-21T22:42:03.700 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1427, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1762, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 365296612, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1174405, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 9, 2023-07-21T22:42:03.701 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 7, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.327895978, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.036567566, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.237699533, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.040679031, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1759, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.176035126, 2023-07-21T22:42:03.702 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000100076 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T22:42:03.703 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 731168 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2814, 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2814, 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 572265053, 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 874365, 2023-07-21T22:42:03.704 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.487064758, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.057713253, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.320590751, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.091955817, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2813, 2023-07-21T22:42:03.705 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.281182178, 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000099958 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.706 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2023-07-21T22:42:03.707 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 10833, 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.571622443, 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000052766 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.708 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.709 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.710 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2023-07-21T22:42:03.711 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 21307, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 6252, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 14815, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 240, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 12375, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 20006, 2023-07-21T22:42:03.712 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:42:03.713 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2023-07-21T22:42:03.714 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2023-07-21T22:42:03.715 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2023-07-21T22:42:03.716 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:42:03.717 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2023-07-21T22:42:03.718 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2023-07-21T22:42:03.719 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:42:03.720 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.721 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2023-07-21T22:42:03.722 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 130, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 604004635, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 0, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 1137735069, 2023-07-21T22:42:03.723 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 187250 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.724 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 1245, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 41373, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 40128, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 45, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 110, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 65, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 21307, 2023-07-21T22:42:03.725 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 41883, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 20576, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 80164, 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 207156 2023-07-21T22:42:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2023-07-21T22:42:03.727 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.728 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.729 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.730 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.731 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.732 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.733 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.734 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2023-07-21T22:42:03.735 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.736 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2023-07-21T22:42:03.737 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.738 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2023-07-21T22:42:03.739 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.740 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.741 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2023-07-21T22:42:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.742 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2023-07-21T22:42:03.743 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2023-07-21T22:42:03.744 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.745 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2023-07-21T22:42:03.746 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.747 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2023-07-21T22:42:03.748 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 3, 2023-07-21T22:42:03.749 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 1, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 3, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 4207 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2023-07-21T22:42:03.750 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:42:03.751 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:42:03.752 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:42:03.753 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2023-07-21T22:42:03.754 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:42:03.755 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 293275, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 1599, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2023-07-21T22:42:03.756 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 8160, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 186, 2023-07-21T22:42:03.757 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 18318007, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 121144, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2023-07-21T22:42:03.758 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 6595, 2023-07-21T22:42:03.759 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 6595, 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6595, 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 20.135602345, 2023-07-21T22:42:03.760 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003053161 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6595, 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 6596 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 6595, 2023-07-21T22:42:03.761 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 6595, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 3552, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 3041, 2023-07-21T22:42:03.762 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2023-07-21T22:42:03.763 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:42:03.764 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2023-07-21T22:42:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2023-07-21T22:42:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2023-07-21T22:42:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:42:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2023-07-21T22:42:03.765 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:42:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2023-07-21T22:42:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2023-07-21T22:42:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2023-07-21T22:42:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2023-07-21T22:42:03.766 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2023-07-21T22:42:03.767 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 130, 2023-07-21T22:42:03.768 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 48, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 11, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 1, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2023-07-21T22:42:03.769 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2023-07-21T22:42:03.770 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2023-07-21T22:42:03.771 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:42:03.772 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 7173, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 1136593685, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 7173, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:03.773 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 7173, 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1136593685, 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.774 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:03.775 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 6595, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1133614591, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 6595, 2023-07-21T22:42:03.776 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1133614591, 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2023-07-21T22:42:03.777 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 6595, 2023-07-21T22:42:03.778 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 6595, 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 6595, 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 6595, 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.779 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:03.780 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2023-07-21T22:42:03.781 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x559df550a460": { 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2023-07-21T22:42:03.782 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2023-07-21T22:42:03.783 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2023-07-21T22:42:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2023-07-21T22:42:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2023-07-21T22:42:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2023-07-21T22:42:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2023-07-21T22:42:03.784 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2023-07-21T22:42:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2023-07-21T22:42:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.785 INFO:teuthology.orchestra.run.smithi194.stdout: } 2023-07-21T22:42:03.785 INFO:teuthology.orchestra.run.smithi194.stdout:} 2023-07-21T22:42:03.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:03 smithi194 ceph-mon[98288]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-07-21T22:42:03.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:03 smithi134 ceph-mon[97873]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-07-21T22:42:03.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:03 smithi093 ceph-mon[96579]: pgmap v816: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-07-21T22:42:04.226 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:42:04.227 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:42:04.228 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-21T22:42:04.238 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:42:04.238 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:42:04.239 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:42:04.277 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:42:04.278 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:42:04.339 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:42:04.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 -- 172.21.15.93:0/2735458205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a080ff380 msgr2=0x7f3a0810b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/2735458205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a080ff380 0x7f3a0810b810 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000b0a0 tx=0x7f3a0000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 -- 172.21.15.93:0/2735458205 shutdown_connections 2023-07-21T22:42:04.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/2735458205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a080ff380 0x7f3a0810b810 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/2735458205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a081049e0 0x7f3a08104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/2735458205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080fe9d0 0x7f3a080fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.486+0000 7f3a0eda6700 1 -- 172.21.15.93:0/2735458205 >> 172.21.15.93:0/2735458205 conn(0x7f3a080fa490 msgr2=0x7f3a080fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:04.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- 172.21.15.93:0/2735458205 shutdown_connections 2023-07-21T22:42:04.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- 172.21.15.93:0/2735458205 wait complete. 2023-07-21T22:42:04.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 Processor -- start 2023-07-21T22:42:04.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- start start 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a080fe9d0 0x7f3a08196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080ff380 0x7f3a08196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081978e0 con 0x7f3a081049e0 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081a2050 con 0x7f3a080ff380 2023-07-21T22:42:04.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.487+0000 7f3a0eda6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a081a21c0 con 0x7f3a080fe9d0 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0cb42700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a080fe9d0 0x7f3a08196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50462/0 (socket says 172.21.15.93:50462) 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 -- 172.21.15.93:0/1779966761 learned_addr learned my addr 172.21.15.93:0/1779966761 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a07fff700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080ff380 0x7f3a08196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 -- 172.21.15.93:0/1779966761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a080fe9d0 msgr2=0x7f3a08196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a080fe9d0 0x7f3a08196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 -- 172.21.15.93:0/1779966761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080ff380 msgr2=0x7f3a08196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080ff380 0x7f3a08196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 -- 172.21.15.93:0/1779966761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a00009d00 con 0x7f3a081049e0 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0d343700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009fd0 tx=0x7f3a0000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000e040 con 0x7f3a081049e0 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081a2440 con 0x7f3a081049e0 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a00004de0 con 0x7f3a081049e0 2023-07-21T22:42:04.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.488+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0001b6b0 con 0x7f3a081049e0 2023-07-21T22:42:04.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.489+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081a28c0 con 0x7f3a081049e0 2023-07-21T22:42:04.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.489+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08067300 con 0x7f3a081049e0 2023-07-21T22:42:04.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.491+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3a00003a00 con 0x7f3a081049e0 2023-07-21T22:42:04.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.491+0000 7f3a05ffb700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 0x7f39f80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.491+0000 7f3a0cb42700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 0x7f39f80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.491+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3a000ce890 con 0x7f3a081049e0 2023-07-21T22:42:04.535 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.492+0000 7f3a0cb42700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 0x7f39f80a37b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a080ffb10 tx=0x7f39fc00b410 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:04.536 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.495+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3a00097d30 con 0x7f3a081049e0 2023-07-21T22:42:04.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.689+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a081a2c40 con 0x7f3a081049e0 2023-07-21T22:42:04.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.691+0000 7f3a05ffb700 1 -- 172.21.15.93:0/1779966761 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+22948 (secure 0 0 0) 0x7f3a00011390 con 0x7f3a081049e0 2023-07-21T22:42:04.732 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:42:04.732 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":130,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:41:17.425830+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/2266779369":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1882208449":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/503504133":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6801/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3365813988":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2153177670":"2023-07-22T22:16:25.946546+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:42:04.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.693+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 msgr2=0x7f39f80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.693+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 0x7f39f80a37b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a080ffb10 tx=0x7f39fc00b410 comp rx=0 tx=0).stop 2023-07-21T22:42:04.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.693+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 msgr2=0x7f3a081a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.693+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009fd0 tx=0x7f3a0000ce90 comp rx=0 tx=0).stop 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 shutdown_connections 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f39f80a1300 0x7f39f80a37b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a081049e0 0x7f3a081a1b10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a080fe9d0 0x7f3a08196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 --2- 172.21.15.93:0/1779966761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a080ff380 0x7f3a08196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 >> 172.21.15.93:0/1779966761 conn(0x7f3a080fa490 msgr2=0x7f3a08102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:04.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 shutdown_connections 2023-07-21T22:42:04.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.694+0000 7f3a0eda6700 1 -- 172.21.15.93:0/1779966761 wait complete. 2023-07-21T22:42:04.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:10:57.465993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:14:48.511614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:14:49.823437+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:42:04.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:42:04.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- 172.21.15.93:0/448803867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 msgr2=0x7f1ba00fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/448803867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba00fee60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009990 tx=0x7f1b9c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- 172.21.15.93:0/448803867 shutdown_connections 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/448803867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba00ff3a0 0x7f1ba010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/448803867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba0104a00 0x7f1ba0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/448803867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba00fee60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- 172.21.15.93:0/448803867 >> 172.21.15.93:0/448803867 conn(0x7f1ba00fa4d0 msgr2=0x7f1ba00fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- 172.21.15.93:0/448803867 shutdown_connections 2023-07-21T22:42:04.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- 172.21.15.93:0/448803867 wait complete. 2023-07-21T22:42:04.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 Processor -- start 2023-07-21T22:42:04.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.832+0000 7f1ba7e09700 1 -- start start 2023-07-21T22:42:04.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba00ff3a0 0x7f1ba019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba0104a00 0x7f1ba01959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba0196010 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba0196180 con 0x7f1ba0104a00 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba0196340 con 0x7f1ba00ff3a0 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50472/0 (socket says 172.21.15.93:50472) 2023-07-21T22:42:04.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 -- 172.21.15.93:0/33718064 learned_addr learned my addr 172.21.15.93:0/33718064 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba53a4700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba00ff3a0 0x7f1ba019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba63a6700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba0104a00 0x7f1ba01959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 -- 172.21.15.93:0/33718064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba00ff3a0 msgr2=0x7f1ba019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba00ff3a0 0x7f1ba019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 -- 172.21.15.93:0/33718064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba0104a00 msgr2=0x7f1ba01959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba0104a00 0x7f1ba01959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 -- 172.21.15.93:0/33718064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0095f0 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba5ba5700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000d980 tx=0x7f1b9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90009930 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b90010460 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba0196620 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000f620 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.833+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba01a2920 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.834+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba00676e0 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.835+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1b90010860 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.836+0000 7f1b96ffd700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 0x7f1b8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.836+0000 7f1ba53a4700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 0x7f1b8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.836+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1b900cc750 con 0x7f1ba00fe9f0 2023-07-21T22:42:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.836+0000 7f1ba53a4700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 0x7f1b8c0a3770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00fdfd0 tx=0x7f1b9c005fd0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:04.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:04.840+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1b90095b60 con 0x7f1ba00fe9f0 2023-07-21T22:42:05.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.029+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ba01a2cb0 con 0x7f1ba00fe9f0 2023-07-21T22:42:05.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.030+0000 7f1b96ffd700 1 -- 172.21.15.93:0/33718064 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b90099180 con 0x7f1ba00fe9f0 2023-07-21T22:42:05.070 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-07-21T22:42:05.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 msgr2=0x7f1b8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 0x7f1b8c0a3770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00fdfd0 tx=0x7f1b9c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 msgr2=0x7f1ba019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000d980 tx=0x7f1b9000dc90 comp rx=0 tx=0).stop 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 shutdown_connections 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ba0104a00 0x7f1ba01959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f1b8c0a12c0 0x7f1b8c0a3770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ba00fe9f0 0x7f1ba019b930 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 --2- 172.21.15.93:0/33718064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ba00ff3a0 0x7f1ba019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 >> 172.21.15.93:0/33718064 conn(0x7f1ba00fa4d0 msgr2=0x7f1ba00fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 shutdown_connections 2023-07-21T22:42:05.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.032+0000 7f1ba7e09700 1 -- 172.21.15.93:0/33718064 wait complete. 2023-07-21T22:42:05.084 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:42:05.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 -- 172.21.15.93:0/3207351947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 msgr2=0x7f3774101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 --2- 172.21.15.93:0/3207351947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774101070 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768009ca0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 -- 172.21.15.93:0/3207351947 shutdown_connections 2023-07-21T22:42:05.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 --2- 172.21.15.93:0/3207351947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 0x7f377410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 --2- 172.21.15.93:0/3207351947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774101070 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 --2- 172.21.15.93:0/3207351947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 -- 172.21.15.93:0/3207351947 >> 172.21.15.93:0/3207351947 conn(0x7f37740fa4b0 msgr2=0x7f37740fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 -- 172.21.15.93:0/3207351947 shutdown_connections 2023-07-21T22:42:05.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.168+0000 7f3778e2e700 1 -- 172.21.15.93:0/3207351947 wait complete. 2023-07-21T22:42:05.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 Processor -- start 2023-07-21T22:42:05.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 -- start start 2023-07-21T22:42:05.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 0x7f377419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419add0 con 0x7f3774100ca0 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419af40 con 0x7f377410a070 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3778e2e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419b100 con 0x7f37741015b0 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50486/0 (socket says 172.21.15.93:50486) 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 -- 172.21.15.93:0/2611591329 learned_addr learned my addr 172.21.15.93:0/2611591329 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3771d9b700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3772d9d700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 0x7f377419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 -- 172.21.15.93:0/2611591329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 msgr2=0x7f3774196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 -- 172.21.15.93:0/2611591329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 msgr2=0x7f377419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 0x7f377419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f377259c700 1 -- 172.21.15.93:0/2611591329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680095f0 con 0x7f3774100ca0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.169+0000 7f3771d9b700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f377259c700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f376800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376801d070 con 0x7f3774100ca0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377419b380 con 0x7f3774100ca0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376800fca0 con 0x7f3774100ca0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680176e0 con 0x7f3774100ca0 2023-07-21T22:42:05.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.170+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37741a2c20 con 0x7f3774100ca0 2023-07-21T22:42:05.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.171+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37741051f0 con 0x7f3774100ca0 2023-07-21T22:42:05.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.172+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3768017840 con 0x7f3774100ca0 2023-07-21T22:42:05.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.173+0000 7f37637fe700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 0x7f375c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.173+0000 7f3771d9b700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 0x7f375c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.173+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f37680cdf90 con 0x7f3774100ca0 2023-07-21T22:42:05.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.173+0000 7f3771d9b700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 0x7f375c0a3770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3774197400 tx=0x7f3764005f50 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.177+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3768097430 con 0x7f3774100ca0 2023-07-21T22:42:05.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.367+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37740fc6c0 con 0x7f3774100ca0 2023-07-21T22:42:05.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.367+0000 7f37637fe700 1 -- 172.21.15.93:0/2611591329 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) v1 ==== 104+0+11 (secure 0 0 0) 0x7f376809aa50 con 0x7f3774100ca0 2023-07-21T22:42:05.408 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-07-21T22:42:05.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.369+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 msgr2=0x7f375c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.369+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 0x7f375c0a3770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3774197400 tx=0x7f3764005f50 comp rx=0 tx=0).stop 2023-07-21T22:42:05.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.369+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 msgr2=0x7f3774195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.369+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f376800f7b0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 shutdown_connections 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377410a070 0x7f377419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f375c0a12c0 0x7f375c0a3770 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3774100ca0 0x7f3774195df0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 --2- 172.21.15.93:0/2611591329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37741015b0 0x7f3774196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 >> 172.21.15.93:0/2611591329 conn(0x7f37740fa4b0 msgr2=0x7f377410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 shutdown_connections 2023-07-21T22:42:05.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.370+0000 7f3778e2e700 1 -- 172.21.15.93:0/2611591329 wait complete. 2023-07-21T22:42:05.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:42:05.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- 172.21.15.93:0/544105262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800ff580 msgr2=0x7ff3800ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 --2- 172.21.15.93:0/544105262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800ff580 0x7ff3800ff950 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7ff3740099c0 tx=0x7ff374009cd0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- 172.21.15.93:0/544105262 shutdown_connections 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 --2- 172.21.15.93:0/544105262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff380103ce0 0x7ff38010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 --2- 172.21.15.93:0/544105262 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800ff580 0x7ff3800ff950 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 --2- 172.21.15.93:0/544105262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3800fff20 0x7ff380103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- 172.21.15.93:0/544105262 >> 172.21.15.93:0/544105262 conn(0x7ff380076200 msgr2=0x7ff380076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- 172.21.15.93:0/544105262 shutdown_connections 2023-07-21T22:42:05.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- 172.21.15.93:0/544105262 wait complete. 2023-07-21T22:42:05.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 Processor -- start 2023-07-21T22:42:05.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.506+0000 7ff386b6a700 1 -- start start 2023-07-21T22:42:05.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3800ff580 0x7ff380195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff380103ce0 0x7ff38019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38019ada0 con 0x7ff3800fff20 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38019af10 con 0x7ff380103ce0 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38019b0d0 con 0x7ff3800ff580 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50502/0 (socket says 172.21.15.93:50502) 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 -- 172.21.15.93:0/2770924628 learned_addr learned my addr 172.21.15.93:0/2770924628 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff385107700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff380103ce0 0x7ff38019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff384906700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3800ff580 0x7ff380195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 -- 172.21.15.93:0/2770924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3800ff580 msgr2=0x7ff380195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3800ff580 0x7ff380195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 -- 172.21.15.93:0/2770924628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff380103ce0 msgr2=0x7ff38019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff380103ce0 0x7ff38019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 -- 172.21.15.93:0/2770924628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740095f0 con 0x7ff3800fff20 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37ffff700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00d950 tx=0x7ff36c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c009930 con 0x7ff3800fff20 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38019b3b0 con 0x7ff3800fff20 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36c010460 con 0x7ff3800fff20 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.507+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00f620 con 0x7ff3800fff20 2023-07-21T22:42:05.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.508+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3801a2bf0 con 0x7ff3800fff20 2023-07-21T22:42:05.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.508+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff380066a00 con 0x7ff3800fff20 2023-07-21T22:42:05.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.510+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff36c010860 con 0x7ff3800fff20 2023-07-21T22:42:05.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.510+0000 7ff37dffb700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 0x7ff3700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.510+0000 7ff384906700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 0x7ff3700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.510+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff36c0cc8c0 con 0x7ff3800fff20 2023-07-21T22:42:05.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.511+0000 7ff384906700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 0x7ff3700a3770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff374009fd0 tx=0x7ff374005c00 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.514+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff36c095cd0 con 0x7ff3800fff20 2023-07-21T22:42:05.687 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:05 smithi093 ceph-mon[96579]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 98 KiB/s wr, 27 op/s 2023-07-21T22:42:05.687 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1779966761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:05.687 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/33718064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.688 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:05 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2611591329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.702+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3801a2ee0 con 0x7ff3800fff20 2023-07-21T22:42:05.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.702+0000 7ff37dffb700 1 -- 172.21.15.93:0/2770924628 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff36c0170a0 con 0x7ff3800fff20 2023-07-21T22:42:05.743 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-07-21T22:42:05.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 msgr2=0x7ff3700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 0x7ff3700a3770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff374009fd0 tx=0x7ff374005c00 comp rx=0 tx=0).stop 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 msgr2=0x7ff380196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00d950 tx=0x7ff36c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 shutdown_connections 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff380103ce0 0x7ff38019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3800ff580 0x7ff380195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7ff3700a12c0 0x7ff3700a3770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 --2- 172.21.15.93:0/2770924628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3800fff20 0x7ff380196300 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 >> 172.21.15.93:0/2770924628 conn(0x7ff380076200 msgr2=0x7ff3800fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 shutdown_connections 2023-07-21T22:42:05.746 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.704+0000 7ff386b6a700 1 -- 172.21.15.93:0/2770924628 wait complete. 2023-07-21T22:42:05.757 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T22:42:05.833 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:05 smithi194 ceph-mon[98288]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 98 KiB/s wr, 27 op/s 2023-07-21T22:42:05.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1779966761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:05.834 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/33718064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.835 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:05 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2611591329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.875 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:05 smithi134 ceph-mon[97873]: pgmap v817: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 98 KiB/s wr, 27 op/s 2023-07-21T22:42:05.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1779966761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:05.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/33718064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.876 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:05 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2611591329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:42:05.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 -- 172.21.15.93:0/1735345273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 msgr2=0x7fc19010b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 --2- 172.21.15.93:0/1735345273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc19010b790 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b0a0 tx=0x7fc18800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 -- 172.21.15.93:0/1735345273 shutdown_connections 2023-07-21T22:42:05.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 --2- 172.21.15.93:0/1735345273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc19010b790 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 --2- 172.21.15.93:0/1735345273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc1901001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 --2- 172.21.15.93:0/1735345273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 0x7fc190108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 -- 172.21.15.93:0/1735345273 >> 172.21.15.93:0/1735345273 conn(0x7fc1900742e0 msgr2=0x7fc1900746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:05.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 -- 172.21.15.93:0/1735345273 shutdown_connections 2023-07-21T22:42:05.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.838+0000 7fc195222700 1 -- 172.21.15.93:0/1735345273 wait complete. 2023-07-21T22:42:05.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.839+0000 7fc195222700 1 Processor -- start 2023-07-21T22:42:05.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.839+0000 7fc195222700 1 -- start start 2023-07-21T22:42:05.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc19019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 0x7fc19019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190194fd0 con 0x7fc1901093b0 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190195140 con 0x7fc190100710 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc195222700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc190195300 con 0x7fc1900ffe00 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50518/0 (socket says 172.21.15.93:50518) 2023-07-21T22:42:05.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 -- 172.21.15.93:0/975333179 learned_addr learned my addr 172.21.15.93:0/975333179 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18ed9d700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc19019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18e59c700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 0x7fc19019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 -- 172.21.15.93:0/975333179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 msgr2=0x7fc19019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc19019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 -- 172.21.15.93:0/975333179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 msgr2=0x7fc19019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 0x7fc19019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:05.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 -- 172.21.15.93:0/975333179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188009d00 con 0x7fc1901093b0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18ed9d700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc19019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.840+0000 7fc18f59e700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b780 tx=0x7fc18800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800e040 con 0x7fc1901093b0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc188004e60 con 0x7fc1901093b0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc190195580 con 0x7fc1901093b0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188005280 con 0x7fc1901093b0 2023-07-21T22:42:05.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1901a18e0 con 0x7fc1901093b0 2023-07-21T22:42:05.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.841+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc190103c70 con 0x7fc1901093b0 2023-07-21T22:42:05.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.843+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc18801d470 con 0x7fc1901093b0 2023-07-21T22:42:05.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.843+0000 7fc177fff700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 0x7fc1780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:05.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.843+0000 7fc18ed9d700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 0x7fc1780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:05.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.843+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc1880ce6f0 con 0x7fc1901093b0 2023-07-21T22:42:05.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.844+0000 7fc18ed9d700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 0x7fc1780a3770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc180009cb0 tx=0x7fc180009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:05.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:05.847+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc188097b90 con 0x7fc1901093b0 2023-07-21T22:42:06.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.050+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1900676e0 con 0x7fc1901093b0 2023-07-21T22:42:06.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.050+0000 7fc177fff700 1 -- 172.21.15.93:0/975333179 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v130) v1 ==== 81+0+1002 (secure 0 0 0) 0x7fc18809b1b0 con 0x7fc1901093b0 2023-07-21T22:42:06.091 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2266779369 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.091 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3858928377 2023-07-22T22:08:29.027813+0000 2023-07-21T22:42:06.091 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1882208449 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2824313704 2023-07-22T22:08:29.027813+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2873292277 2023-07-22T22:08:51.131041+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3733549091 2023-07-22T22:08:29.027813+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2048241181 2023-07-22T22:08:29.027813+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1829577156 2023-07-22T22:08:51.131041+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/854330056 2023-07-22T22:08:51.131041+0000 2023-07-21T22:42:06.092 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2721150511 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/3590233301 2023-07-22T22:08:51.131041+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/3590233301 2023-07-22T22:08:51.131041+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/503504133 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2721150511 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2048241181 2023-07-22T22:08:29.027813+0000 2023-07-21T22:42:06.093 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3365813988 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.094 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2153177670 2023-07-22T22:16:25.946546+0000 2023-07-21T22:42:06.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 msgr2=0x7fc1780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 0x7fc1780a3770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc180009cb0 tx=0x7fc180009260 comp rx=0 tx=0).stop 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 msgr2=0x7fc190194960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b780 tx=0x7fc18800be10 comp rx=0 tx=0).stop 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 shutdown_connections 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc1780a12c0 0x7fc1780a3770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1901093b0 0x7fc190194960 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:06.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1900ffe00 0x7fc19019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:06.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 --2- 172.21.15.93:0/975333179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc190100710 0x7fc19019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:06.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 >> 172.21.15.93:0/975333179 conn(0x7fc1900742e0 msgr2=0x7fc190100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:06.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 shutdown_connections 2023-07-21T22:42:06.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:06.053+0000 7fc195222700 1 -- 172.21.15.93:0/975333179 wait complete. 2023-07-21T22:42:06.096 INFO:teuthology.orchestra.run.smithi093.stderr:listed 17 entries 2023-07-21T22:42:06.105 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-21T22:42:06.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:06.106 DEBUG:teuthology.orchestra.run.smithi093:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-21T22:42:06.462 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:42:06.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:06.463 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-21T22:42:06.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:06.676 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-07-21T22:42:06.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:06 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2770924628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:42:06.877 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:06 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/975333179' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:42:06.926 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-21T22:42:06.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:06.927 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:42:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:06 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2770924628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:42:06.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:06 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/975333179' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:42:07.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:06 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2770924628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:42:07.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:06 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/975333179' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:42:07.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:07.183 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-07-21T22:42:07.345 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:42:07.346 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-07-21T22:42:07.346 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:42:07.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18750 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/079eaffdc2fc538cf0972a42979530599819e73c74cf0844e23836a8ca44e3ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/079eaffdc2fc538cf0972a42979530599819e73c74cf0844e23836a8ca44e3ec/diff,workdir=/var/lib/containers/storage/overlay/079eaffdc2fc538cf0972a42979530599819e73c74cf0844e23836a8ca44e3ec/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f738e41a6e224a857239e500c777007491696d82d6b0d6a093f7738b0012d5ba/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/f738e41a6e224a857239e500c777007491696d82d6b0d6a093f7738b0012d5ba/diff,workdir=/var/lib/containers/storage/overlay/f738e41a6e224a857239e500c777007491696d82d6b0d6a093f7738b0012d5ba/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/42c20f7c1a20d4ac4f94d7f74f30e45e1040b2d43336d2c56e9bbb807e65c572/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/42c20f7c1a20d4ac4f94d7f74f30e45e1040b2d43336d2c56e9bbb807e65c572/diff,workdir=/var/lib/containers/storage/overlay/42c20f7c1a20d4ac4f94d7f74f30e45e1040b2d43336d2c56e9bbb807e65c572/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/853111c139a9f10b1b44edf3d6e7e991524f8f62d9a37120d01675ffc0bb1897/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/853111c139a9f10b1b44edf3d6e7e991524f8f62d9a37120d01675ffc0bb1897/diff,workdir=/var/lib/containers/storage/overlay/853111c139a9f10b1b44edf3d6e7e991524f8f62d9a37120d01675ffc0bb1897/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b214a455a77f49996ce80226436cf55f46c158334d1ebf2a87e700bc3bd82737/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/b214a455a77f49996ce80226436cf55f46c158334d1ebf2a87e700bc3bd82737/diff,workdir=/var/lib/containers/storage/overlay/b214a455a77f49996ce80226436cf55f46c158334d1ebf2a87e700bc3bd82737/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d59e1001087c6b15dea7307c59c8a5c2be19db7ff78e48f506d2c3721e142e9f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/d59e1001087c6b15dea7307c59c8a5c2be19db7ff78e48f506d2c3721e142e9f/diff,workdir=/var/lib/containers/storage/overlay/d59e1001087c6b15dea7307c59c8a5c2be19db7ff78e48f506d2c3721e142e9f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5878d306d2346633a04dfb5248d8ee9c8b257b04d0d451ed471eb5625f08c2e9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/5878d306d2346633a04dfb5248d8ee9c8b257b04d0d451ed471eb5625f08c2e9/diff,workdir=/var/lib/containers/storage/overlay/5878d306d2346633a04dfb5248d8ee9c8b257b04d0d451ed471eb5625f08c2e9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6a24e04a79986175dee8f30d420a471a408a8f515f863ab252dfc25c0e4b10a9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/6a24e04a79986175dee8f30d420a471a408a8f515f863ab252dfc25c0e4b10a9/diff,workdir=/var/lib/containers/storage/overlay/6a24e04a79986175dee8f30d420a471a408a8f515f863ab252dfc25c0e4b10a9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db660ef886b3e6d031c37ea23d1e4e0ac44b176b4e7c8cd6c098f4a64bc75754/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/db660ef886b3e6d031c37ea23d1e4e0ac44b176b4e7c8cd6c098f4a64bc75754/diff,workdir=/var/lib/containers/storage/overlay/db660ef886b3e6d031c37ea23d1e4e0ac44b176b4e7c8cd6c098f4a64bc75754/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b932cc96901a88c16371ce5e069a2ac1059afa5039be7f3937f93b8417050782/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZVSFR2PQ7XGW65DMJTTGHIMRAV:/var/lib/containers/storage/overlay/l/AZEIUQSBN4EOSGTVEESBS3NK5P,upperdir=/var/lib/containers/storage/overlay/b932cc96901a88c16371ce5e069a2ac1059afa5039be7f3937f93b8417050782/diff,workdir=/var/lib/containers/storage/overlay/b932cc96901a88c16371ce5e069a2ac1059afa5039be7f3937f93b8417050782/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-21T22:42:07.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:07.383 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-07-21T22:42:07.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:42:07.440 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-07-21T22:42:07.568 INFO:teuthology.orchestra.run.smithi093.stderr:Cannot find device "ceph-brx" 2023-07-21T22:42:07.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:42:07.569 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-21T22:42:07.582 INFO:tasks.cephadm:Teardown begin 2023-07-21T22:42:07.582 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-21T22:42:07.583 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:42:07.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 -- 172.21.15.93:0/125871155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218100780 msgr2=0x7f6218104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 --2- 172.21.15.93:0/125871155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218100780 0x7f6218104740 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f62040099c0 tx=0x7f6204009cd0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 -- 172.21.15.93:0/125871155 shutdown_connections 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 --2- 172.21.15.93:0/125871155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218104e20 0x7f621810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 --2- 172.21.15.93:0/125871155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62180ffde0 0x7f62181001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 --2- 172.21.15.93:0/125871155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218100780 0x7f6218104740 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 -- 172.21.15.93:0/125871155 >> 172.21.15.93:0/125871155 conn(0x7f62180fb690 msgr2=0x7f62180fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:07.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 -- 172.21.15.93:0/125871155 shutdown_connections 2023-07-21T22:42:07.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.665+0000 7f621eda1700 1 -- 172.21.15.93:0/125871155 wait complete. 2023-07-21T22:42:07.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 Processor -- start 2023-07-21T22:42:07.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 -- start start 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62180ffde0 0x7f621819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218100780 0x7f621819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62181960b0 con 0x7f6218104e20 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6218196220 con 0x7f6218100780 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621eda1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62181963e0 con 0x7f62180ffde0 2023-07-21T22:42:07.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50530/0 (socket says 172.21.15.93:50530) 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 -- 172.21.15.93:0/1555473552 learned_addr learned my addr 172.21.15.93:0/1555473552 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 -- 172.21.15.93:0/1555473552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62180ffde0 msgr2=0x7f621819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62180ffde0 0x7f621819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 -- 172.21.15.93:0/1555473552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218100780 msgr2=0x7f621819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f6217fff700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218100780 0x7f621819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218100780 0x7f621819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 -- 172.21.15.93:0/1555473552 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040095f0 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f621d33e700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6210007ee0 tx=0x7f621000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.666+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000ddd0 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.667+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62181966c0 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.667+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621000f040 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.667+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210015400 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.667+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62181a29c0 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.668+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6218067750 con 0x7f6218104e20 2023-07-21T22:42:07.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.669+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6210004020 con 0x7f6218104e20 2023-07-21T22:42:07.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.669+0000 7f6215ffb700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 0x7f62080a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:07.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.669+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f62100cc670 con 0x7f6218104e20 2023-07-21T22:42:07.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.669+0000 7f621cb3d700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 0x7f62080a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:07.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.670+0000 7f621cb3d700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 0x7f62080a37b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f620c00a990 tx=0x7f620c005c80 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:07.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.673+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f621004e770 con 0x7f6218104e20 2023-07-21T22:42:07.946 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.906+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62181a2cb0 con 0x7f6218104e20 2023-07-21T22:42:07.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.907+0000 7f6215ffb700 1 -- 172.21.15.93:0/1555473552 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f6210095a80 con 0x7f6218104e20 2023-07-21T22:42:07.947 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:42:07.948 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":96,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:41:45.475577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_17619":{"gid":17619,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/4292712299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4292712299},{"type":"v1","addr":"172.21.15.134:6837","nonce":4292712299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":54,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:42:07.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 msgr2=0x7f62080a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 0x7f62080a37b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f620c00a990 tx=0x7f620c005c80 comp rx=0 tx=0).stop 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 msgr2=0x7f6218195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6210007ee0 tx=0x7f621000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 shutdown_connections 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f62080a1300 0x7f62080a37b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6218104e20 0x7f6218195a40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62180ffde0 0x7f621819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 --2- 172.21.15.93:0/1555473552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6218100780 0x7f621819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:07.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 >> 172.21.15.93:0/1555473552 conn(0x7f62180fb690 msgr2=0x7f62180fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:07.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 shutdown_connections 2023-07-21T22:42:07.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:07.909+0000 7f621eda1700 1 -- 172.21.15.93:0/1555473552 wait complete. 2023-07-21T22:42:07.951 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 96 2023-07-21T22:42:07.961 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:07 smithi093 ceph-mon[96579]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-07-21T22:42:07.962 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:42:08.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 -- 172.21.15.93:0/136459152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8680ff3e0 msgr2=0x7fc86810b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 --2- 172.21.15.93:0/136459152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8680ff3e0 0x7fc86810b870 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b0a0 tx=0x7fc86000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 -- 172.21.15.93:0/136459152 shutdown_connections 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 --2- 172.21.15.93:0/136459152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8680ff3e0 0x7fc86810b870 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 --2- 172.21.15.93:0/136459152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc868104a40 0x7fc868104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 --2- 172.21.15.93:0/136459152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680fea30 0x7fc8680feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 -- 172.21.15.93:0/136459152 >> 172.21.15.93:0/136459152 conn(0x7fc8680fa4f0 msgr2=0x7fc8680fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:08.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 -- 172.21.15.93:0/136459152 shutdown_connections 2023-07-21T22:42:08.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.043+0000 7fc86d34d700 1 -- 172.21.15.93:0/136459152 wait complete. 2023-07-21T22:42:08.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:07 smithi194 ceph-mon[98288]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-07-21T22:42:08.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 Processor -- start 2023-07-21T22:42:08.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 -- start start 2023-07-21T22:42:08.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 0x7fc868196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 0x7fc868197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681978d0 con 0x7fc868104a40 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681a20b0 con 0x7fc8680fea30 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc86d34d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681a21f0 con 0x7fc8680ff3e0 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50544/0 (socket says 172.21.15.93:50544) 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 -- 172.21.15.93:0/1627396805 learned_addr learned my addr 172.21.15.93:0/1627396805 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:08.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc866ffd700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 0x7fc868196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8667fc700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 0x7fc868197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 -- 172.21.15.93:0/1627396805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 msgr2=0x7fc868197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 0x7fc868197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 -- 172.21.15.93:0/1627396805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 msgr2=0x7fc868196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 0x7fc868196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8677fe700 1 -- 172.21.15.93:0/1627396805 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860009d00 con 0x7fc868104a40 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc8667fc700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 0x7fc868197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:08.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.044+0000 7fc866ffd700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 0x7fc868196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc8677fe700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b070 tx=0x7fc86000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000e040 con 0x7fc868104a40 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc860004020 con 0x7fc868104a40 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8681a2440 con 0x7fc868104a40 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86001b6b0 con 0x7fc868104a40 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.045+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8681a28c0 con 0x7fc868104a40 2023-07-21T22:42:08.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.046+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868067750 con 0x7fc868104a40 2023-07-21T22:42:08.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.047+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc860004190 con 0x7fc868104a40 2023-07-21T22:42:08.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.047+0000 7fc84ffff700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 0x7fc8500a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.048+0000 7fc866ffd700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 0x7fc8500a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.048+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc8600ce340 con 0x7fc868104a40 2023-07-21T22:42:08.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.048+0000 7fc866ffd700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 0x7fc8500a37b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005950 tx=0x7fc85c0058e0 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.052+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc8600977e0 con 0x7fc868104a40 2023-07-21T22:42:08.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:07 smithi134 ceph-mon[97873]: pgmap v818: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 121 KiB/s wr, 29 op/s 2023-07-21T22:42:08.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.281+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc868068ec0 con 0x7fc868104a40 2023-07-21T22:42:08.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.281+0000 7fc84ffff700 1 -- 172.21.15.93:0/1627396805 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fc860011390 con 0x7fc868104a40 2023-07-21T22:42:08.322 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:42:08.322 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3390846004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3390846004},{"type":"v1","addr":"172.21.15.93:6835","nonce":3390846004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3934873649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":3934873649},{"type":"v1","addr":"172.21.15.93:6837","nonce":3934873649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":96,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:14:50.635520+0000","modified":"2023-07-21T22:41:45.475577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24484,"mds_1":24482,"mds_2":24460,"mds_3":24448,"mds_4":14589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.93:6839/3482994338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":3482994338},{"type":"v1","addr":"172.21.15.93:6839","nonce":3482994338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14589":{"gid":14589,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.93:6841/4095455709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4095455709},{"type":"v1","addr":"172.21.15.93:6841","nonce":4095455709}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_17619":{"gid":17619,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/4292712299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":4292712299},{"type":"v1","addr":"172.21.15.134:6837","nonce":4292712299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/2616544286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2616544286},{"type":"v1","addr":"172.21.15.134:6833","nonce":2616544286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24416":{"gid":24416,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/3262717838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3262717838},{"type":"v1","addr":"172.21.15.134:6835","nonce":3262717838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24421":{"gid":24421,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/3279794938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":3279794938},{"type":"v1","addr":"172.21.15.194:6833","nonce":3279794938}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":54,"addr":"172.21.15.194:6835/127707947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":127707947},{"type":"v1","addr":"172.21.15.194:6835","nonce":127707947}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":53,"addr":"172.21.15.194:6837/714728190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":714728190},{"type":"v1","addr":"172.21.15.194:6837","nonce":714728190}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.134:6839/3043386858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3043386858},{"type":"v1","addr":"172.21.15.134:6839","nonce":3043386858}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24484":{"gid":24484,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.194:6839/1819166614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":1819166614},{"type":"v1","addr":"172.21.15.194:6839","nonce":1819166614}]},"join_fscid":-1,"export_targets":[1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:42:08.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 msgr2=0x7fc8500a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 0x7fc8500a37b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005950 tx=0x7fc85c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 msgr2=0x7fc8681a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc86000b070 tx=0x7fc86000cf40 comp rx=0 tx=0).stop 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 shutdown_connections 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fc8500a1300 0x7fc8500a37b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc868104a40 0x7fc8681a1b70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680fea30 0x7fc868196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 --2- 172.21.15.93:0/1627396805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8680ff3e0 0x7fc868197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 >> 172.21.15.93:0/1627396805 conn(0x7fc8680fa4f0 msgr2=0x7fc8681058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:08.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 shutdown_connections 2023-07-21T22:42:08.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.284+0000 7fc86d34d700 1 -- 172.21.15.93:0/1627396805 wait complete. 2023-07-21T22:42:08.326 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 96 2023-07-21T22:42:08.336 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:42:08.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 -- 172.21.15.93:0/2933795326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99600ffde0 msgr2=0x7f99601001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 --2- 172.21.15.93:0/2933795326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99600ffde0 0x7f99601001b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f995c0099c0 tx=0x7f995c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 -- 172.21.15.93:0/2933795326 shutdown_connections 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 --2- 172.21.15.93:0/2933795326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 0x7f996010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 --2- 172.21.15.93:0/2933795326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f99600ffde0 0x7f99601001b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 --2- 172.21.15.93:0/2933795326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9960100780 0x7f9960104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.416+0000 7f9967300700 1 -- 172.21.15.93:0/2933795326 >> 172.21.15.93:0/2933795326 conn(0x7f99600fb690 msgr2=0x7f99600fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- 172.21.15.93:0/2933795326 shutdown_connections 2023-07-21T22:42:08.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- 172.21.15.93:0/2933795326 wait complete. 2023-07-21T22:42:08.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 Processor -- start 2023-07-21T22:42:08.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- start start 2023-07-21T22:42:08.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99600ffde0 0x7f996019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 0x7f9960195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99601960a0 con 0x7f9960100780 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9960196210 con 0x7f9960104e20 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f9967300700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99601963d0 con 0x7f99600ffde0 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f996489b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f996489b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50556/0 (socket says 172.21.15.93:50556) 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.417+0000 7f996489b700 1 -- 172.21.15.93:0/2673303607 learned_addr learned my addr 172.21.15.93:0/2673303607 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:08.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996509c700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99600ffde0 0x7f996019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996589d700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 0x7f9960195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 -- 172.21.15.93:0/2673303607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99600ffde0 msgr2=0x7f996019b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99600ffde0 0x7f996019b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 -- 172.21.15.93:0/2673303607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 msgr2=0x7f9960195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 0x7f9960195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 -- 172.21.15.93:0/2673303607 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0095f0 con 0x7f9960100780 2023-07-21T22:42:08.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f996489b700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f995000c970 tx=0x7f995000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950007b00 con 0x7f9960100780 2023-07-21T22:42:08.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995000ce60 con 0x7f9960100780 2023-07-21T22:42:08.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99601966b0 con 0x7f9960100780 2023-07-21T22:42:08.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950005440 con 0x7f9960100780 2023-07-21T22:42:08.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.418+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99601a29b0 con 0x7f9960100780 2023-07-21T22:42:08.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.419+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9960067750 con 0x7f9960100780 2023-07-21T22:42:08.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.420+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9950004750 con 0x7f9960100780 2023-07-21T22:42:08.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.420+0000 7f99567fc700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 0x7f994c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.421+0000 7f996509c700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 0x7f994c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.421+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f99500cc880 con 0x7f9960100780 2023-07-21T22:42:08.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.421+0000 7f996509c700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 0x7f994c0a37b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f995c000c00 tx=0x7f995c005c40 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.425+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9950095c90 con 0x7f9960100780 2023-07-21T22:42:08.656 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.616+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9960196f40 con 0x7f9960100780 2023-07-21T22:42:08.657 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.617+0000 7f99567fc700 1 -- 172.21.15.93:0/2673303607 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+22948 (secure 0 0 0) 0x7f995001d090 con 0x7f9960100780 2023-07-21T22:42:08.658 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-07-21T22:42:08.658 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":130,"fsid":"de1cec7a-2812-11ee-9b34-001a4aab830c","created":"2023-07-21T22:08:02.004166+0000","modified":"2023-07-21T22:41:17.425830+0000","last_up_change":"2023-07-21T22:14:09.362275+0000","last_in_change":"2023-07-21T22:13:51.508548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:10:57.465993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:14:48.511614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:14:49.823437+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a67ed18-5188-4868-8069-de6b0b8165cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6803","nonce":2127775933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6805","nonce":2127775933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6809","nonce":2127775933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2127775933},{"type":"v1","addr":"172.21.15.93:6807","nonce":2127775933}]},"public_addr":"172.21.15.93:6803/2127775933","cluster_addr":"172.21.15.93:6805/2127775933","heartbeat_back_addr":"172.21.15.93:6809/2127775933","heartbeat_front_addr":"172.21.15.93:6807/2127775933","state":["exists","up"]},{"osd":1,"uuid":"6567d033-de01-4296-ab88-b6781b7519ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6801","nonce":2449999795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6803","nonce":2449999795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6807","nonce":2449999795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":2449999795},{"type":"v1","addr":"172.21.15.134:6805","nonce":2449999795}]},"public_addr":"172.21.15.134:6801/2449999795","cluster_addr":"172.21.15.134:6803/2449999795","heartbeat_back_addr":"172.21.15.134:6807/2449999795","heartbeat_front_addr":"172.21.15.134:6805/2449999795","state":["exists","up"]},{"osd":2,"uuid":"4704710d-70ac-4f07-8dbd-509266a7d1c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6801","nonce":866955250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6803","nonce":866955250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6807","nonce":866955250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":866955250},{"type":"v1","addr":"172.21.15.194:6805","nonce":866955250}]},"public_addr":"172.21.15.194:6801/866955250","cluster_addr":"172.21.15.194:6803/866955250","heartbeat_back_addr":"172.21.15.194:6807/866955250","heartbeat_front_addr":"172.21.15.194:6805/866955250","state":["exists","up"]},{"osd":3,"uuid":"5622ee06-de4f-4d56-9829-efced7a3fbcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6811","nonce":921753281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6813","nonce":921753281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6817","nonce":921753281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":921753281},{"type":"v1","addr":"172.21.15.93:6815","nonce":921753281}]},"public_addr":"172.21.15.93:6811/921753281","cluster_addr":"172.21.15.93:6813/921753281","heartbeat_back_addr":"172.21.15.93:6817/921753281","heartbeat_front_addr":"172.21.15.93:6815/921753281","state":["exists","up"]},{"osd":4,"uuid":"8bfd6964-22c3-4847-b0a2-095f09b257e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6809","nonce":2960934129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6811","nonce":2960934129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6815","nonce":2960934129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2960934129},{"type":"v1","addr":"172.21.15.134:6813","nonce":2960934129}]},"public_addr":"172.21.15.134:6809/2960934129","cluster_addr":"172.21.15.134:6811/2960934129","heartbeat_back_addr":"172.21.15.134:6815/2960934129","heartbeat_front_addr":"172.21.15.134:6813/2960934129","state":["exists","up"]},{"osd":5,"uuid":"ece1a413-eba9-4d35-bc27-352be61010b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6809","nonce":1989091547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6811","nonce":1989091547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6815","nonce":1989091547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1989091547},{"type":"v1","addr":"172.21.15.194:6813","nonce":1989091547}]},"public_addr":"172.21.15.194:6809/1989091547","cluster_addr":"172.21.15.194:6811/1989091547","heartbeat_back_addr":"172.21.15.194:6815/1989091547","heartbeat_front_addr":"172.21.15.194:6813/1989091547","state":["exists","up"]},{"osd":6,"uuid":"f05d761d-7844-4e6d-99e7-94acab20daad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6819","nonce":4017577032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6821","nonce":4017577032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6825","nonce":4017577032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":4017577032},{"type":"v1","addr":"172.21.15.93:6823","nonce":4017577032}]},"public_addr":"172.21.15.93:6819/4017577032","cluster_addr":"172.21.15.93:6821/4017577032","heartbeat_back_addr":"172.21.15.93:6825/4017577032","heartbeat_front_addr":"172.21.15.93:6823/4017577032","state":["exists","up"]},{"osd":7,"uuid":"1147542a-1a5b-4935-99ae-2c196c6bb096","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6817","nonce":4117548000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6819","nonce":4117548000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6823","nonce":4117548000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":4117548000},{"type":"v1","addr":"172.21.15.134:6821","nonce":4117548000}]},"public_addr":"172.21.15.134:6817/4117548000","cluster_addr":"172.21.15.134:6819/4117548000","heartbeat_back_addr":"172.21.15.134:6823/4117548000","heartbeat_front_addr":"172.21.15.134:6821/4117548000","state":["exists","up"]},{"osd":8,"uuid":"f34cab3d-d40c-47b9-a691-b95f36965de5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6817","nonce":773360431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6819","nonce":773360431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6823","nonce":773360431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":773360431},{"type":"v1","addr":"172.21.15.194:6821","nonce":773360431}]},"public_addr":"172.21.15.194:6817/773360431","cluster_addr":"172.21.15.194:6819/773360431","heartbeat_back_addr":"172.21.15.194:6823/773360431","heartbeat_front_addr":"172.21.15.194:6821/773360431","state":["exists","up"]},{"osd":9,"uuid":"7aca17f8-875e-421f-a4b8-9b85656aa761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6827","nonce":692701270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6829","nonce":692701270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6833","nonce":692701270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":692701270},{"type":"v1","addr":"172.21.15.93:6831","nonce":692701270}]},"public_addr":"172.21.15.93:6827/692701270","cluster_addr":"172.21.15.93:6829/692701270","heartbeat_back_addr":"172.21.15.93:6833/692701270","heartbeat_front_addr":"172.21.15.93:6831/692701270","state":["exists","up"]},{"osd":10,"uuid":"ef648196-6eb5-4d15-bb0f-b6dedf1c03b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6825","nonce":51617631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6827","nonce":51617631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6831","nonce":51617631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":51617631},{"type":"v1","addr":"172.21.15.134:6829","nonce":51617631}]},"public_addr":"172.21.15.134:6825/51617631","cluster_addr":"172.21.15.134:6827/51617631","heartbeat_back_addr":"172.21.15.134:6831/51617631","heartbeat_front_addr":"172.21.15.134:6829/51617631","state":["exists","up"]},{"osd":11,"uuid":"68ec4cd3-029b-45b3-af2a-16ef9ea3be47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6825","nonce":4036125050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6827","nonce":4036125050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6831","nonce":4036125050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":4036125050},{"type":"v1","addr":"172.21.15.194:6829","nonce":4036125050}]},"public_addr":"172.21.15.194:6825/4036125050","cluster_addr":"172.21.15.194:6827/4036125050","heartbeat_back_addr":"172.21.15.194:6831/4036125050","heartbeat_front_addr":"172.21.15.194:6829/4036125050","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:12.801757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:34.022815+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:10:55.637522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:17.762503+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:11:39.004935+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:00.775612+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:22.126417+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:12:43.128947+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:04.313216+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:25.312112+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:13:46.450048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:07.713416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/2266779369":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/3858928377":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1882208449":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2824313704":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/2873292277":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/3733549091":"2023-07-22T22:08:29.027813+0000","172.21.15.93:6801/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/1829577156":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/854330056":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6800/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:6801/3590233301":"2023-07-22T22:08:51.131041+0000","172.21.15.93:0/503504133":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6801/2721150511":"2023-07-22T22:16:25.946546+0000","172.21.15.93:6800/2048241181":"2023-07-22T22:08:29.027813+0000","172.21.15.93:0/3365813988":"2023-07-22T22:16:25.946546+0000","172.21.15.93:0/2153177670":"2023-07-22T22:16:25.946546+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:42:08.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 msgr2=0x7f994c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 0x7f994c0a37b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f995c000c00 tx=0x7f995c005c40 comp rx=0 tx=0).stop 2023-07-21T22:42:08.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 msgr2=0x7f996019bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f995000c970 tx=0x7f995000cc80 comp rx=0 tx=0).stop 2023-07-21T22:42:08.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 shutdown_connections 2023-07-21T22:42:08.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9960104e20 0x7f9960195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99600ffde0 0x7f996019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f994c0a1300 0x7f994c0a37b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 --2- 172.21.15.93:0/2673303607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9960100780 0x7f996019bf00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 >> 172.21.15.93:0/2673303607 conn(0x7f99600fb690 msgr2=0x7f9960103720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:08.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 shutdown_connections 2023-07-21T22:42:08.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.619+0000 7f9967300700 1 -- 172.21.15.93:0/2673303607 wait complete. 2023-07-21T22:42:08.672 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-21T22:42:08.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 -- 172.21.15.93:0/740258300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fee10 msgr2=0x7f07a8102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 --2- 172.21.15.93:0/740258300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fee10 0x7f07a8102d50 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f07a40099c0 tx=0x7f07a4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 -- 172.21.15.93:0/740258300 shutdown_connections 2023-07-21T22:42:08.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 --2- 172.21.15.93:0/740258300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a8103430 0x7f07a810b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 --2- 172.21.15.93:0/740258300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a80fe470 0x7f07a80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 --2- 172.21.15.93:0/740258300 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fee10 0x7f07a8102d50 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 -- 172.21.15.93:0/740258300 >> 172.21.15.93:0/740258300 conn(0x7f07a8076200 msgr2=0x7f07a8076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:08.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 -- 172.21.15.93:0/740258300 shutdown_connections 2023-07-21T22:42:08.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.754+0000 7f07b0205700 1 -- 172.21.15.93:0/740258300 wait complete. 2023-07-21T22:42:08.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 Processor -- start 2023-07-21T22:42:08.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 -- start start 2023-07-21T22:42:08.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.798 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a80fee10 0x7f07a819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a8103430 0x7f07a8195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a81960e0 con 0x7f07a80fe470 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8196250 con 0x7f07a80fee10 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.755+0000 7f07b0205700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8196410 con 0x7f07a8103430 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50568/0 (socket says 172.21.15.93:50568) 2023-07-21T22:42:08.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 -- 172.21.15.93:0/1027784965 learned_addr learned my addr 172.21.15.93:0/1027784965 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07ad7a0700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a80fee10 0x7f07a819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07ae7a2700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a8103430 0x7f07a8195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 -- 172.21.15.93:0/1027784965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a8103430 msgr2=0x7f07a8195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a8103430 0x7f07a8195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 -- 172.21.15.93:0/1027784965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a80fee10 msgr2=0x7f07a819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:08.800 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a80fee10 0x7f07a819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 -- 172.21.15.93:0/1027784965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a40095f0 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07adfa1700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f079800e9a0 tx=0x7f079800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798009790 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a81966f0 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079800cbd0 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798018680 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.756+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81a29f0 con 0x7f07a80fe470 2023-07-21T22:42:08.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.757+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8196f80 con 0x7f07a80fe470 2023-07-21T22:42:08.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.759+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f079800cd40 con 0x7f07a80fe470 2023-07-21T22:42:08.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.759+0000 7f079effd700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 0x7f07940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:08.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.760+0000 7f07ad7a0700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 0x7f07940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:08.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.760+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0798014070 con 0x7f07a80fe470 2023-07-21T22:42:08.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.760+0000 7f07ad7a0700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 0x7f07940a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07a40098e0 tx=0x7f07a401a040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:08.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.762+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0798096920 con 0x7f07a80fe470 2023-07-21T22:42:08.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:08 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1555473552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:08.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:08 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1627396805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:08.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:08 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2673303607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:08.995+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f07a8066a00 con 0x7f07a80fe470 2023-07-21T22:42:09.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.006+0000 7f079effd700 1 -- 172.21.15.93:0/1027784965 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0798099f40 con 0x7f07a80fe470 2023-07-21T22:42:09.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.017+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 msgr2=0x7f07940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:09.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.017+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 0x7f07940a3770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07a40098e0 tx=0x7f07a401a040 comp rx=0 tx=0).stop 2023-07-21T22:42:09.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.017+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 msgr2=0x7f07a819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:09.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.017+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f079800e9a0 tx=0x7f079800ed60 comp rx=0 tx=0).stop 2023-07-21T22:42:09.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 shutdown_connections 2023-07-21T22:42:09.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07a8103430 0x7f07a8195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f07940a12c0 0x7f07940a3770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07a80fe470 0x7f07a819ba00 secure :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f079800e9a0 tx=0x7f079800ed60 comp rx=0 tx=0).stop 2023-07-21T22:42:09.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 --2- 172.21.15.93:0/1027784965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07a80fee10 0x7f07a819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.022+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 >> 172.21.15.93:0/1027784965 conn(0x7f07a8076200 msgr2=0x7f07a810b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:09.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.027+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 shutdown_connections 2023-07-21T22:42:09.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.031+0000 7f07b0205700 1 -- 172.21.15.93:0/1027784965 wait complete. 2023-07-21T22:42:09.074 INFO:teuthology.orchestra.run.smithi093.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-21T22:42:09.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:08 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1555473552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:08 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1627396805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:08 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2673303607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.098 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-21T22:42:09.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:08 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1555473552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:08 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1627396805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:08 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2673303607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:42:09.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.322+0000 7f0e59a52700 1 -- 172.21.15.93:0/997444661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e540fe9d0 msgr2=0x7f0e540fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:09.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.322+0000 7f0e59a52700 1 --2- 172.21.15.93:0/997444661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e540fe9d0 0x7f0e540fee40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009990 tx=0x7f0e44009ca0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.326+0000 7f0e59a52700 1 -- 172.21.15.93:0/997444661 shutdown_connections 2023-07-21T22:42:09.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.326+0000 7f0e59a52700 1 --2- 172.21.15.93:0/997444661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.326+0000 7f0e59a52700 1 --2- 172.21.15.93:0/997444661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e541049e0 0x7f0e54104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.326+0000 7f0e59a52700 1 --2- 172.21.15.93:0/997444661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e540fe9d0 0x7f0e540fee40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.326+0000 7f0e59a52700 1 -- 172.21.15.93:0/997444661 >> 172.21.15.93:0/997444661 conn(0x7f0e540fa4b0 msgr2=0x7f0e540fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:09.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.330+0000 7f0e59a52700 1 -- 172.21.15.93:0/997444661 shutdown_connections 2023-07-21T22:42:09.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.332+0000 7f0e59a52700 1 -- 172.21.15.93:0/997444661 wait complete. 2023-07-21T22:42:09.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.333+0000 7f0e59a52700 1 Processor -- start 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.333+0000 7f0e59a52700 1 -- start start 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e540fe9d0 0x7f0e5419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e54193b40 0x7f0e54193fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e54194760 con 0x7f0e54193b40 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541948d0 con 0x7f0e540ff380 2023-07-21T22:42:09.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.334+0000 7f0e59a52700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541986c0 con 0x7f0e540fe9d0 2023-07-21T22:42:09.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.335+0000 7f0e527fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.335+0000 7f0e527fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33532/0 (socket says 172.21.15.93:33532) 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.335+0000 7f0e527fc700 1 -- 172.21.15.93:0/2533611359 learned_addr learned my addr 172.21.15.93:0/2533611359 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.335+0000 7f0e52ffd700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e540fe9d0 0x7f0e5419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.336+0000 7f0e527fc700 1 -- 172.21.15.93:0/2533611359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e540fe9d0 msgr2=0x7f0e5419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.336+0000 7f0e527fc700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e540fe9d0 0x7f0e5419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.336+0000 7f0e527fc700 1 -- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e54193b40 msgr2=0x7f0e54193fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:09.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.336+0000 7f0e527fc700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e54193b40 0x7f0e54193fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:09.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.336+0000 7f0e527fc700 1 -- 172.21.15.93:0/2533611359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440095f0 con 0x7f0e540ff380 2023-07-21T22:42:09.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.339+0000 7f0e527fc700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e44003bc0 tx=0x7f0e44003bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:09.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.342+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4401d070 con 0x7f0e540ff380 2023-07-21T22:42:09.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.343+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e4400f460 con 0x7f0e540ff380 2023-07-21T22:42:09.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.343+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44017650 con 0x7f0e540ff380 2023-07-21T22:42:09.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.343+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e54198940 con 0x7f0e540ff380 2023-07-21T22:42:09.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.343+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e54198da0 con 0x7f0e540ff380 2023-07-21T22:42:09.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.344+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e340052f0 con 0x7f0e540ff380 2023-07-21T22:42:09.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.348+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0e440052d0 con 0x7f0e540ff380 2023-07-21T22:42:09.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.349+0000 7f0e58a50700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 0x7f0e400a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:09.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.349+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9001+0+0 (secure 0 0 0) 0x7f0e440ce030 con 0x7f0e540ff380 2023-07-21T22:42:09.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.350+0000 7f0e52ffd700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 0x7f0e400a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:09.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.351+0000 7f0e52ffd700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 0x7f0e400a3860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006fd0 tx=0x7f0e3c009190 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:09.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.357+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0e44097290 con 0x7f0e540ff380 2023-07-21T22:42:09.439 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:42:08.996+0000 7ff4d9b3b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:42:09.439 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.439 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.440 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: starting mds.a at 2023-07-21T22:42:09.440 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.440 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.440 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: starting mds.g at 2023-07-21T22:42:09.585 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.585 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.585 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: starting mds.c at 2023-07-21T22:42:09.586 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.586 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.586 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: starting mds.f at 2023-07-21T22:42:09.586 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.587 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.587 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: starting mds.i at 2023-07-21T22:42:09.587 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.587 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.588 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: starting mds.l at 2023-07-21T22:42:09.627 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.627 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.627 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: starting mds.b at 2023-07-21T22:42:09.628 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.628 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.628 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: starting mds.k at 2023-07-21T22:42:09.628 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.629 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.629 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mds[122535]: -- 172.21.15.134:0/1675524725 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee80f0d1e0 con 0x55ee80ee2800 2023-07-21T22:42:09.629 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: starting mds.e at 2023-07-21T22:42:09.629 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setuser ceph since I am not root 2023-07-21T22:42:09.629 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: ignoring --setgroup ceph since I am not root 2023-07-21T22:42:09.630 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mds[122877]: -- 172.21.15.134:0/1940634001 <== mon.2 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa5cfef1e0 con 0x55fa5cfc5000 2023-07-21T22:42:09.630 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: starting mds.h at 2023-07-21T22:42:09.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:09.775+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0e34005f40 con 0x7f0e540ff380 2023-07-21T22:42:10.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.009+0000 7f0e58a50700 1 -- 172.21.15.93:0/2533611359 <== mon.2 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e4409a8b0 con 0x7f0e540ff380 2023-07-21T22:42:10.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 msgr2=0x7f0e400a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:10.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 0x7f0e400a3860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c006fd0 tx=0x7f0e3c009190 comp rx=0 tx=0).stop 2023-07-21T22:42:10.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 msgr2=0x7f0e5419b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:10.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e44003bc0 tx=0x7f0e44003bf0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 shutdown_connections 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f0e400a13b0 0x7f0e400a3860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e54193b40 0x7f0e54193fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0e540fe9d0 0x7f0e5419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 --2- 172.21.15.93:0/2533611359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e540ff380 0x7f0e5419b0b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 >> 172.21.15.93:0/2533611359 conn(0x7f0e540fa4b0 msgr2=0x7f0e540fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:10.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 shutdown_connections 2023-07-21T22:42:10.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.018+0000 7f0e59a52700 1 -- 172.21.15.93:0/2533611359 wait complete. 2023-07-21T22:42:10.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-21T22:42:10.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:42:10.083 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:09 smithi093 ceph-mon[96579]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:42:10.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-07-21T22:42:10.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:42:10.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:09 smithi194 ceph-mon[98288]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:42:10.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: pgmap v819: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 66 KiB/s wr, 29 op/s 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T22:42:10.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/1027784965' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:42:10.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:09 smithi134 ceph-mon[97873]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:42:10.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.279+0000 7fee464a7700 1 -- 172.21.15.93:0/1742709375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4010e280 msgr2=0x7fee4010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:10.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.279+0000 7fee464a7700 1 --2- 172.21.15.93:0/1742709375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4010e280 0x7fee4010e650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee3801c0d0 tx=0x7fee3801c3e0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.284+0000 7fee464a7700 1 -- 172.21.15.93:0/1742709375 shutdown_connections 2023-07-21T22:42:10.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.284+0000 7fee464a7700 1 --2- 172.21.15.93:0/1742709375 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4006d3a0 0x7fee4011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.284+0000 7fee464a7700 1 --2- 172.21.15.93:0/1742709375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4010e280 0x7fee4010e650 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.284+0000 7fee464a7700 1 --2- 172.21.15.93:0/1742709375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee4006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.284+0000 7fee464a7700 1 -- 172.21.15.93:0/1742709375 >> 172.21.15.93:0/1742709375 conn(0x7fee4006bf70 msgr2=0x7fee40070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:10.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.288+0000 7fee464a7700 1 -- 172.21.15.93:0/1742709375 shutdown_connections 2023-07-21T22:42:10.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.291+0000 7fee464a7700 1 -- 172.21.15.93:0/1742709375 wait complete. 2023-07-21T22:42:10.334 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.291+0000 7fee464a7700 1 Processor -- start 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.291+0000 7fee464a7700 1 -- start start 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee40116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 0x7fee40117390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a25c0 con 0x7fee4010e280 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a2730 con 0x7fee4006c9a0 2023-07-21T22:42:10.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee464a7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401a28f0 con 0x7fee4006d3a0 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee44a44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee3f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 0x7fee40117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee44a44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50618/0 (socket says 172.21.15.93:50618) 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee3f7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 0x7fee40117390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.93:37696/0 (socket says 172.21.15.93:37696) 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee44a44700 1 -- 172.21.15.93:0/2648119563 learned_addr learned my addr 172.21.15.93:0/2648119563 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.292+0000 7fee3ffff700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee40116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:10.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 -- 172.21.15.93:0/2648119563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 msgr2=0x7fee40117390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 0x7fee40117390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 -- 172.21.15.93:0/2648119563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 msgr2=0x7fee40116e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee40116e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 -- 172.21.15.93:0/2648119563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3801ad10 con 0x7fee4010e280 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee3ffff700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee40116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee44a44700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fee3400b550 tx=0x7fee3400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34011830 con 0x7fee4010e280 2023-07-21T22:42:10.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee34011e50 con 0x7fee4010e280 2023-07-21T22:42:10.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee401a2bd0 con 0x7fee4010e280 2023-07-21T22:42:10.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3400f2e0 con 0x7fee4010e280 2023-07-21T22:42:10.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.293+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee40078950 con 0x7fee4010e280 2023-07-21T22:42:10.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.296+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee40067290 con 0x7fee4010e280 2023-07-21T22:42:10.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.296+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fee340105a0 con 0x7fee4010e280 2023-07-21T22:42:10.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.297+0000 7fee3d7fa700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 0x7fee280e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:10.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.297+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 9001+0+0 (secure 0 0 0) 0x7fee340cc760 con 0x7fee4010e280 2023-07-21T22:42:10.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.298+0000 7fee3ffff700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 0x7fee280e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:10.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.298+0000 7fee3ffff700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 0x7fee280e7840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee3801c860 tx=0x7fee3801c040 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:10.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.305+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fee34095aa0 con 0x7fee4010e280 2023-07-21T22:42:10.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:10.656+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee40078b70 con 0x7fee4010e280 2023-07-21T22:42:11.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.011+0000 7fee3d7fa700 1 -- 172.21.15.93:0/2648119563 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) v1 ==== 181+0+0 (secure 0 0 0) 0x7fee340990c0 con 0x7fee4010e280 2023-07-21T22:42:11.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 msgr2=0x7fee280e7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 0x7fee280e7840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee3801c860 tx=0x7fee3801c040 comp rx=0 tx=0).stop 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 msgr2=0x7fee401a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fee3400b550 tx=0x7fee3400b860 comp rx=0 tx=0).stop 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 shutdown_connections 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7fee280e5390 0x7fee280e7840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee4010e280 0x7fee401a1f80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee4006c9a0 0x7fee40116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 --2- 172.21.15.93:0/2648119563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee4006d3a0 0x7fee40117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 >> 172.21.15.93:0/2648119563 conn(0x7fee4006bf70 msgr2=0x7fee40070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 shutdown_connections 2023-07-21T22:42:11.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.018+0000 7fee464a7700 1 -- 172.21.15.93:0/2648119563 wait complete. 2023-07-21T22:42:11.066 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' removed 2023-07-21T22:42:11.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2533611359' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:42:11.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:42:11.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: Cluster is now healthy 2023-07-21T22:42:11.084 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6839/2562278008,v1:172.21.15.93:6840/2562278008] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.93:6838/2742187931,v1:172.21.15.93:6841/2742187931] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6832/2965982243,v1:172.21.15.134:6833/2965982243] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6836/117273012,v1:172.21.15.134:6837/117273012] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6834/312968937,v1:172.21.15.134:6835/312968937] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.134:6838/670675892,v1:172.21.15.134:6839/670675892] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6832/2696847588,v1:172.21.15.194:6833/2696847588] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6838/1226330472,v1:172.21.15.194:6839/1226330472] up:boot 2023-07-21T22:42:11.085 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6836/2300344455,v1:172.21.15.194:6837/2300344455] up:boot 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: mds.? [v2:172.21.15.194:6834/3434469683,v1:172.21.15.194:6835/3434469683] up:boot 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: fsmap 12 up:standby 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:42:11.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:42:11.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:10 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.088 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-21T22:42:11.089 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2533611359' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.089 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.089 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:42:11.089 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: Cluster is now healthy 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6839/2562278008,v1:172.21.15.93:6840/2562278008] up:boot 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.93:6838/2742187931,v1:172.21.15.93:6841/2742187931] up:boot 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6832/2965982243,v1:172.21.15.134:6833/2965982243] up:boot 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6836/117273012,v1:172.21.15.134:6837/117273012] up:boot 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6834/312968937,v1:172.21.15.134:6835/312968937] up:boot 2023-07-21T22:42:11.090 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.134:6838/670675892,v1:172.21.15.134:6839/670675892] up:boot 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6832/2696847588,v1:172.21.15.194:6833/2696847588] up:boot 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6838/1226330472,v1:172.21.15.194:6839/1226330472] up:boot 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6836/2300344455,v1:172.21.15.194:6837/2300344455] up:boot 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: mds.? [v2:172.21.15.194:6834/3434469683,v1:172.21.15.194:6835/3434469683] up:boot 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: fsmap 12 up:standby 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:42:11.091 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:42:11.092 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:42:11.093 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:10 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2533611359' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: Cluster is now healthy 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6839/2562278008,v1:172.21.15.93:6840/2562278008] up:boot 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.93:6838/2742187931,v1:172.21.15.93:6841/2742187931] up:boot 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6832/2965982243,v1:172.21.15.134:6833/2965982243] up:boot 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6836/117273012,v1:172.21.15.134:6837/117273012] up:boot 2023-07-21T22:42:11.127 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6834/312968937,v1:172.21.15.134:6835/312968937] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.134:6838/670675892,v1:172.21.15.134:6839/670675892] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6832/2696847588,v1:172.21.15.194:6833/2696847588] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6838/1226330472,v1:172.21.15.194:6839/1226330472] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6836/2300344455,v1:172.21.15.194:6837/2300344455] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: mds.? [v2:172.21.15.194:6834/3434469683,v1:172.21.15.194:6835/3434469683] up:boot 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:42:11.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: fsmap 12 up:standby 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:42:11.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:42:11.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:42:11.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='mgr.25079 172.21.15.93:0/1199809021' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:42:11.130 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:10 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:11.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 -- 172.21.15.93:0/3462184682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 msgr2=0x7f09d410c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 --2- 172.21.15.93:0/3462184682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d410c500 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b0a0 tx=0x7f09cc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 -- 172.21.15.93:0/3462184682 shutdown_connections 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 --2- 172.21.15.93:0/3462184682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d410c500 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 --2- 172.21.15.93:0/3462184682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d4100ba0 0x7f09d4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 --2- 172.21.15.93:0/3462184682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d41014b0 0x7f09d4109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 -- 172.21.15.93:0/3462184682 >> 172.21.15.93:0/3462184682 conn(0x7f09d40fa530 msgr2=0x7f09d40fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:11.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 -- 172.21.15.93:0/3462184682 shutdown_connections 2023-07-21T22:42:11.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.276+0000 7f09db526700 1 -- 172.21.15.93:0/3462184682 wait complete. 2023-07-21T22:42:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 Processor -- start 2023-07-21T22:42:11.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 -- start start 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 0x7f09d40fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 0x7f09d40fef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40ff740 con 0x7f09d410a0c0 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d41a43c0 con 0x7f09d41014b0 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09db526700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d41a4500 con 0x7f09d4100ba0 2023-07-21T22:42:11.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d9ac3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d8ac1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 0x7f09d40fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d9ac3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50632/0 (socket says 172.21.15.93:50632) 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d8ac1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 0x7f09d40fef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.93:33560/0 (socket says 172.21.15.93:33560) 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d9ac3700 1 -- 172.21.15.93:0/3929165176 learned_addr learned my addr 172.21.15.93:0/3929165176 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d92c2700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 0x7f09d40fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d9ac3700 1 -- 172.21.15.93:0/3929165176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 msgr2=0x7f09d40fe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.277+0000 7f09d9ac3700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 0x7f09d40fe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09d9ac3700 1 -- 172.21.15.93:0/3929165176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 msgr2=0x7f09d40fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09d9ac3700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 0x7f09d40fef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09d9ac3700 1 -- 172.21.15.93:0/3929165176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc009d00 con 0x7f09d410a0c0 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09d92c2700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 0x7f09d40fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09d9ac3700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b780 tx=0x7f09cc007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00e040 con 0x7f09d410a0c0 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d41a4700 con 0x7f09d410a0c0 2023-07-21T22:42:11.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09cc00be40 con 0x7f09d410a0c0 2023-07-21T22:42:11.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0052a0 con 0x7f09d410a0c0 2023-07-21T22:42:11.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.278+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d41a4b90 con 0x7f09d410a0c0 2023-07-21T22:42:11.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.279+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d41049e0 con 0x7f09d410a0c0 2023-07-21T22:42:11.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.281+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f09cc01d470 con 0x7f09d410a0c0 2023-07-21T22:42:11.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.281+0000 7f09ca7fc700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 0x7f09c00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:42:11.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.282+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 8461+0+0 (secure 0 0 0) 0x7f09cc0ce340 con 0x7f09d410a0c0 2023-07-21T22:42:11.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.282+0000 7f09d92c2700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 0x7f09c00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:42:11.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.282+0000 7f09d92c2700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 0x7f09c00a3770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009cb0 tx=0x7f09d0009260 comp rx=0 tx=0).ready entity=mgr.25079 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:42:11.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.287+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f09cc097820 con 0x7f09d410a0c0 2023-07-21T22:42:11.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:11.493+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09d4066a00 con 0x7f09d410a0c0 2023-07-21T22:42:12.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.016+0000 7f09ca7fc700 1 -- 172.21.15.93:0/3929165176 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) v1 ==== 169+0+0 (secure 0 0 0) 0x7f09cc02e350 con 0x7f09d410a0c0 2023-07-21T22:42:12.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 msgr2=0x7f09c00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:12.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 0x7f09c00a3770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009cb0 tx=0x7f09d0009260 comp rx=0 tx=0).stop 2023-07-21T22:42:12.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 msgr2=0x7f09d41a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:42:12.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b780 tx=0x7f09cc007990 comp rx=0 tx=0).stop 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 shutdown_connections 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:6800/904524118,v1:172.21.15.93:6801/904524118] conn(0x7f09c00a12c0 0x7f09c00a3770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09d410a0c0 0x7f09d41a3e80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f09d4100ba0 0x7f09d40fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 --2- 172.21.15.93:0/3929165176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09d41014b0 0x7f09d40fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:42:12.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 >> 172.21.15.93:0/3929165176 conn(0x7f09d40fa530 msgr2=0x7f09d4101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:42:12.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 shutdown_connections 2023-07-21T22:42:12.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-07-21T22:42:12.021+0000 7f09db526700 1 -- 172.21.15.93:0/3929165176 wait complete. 2023-07-21T22:42:12.063 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' removed 2023-07-21T22:42:12.079 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:42:12.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:11 smithi093 ceph-mon[96579]: pgmap v821: 97 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 69 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 KiB/s wr, 32 op/s 2023-07-21T22:42:12.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:11 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:42:12.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:11 smithi093 ceph-mon[96579]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T22:42:12.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:11 smithi093 ceph-mon[96579]: from='client.? 172.21.15.93:0/3929165176' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:12.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:11 smithi194 ceph-mon[98288]: pgmap v821: 97 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 69 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 KiB/s wr, 32 op/s 2023-07-21T22:42:12.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:11 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:42:12.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:11 smithi194 ceph-mon[98288]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T22:42:12.083 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:11 smithi194 ceph-mon[98288]: from='client.? 172.21.15.93:0/3929165176' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:12.105 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:42:12.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:11 smithi134 ceph-mon[97873]: pgmap v821: 97 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 69 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 KiB/s wr, 32 op/s 2023-07-21T22:42:12.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:11 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/2648119563' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:42:12.125 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:11 smithi134 ceph-mon[97873]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T22:42:12.126 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:11 smithi134 ceph-mon[97873]: from='client.? 172.21.15.93:0/3929165176' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:42:12.164 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:42:12.217 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-21T22:42:12.217 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:42:12.234 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:42:12.250 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:42:12.267 INFO:tasks.cephadm:Stopping all daemons... 2023-07-21T22:42:12.267 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-21T22:42:12.268 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:42:12.681 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:12 smithi093 systemd[1]: Stopping Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:12.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:42:12.641+0000 7ff4df346700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:12.937 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a[96555]: 2023-07-21T22:42:12.641+0000 7ff4df346700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-21T22:42:13.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-21T22:42:13.262 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:12 smithi093 bash[163184]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:42:13.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:13 smithi093 systemd[1]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T22:42:13.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Jul 21 22:42:13 smithi093 systemd[1]: Stopped Ceph mon.a for de1cec7a-2812-11ee-9b34-001a4aab830c. 2023-07-21T22:42:13.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:13.287 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-21T22:42:13.288 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-21T22:42:13.288 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.b 2023-07-21T22:42:13.625 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:13 smithi134 systemd[1]: Stopping Ceph mon.b for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:14.375 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-b[97849]: 2023-07-21T22:42:13.849+0000 7f3d2ea03700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:14.376 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-b[97849]: 2023-07-21T22:42:13.849+0000 7f3d2ea03700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:42:14.830 INFO:journalctl@ceph.mon.b.smithi134.stdout:Jul 21 22:42:14 smithi134 bash[135271]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-b 2023-07-21T22:42:14.835 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-21T22:42:14.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:14.879 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-21T22:42:14.880 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-21T22:42:14.880 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.c 2023-07-21T22:42:15.320 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:14 smithi194 systemd[1]: Stopping Ceph mon.c for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:15.583 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-c[98244]: 2023-07-21T22:42:15.280+0000 7fb8fd943700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:15.584 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-c[98244]: 2023-07-21T22:42:15.280+0000 7fb8fd943700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:42:16.046 INFO:journalctl@ceph.mon.c.smithi194.stdout:Jul 21 22:42:15 smithi194 bash[133856]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mon-c 2023-07-21T22:42:16.253 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-21T22:42:16.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:16.313 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-21T22:42:16.314 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-21T22:42:16.314 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:42:16.687 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:42:16 smithi093 systemd[1]: Stopping Ceph mgr.x for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:16.971 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:42:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x[96828]: 2023-07-21T22:42:16.674+0000 7fb6fd97a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:17.276 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Jul 21 22:42:16 smithi093 bash[163321]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-x 2023-07-21T22:42:17.283 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-21T22:42:17.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:17.320 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-21T22:42:17.320 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-21T22:42:17.320 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.y 2023-07-21T22:42:17.625 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:42:17 smithi134 systemd[1]: Stopping Ceph mgr.y for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:18.043 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:42:17 smithi134 bash[135411]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-y 2023-07-21T22:42:18.223 INFO:journalctl@ceph.mgr.y.smithi134.stdout:Jul 21 22:42:18 smithi134 systemd[1]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-21T22:42:18.227 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-21T22:42:18.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:18.265 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-21T22:42:18.265 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-21T22:42:18.265 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.z 2023-07-21T22:42:18.583 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:42:18 smithi194 systemd[1]: Stopping Ceph mgr.z for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:19.240 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:42:18 smithi194 bash[133995]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mgr-z 2023-07-21T22:42:19.241 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Jul 21 22:42:18 smithi194 systemd[1]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-21T22:42:19.248 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-21T22:42:19.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:19.283 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-21T22:42:19.283 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-21T22:42:19.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.0 2023-07-21T22:42:19.687 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:42:19 smithi093 systemd[1]: Stopping Ceph osd.0 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:20.437 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:42:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:42:19.898+0000 7f559599f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:20.438 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:42:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:42:19.898+0000 7f559599f700 -1 osd.0 133 *** Got signal Terminated *** 2023-07-21T22:42:20.438 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:42:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0[109510]: 2023-07-21T22:42:19.898+0000 7f559599f700 -1 osd.0 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:26.187 INFO:journalctl@ceph.osd.0.smithi093.stdout:Jul 21 22:42:25 smithi093 bash[163466]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-0 2023-07-21T22:42:27.402 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-21T22:42:27.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:27.443 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-21T22:42:27.443 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-21T22:42:27.444 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.1 2023-07-21T22:42:27.799 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:42:27 smithi134 systemd[1]: Stopping Ceph osd.1 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:28.125 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:42:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:42:27.758+0000 7f50f4410700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:28.125 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:42:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:42:27.758+0000 7f50f4410700 -1 osd.1 133 *** Got signal Terminated *** 2023-07-21T22:42:28.125 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:42:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1[102847]: 2023-07-21T22:42:27.758+0000 7f50f4410700 -1 osd.1 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:33.875 INFO:journalctl@ceph.osd.1.smithi134.stdout:Jul 21 22:42:33 smithi134 bash[135558]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-1 2023-07-21T22:42:35.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-21T22:42:35.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:35.297 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-21T22:42:35.297 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-21T22:42:35.298 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.2 2023-07-21T22:42:35.584 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:42:35 smithi194 systemd[1]: Stopping Ceph osd.2 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:36.083 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:42:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:42:35.625+0000 7f9f75754700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:36.083 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:42:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:42:35.625+0000 7f9f75754700 -1 osd.2 133 *** Got signal Terminated *** 2023-07-21T22:42:36.083 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:42:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2[104080]: 2023-07-21T22:42:35.625+0000 7f9f75754700 -1 osd.2 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:41.535 INFO:journalctl@ceph.osd.2.smithi194.stdout:Jul 21 22:42:41 smithi194 bash[134142]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-2 2023-07-21T22:42:42.996 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-21T22:42:43.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:43.036 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-21T22:42:43.036 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-21T22:42:43.036 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.3 2023-07-21T22:42:43.395 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:43 smithi093 systemd[1]: Stopping Ceph osd.3 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:43.687 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:43 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:42:43.354+0000 7f2b85bbd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:43.687 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:43 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:42:43.354+0000 7f2b85bbd700 -1 osd.3 133 *** Got signal Terminated *** 2023-07-21T22:42:43.687 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:43 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:42:43.354+0000 7f2b85bbd700 -1 osd.3 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:47.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:46.779+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:47.187 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:46 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:42:46.741+0000 7f2b809b0700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:24.062126+0000 front 2023-07-21T22:42:24.062156+0000 (oldest deadline 2023-07-21T22:42:46.362306+0000) 2023-07-21T22:42:48.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:47.740+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:48.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:47.649+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:48.187 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:47 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3[114965]: 2023-07-21T22:42:47.767+0000 7f2b809b0700 -1 osd.3 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:24.062126+0000 front 2023-07-21T22:42:24.062156+0000 (oldest deadline 2023-07-21T22:42:46.362306+0000) 2023-07-21T22:42:49.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:48.757+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:49.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:48.656+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:49.141 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:48 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:48.828+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:49.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:48 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:48.983+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:49.437 INFO:journalctl@ceph.osd.3.smithi093.stdout:Jul 21 22:42:49 smithi093 bash[163737]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-3 2023-07-21T22:42:50.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:49 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:49.735+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:50.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:49 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:49.663+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:50.142 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:49 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:49.800+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:50.142 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:49 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:49.651+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:50.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:49 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:49.972+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:50.538 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-21T22:42:50.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:50.579 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-21T22:42:50.580 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-21T22:42:50.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.4 2023-07-21T22:42:50.847 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:50 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:50.655+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:50.875 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:42:50 smithi134 systemd[1]: Stopping Ceph osd.4 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:51.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:50.759+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:51.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:50 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:50.665+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:51.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:50 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:50.807+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:51.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:50 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:50.962+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:51.375 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:42:50 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:42:50.897+0000 7fb3c72d8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:51.376 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:42:50 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:42:50.897+0000 7fb3c72d8700 -1 osd.4 133 *** Got signal Terminated *** 2023-07-21T22:42:51.376 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:42:50 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4[108375]: 2023-07-21T22:42:50.897+0000 7fb3c72d8700 -1 osd.4 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:52.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:51 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:51.748+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:52.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:51 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:51.639+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:52.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:51.839+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:52.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:51.692+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:52.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:51 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:51.960+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:53.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:52 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:52.784+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:53.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:52 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:52.594+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:53.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:52 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:52.832+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:53.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:52 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:52.674+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:53.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:52 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:52.968+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:54.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:53.803+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:54.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:53 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:53.643+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:54.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:53 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:53.827+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:54.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:53 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:53.723+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:54.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:53 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:53.723+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:54.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:53 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:53.919+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:55.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:54.769+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:55.084 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:54.780+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:42:55.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:54 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:54.682+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:55.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:54 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:54.790+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:55.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:54 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:54.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:55.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:54 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:54.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:55.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:54 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:54.886+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:55.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:54 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:54.886+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:42:56.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:55.775+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:56.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:55.775+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:42:56.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:55.755+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:42:56.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:55 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:55.650+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:56.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:55 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:55.782+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:56.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:55 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:55.782+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:42:56.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:55 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:55.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:56.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:55 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:55.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:56.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:55 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:55.916+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:56.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:55 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:55.916+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:42:56.817 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:42:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:42:56.513+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:42:56.817 INFO:journalctl@ceph.osd.4.smithi134.stdout:Jul 21 22:42:56 smithi134 bash[135828]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-4 2023-07-21T22:42:57.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:56.777+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:57.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:56.777+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:42:57.084 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:56.765+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:42:57.085 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:56.601+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:57.085 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:56 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:56.601+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:42:57.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:56.783+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:57.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:56.783+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:42:57.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:56.695+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:57.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:56.695+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:57.372 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:56.873+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:57.373 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:56 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:56.873+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:42:57.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:42:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:42:57.550+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:42:58.070 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:57.843+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:58.071 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:57 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:57.843+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:42:58.076 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-21T22:42:58.082 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:57 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:57.780+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:42:58.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:57 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:57.812+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:58.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:57 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:57.812+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:42:58.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:57 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:57.600+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:58.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:57 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:57.600+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:42:58.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:42:58.116 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-21T22:42:58.117 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-21T22:42:58.117 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.5 2023-07-21T22:42:58.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:57 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:57.767+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:58.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:57 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:57.767+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:42:58.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:57 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:57.734+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:58.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:57 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:57.734+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:58.477 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 systemd[1]: Stopping Ceph osd.5 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:42:58.833 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:58.438+0000 7f7ab3194700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:42:58.833 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:58.438+0000 7f7ab3194700 -1 osd.5 133 *** Got signal Terminated *** 2023-07-21T22:42:58.833 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:58.438+0000 7f7ab3194700 -1 osd.5 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:42:58.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:58.649+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:42:58.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:58.649+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:42:58.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:42:58 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:42:58.529+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:42:59.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:58 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:58.797+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:42:59.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:58 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:58.797+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:42:59.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:58 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:58.725+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:42:59.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:58 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:58.725+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:42:59.333 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:58.802+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:42:59.333 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:58.829+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:42:59.333 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:58 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:58.829+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:42:59.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:58 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:58.888+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:42:59.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:58 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:58.888+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:42:59.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:42:59 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:42:59.488+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:00.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:42:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:42:59.796+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:00.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:59.832+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:43:00.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:42:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:42:59.832+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:43:00.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:59.627+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:00.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:42:59 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:42:59.627+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:00.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:59 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:59.826+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:00.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:42:59 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:42:59.826+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:00.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:59 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:59.769+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:00.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:42:59 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:42:59.769+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:00.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:59 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:59.922+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:00.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:42:59 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:42:59.922+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:00.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:00 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:00.440+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:01.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:00 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:00.757+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:01.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:00 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:00.796+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:43:01.083 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:00 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:00.796+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:43:01.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:00 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:00.581+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:01.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:00 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:00.581+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:01.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:00.778+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:01.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:00.778+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:01.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:00.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:01.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:00 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:00.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:01.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:00 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:00.902+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:01.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:00 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:00.902+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:01.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:01.710+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:01.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:01.534+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:01.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:01.534+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:01.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:01.534+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:01.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:01 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:01.484+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:02.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:01.806+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:02.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:01.806+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:02.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:01.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:02.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:01.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:02.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:01.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:02.333 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:01.814+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:43:02.333 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:01 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:01.814+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:43:02.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:01 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:01.901+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:02.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:01 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:01.901+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:02.874 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:02 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:02.501+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:02.901 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:02.674+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:02.901 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:02.561+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:02.901 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:02.561+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:02.901 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:02.561+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:03.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:02.854+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:03.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:02.854+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:03.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:02.854+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:03.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:02.810+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:03.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:02.810+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:03.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:02 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:02.810+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:03.332 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:02.861+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.050454+0000 front 2023-07-21T22:42:22.050468+0000 (oldest deadline 2023-07-21T22:42:46.150316+0000) 2023-07-21T22:43:03.333 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:02 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5[109556]: 2023-07-21T22:43:02.861+0000 7f7aabf83700 -1 osd.5 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.251509+0000 front 2023-07-21T22:42:30.251480+0000 (oldest deadline 2023-07-21T22:42:55.551456+0000) 2023-07-21T22:43:03.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:02 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:02.905+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:03.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:02 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:02.905+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:03.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:03 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:03.464+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:03.984 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:03 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:03.653+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:03.984 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:03 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:03.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:03.985 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:03 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:03.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:03.985 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:03 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:03.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:04.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:03.859+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:04.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:03.859+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:04.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:03.859+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:04.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:03.793+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:04.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:03.793+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:04.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:03 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:03.793+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:04.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:03 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:03.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:04.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:03 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:03.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:04.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:03 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:03.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:04.582 INFO:journalctl@ceph.osd.5.smithi194.stdout:Jul 21 22:43:04 smithi194 bash[134413]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-5 2023-07-21T22:43:04.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:04 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:04.439+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:04.907 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:04.573+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:04.907 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:04.573+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:04.907 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:04.573+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:04.907 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:04 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:04.638+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:05.186 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:04.892+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:05.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:04.893+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:05.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:04.893+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:05.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:04.788+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:05.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:04.788+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:05.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:04 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:04.788+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:05.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:04 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:04.897+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:05.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:04 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:04.897+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:05.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:04 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:04.897+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:05.624 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-21T22:43:05.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:05.663 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-21T22:43:05.664 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-21T22:43:05.664 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.6 2023-07-21T22:43:05.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:05 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:05.632+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:05.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:05 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:05.632+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:05.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:05 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:05.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:05.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:05 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:05.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:05.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:05 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:05.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:05.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:05 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:05.403+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:05.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:05 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:05.403+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:06.023 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 systemd[1]: Stopping Ceph osd.6 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:06.023 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:06.023 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:06.024 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:06.024 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:05.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:06.024 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:05.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:06.024 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:05.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:06.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:05 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:05.895+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:06.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:05 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:05.895+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:06.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:05 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:05.895+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:06.436 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.983+0000 7f7de48cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:06.437 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.983+0000 7f7de48cb700 -1 osd.6 133 *** Got signal Terminated *** 2023-07-21T22:43:06.437 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:05 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:05.983+0000 7f7de48cb700 -1 osd.6 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:06.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:06.655+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:06.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:06.655+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:06.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:06.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:06.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:06.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:06.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:06 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:06.543+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:06.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:06.358+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:06.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:06.358+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:07.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:06.730+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:07.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:06.730+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:07.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:06.730+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:07.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:06.831+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:07.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:06.831+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:07.188 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:06 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:06.831+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:07.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:06.879+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:07.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:06.879+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:07.369 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:06.879+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:07.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:07.329+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:07.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:07.329+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:07.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:07 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:07.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:07.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:07 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:07.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:07.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:07 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:07.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:07.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:07 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:07.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:07.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:07 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:07.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:08.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:07.707+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:08.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:07.707+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:08.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:07.707+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:08.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:07.856+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:08.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:07.856+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:08.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:07 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:07.856+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:08.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:07.924+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:08.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:07.924+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:08.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:07 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:07.924+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:08.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:08 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:08.336+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:08.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:08 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:08.336+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:09.082 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:08.632+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:09.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:08.632+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:09.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:08.549+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:09.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:08.549+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:09.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:08 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:08.549+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:09.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:08.703+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:09.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:08.703+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:09.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:08.703+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:09.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:08.883+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:09.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:08.883+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:09.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:08 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:08.883+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:09.333 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:08 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:08.931+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:09.333 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:08 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:08.931+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:09.333 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:08 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:08.931+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:09.624 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:09.293+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:09.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:09.293+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:10.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:09.648+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:10.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:09.648+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:10.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:09.648+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:10.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:09.582+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:10.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:09.582+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:10.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:09 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:09.582+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:10.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:09.697+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:10.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:09.697+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:10.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:09.697+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:10.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:09.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:10.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:09.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:10.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:09 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:09.881+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:10.351 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:09.920+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:10.351 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:09.920+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:10.351 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:09.920+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:10.351 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:09 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:09.920+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:10.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:10.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:10.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:10.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:10.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:10.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:11.082 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:10.624+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:11.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:10.624+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:11.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:10.624+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:11.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:10.555+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:11.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:10.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:11.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:10.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:11.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:10 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:10.556+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:11.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:10.673+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:11.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:10.673+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:11.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:10.673+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:11.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:10.908+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.401319+0000 front 2023-07-21T22:42:23.401356+0000 (oldest deadline 2023-07-21T22:42:48.101524+0000) 2023-07-21T22:43:11.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:10.908+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.103702+0000 front 2023-07-21T22:42:32.103675+0000 (oldest deadline 2023-07-21T22:42:55.603647+0000) 2023-07-21T22:43:11.187 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:10 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6[120414]: 2023-07-21T22:43:10.908+0000 7f7ddfebf700 -1 osd.6 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.305288+0000 front 2023-07-21T22:42:37.305213+0000 (oldest deadline 2023-07-21T22:43:02.005182+0000) 2023-07-21T22:43:11.311 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:10.891+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:11.311 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:10.891+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:11.312 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:10.891+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:11.312 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:10.891+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:11.624 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:11.271+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:11.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:11.271+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:11.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:11.271+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:11.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:11.640+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:11.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:11.640+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:11.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:11.640+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:11.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:11.516+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:11.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:11.516+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:11.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:11.516+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:11.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:11 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:11.516+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:11.915 INFO:journalctl@ceph.osd.6.smithi093.stdout:Jul 21 22:43:11 smithi093 bash[164008]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-6 2023-07-21T22:43:11.917 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:11.675+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:11.917 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:11.675+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:11.917 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:11.675+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:11.918 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:11 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:11.675+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:12.357 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:11.896+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:12.358 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:11.896+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:12.358 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:11.896+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:12.358 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:11 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:11.896+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:12.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:12.317+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:12.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:12.317+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:12.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:12.317+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:12.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:12.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:12.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:12.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:12.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:12.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:12.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:12.499+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:12.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:12.499+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:12.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:12.499+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:12.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:12 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:12.499+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:13.169 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:12.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:13.169 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:12.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:13.169 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:12.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:13.169 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:12 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:12.684+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:13.175 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-21T22:43:13.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:13.216 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-21T22:43:13.216 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-21T22:43:13.216 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.7 2023-07-21T22:43:13.248 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:12.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:13.248 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:12.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:13.248 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:12.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:13.248 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:12 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:12.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:13.574 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 systemd[1]: Stopping Ceph osd.7 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:13.574 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.344+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:13.575 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.344+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:13.575 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.344+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:13.575 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.534+0000 7fddb97f2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:13.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:13.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:13.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:13.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:13.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:13.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:13.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:13.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:13.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:13.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:13.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:13.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:13.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:13 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:13.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:13.874 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.534+0000 7fddb97f2700 -1 osd.7 133 *** Got signal Terminated *** 2023-07-21T22:43:13.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:13.534+0000 7fddb97f2700 -1 osd.7 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:14.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:13.701+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:14.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:13.701+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:14.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:13.701+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:14.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:13 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:13.701+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:14.371 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:13.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:14.371 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:13.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:14.371 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:13.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:14.372 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:13 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:13.857+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:14.624 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:14.331+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:14.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:14.331+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:14.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:14.331+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:14.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:14.641+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:14.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:14.641+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:14.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:14.641+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:14.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:14.506+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:14.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:14.506+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:14.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:14.506+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:14.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:14 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:14.506+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:15.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:14.823+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:15.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:14.823+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:15.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:14.823+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:15.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:14.823+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:15.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:14 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:14.748+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:15.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:14 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:14.748+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:15.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:14 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:14.748+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:15.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:14 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:14.748+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:15.624 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:15.333+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:15.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:15.333+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:15.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:15.333+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:15.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:15.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:15.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:15.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:15.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:15.593+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:15.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:15.513+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:15.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:15.513+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:15.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:15.513+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:15.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:15 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:15.513+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:16.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:15 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:15.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:16.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:15 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:15.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:16.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:15 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:15.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:16.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:15 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:15.772+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:16.374 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:15.851+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:16.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:15.851+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:16.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:15.851+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:16.375 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:15 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:15.851+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:16.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:16.590+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:16.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:16.590+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:16.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:16.590+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:16.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:16.498+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:16.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:16.498+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:16.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:16.498+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:16.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:16 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:16.498+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:16.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:16.340+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:16.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:16.340+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:16.875 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:16.340+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:17.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:16.787+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:17.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:16.787+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:17.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:16.787+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:17.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:16 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:16.787+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:17.350 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:16.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:17.350 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:16.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:17.350 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:16.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:17.350 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:16 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:16.872+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:17.624 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:17.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:17.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:17.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:17.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:17.310+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:17.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:17.558+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:17.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:17.558+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:17.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:17.558+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:17.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:17.530+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:17.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:17.530+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:17.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:17.530+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:17.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:17.530+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:17.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:17 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:17.530+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:18.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:17.830+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:18.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:17.830+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:18.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:17.830+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:18.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:17 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:17.830+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:18.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:17 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:17.792+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:18.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:17 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:17.792+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:18.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:17 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:17.792+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:18.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:17 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:17.792+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:18.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:18.325+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:32.236314+0000 front 2023-07-21T22:42:32.236294+0000 (oldest deadline 2023-07-21T22:42:55.736531+0000) 2023-07-21T22:43:18.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:18.325+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.738527+0000 front 2023-07-21T22:42:39.738294+0000 (oldest deadline 2023-07-21T22:43:05.038215+0000) 2023-07-21T22:43:18.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7[113835]: 2023-07-21T22:43:18.325+0000 7fddb45e5700 -1 osd.7 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:45.039067+0000 front 2023-07-21T22:42:45.038858+0000 (oldest deadline 2023-07-21T22:43:09.738904+0000) 2023-07-21T22:43:18.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:18.530+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:18.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:18.530+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:18.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:18.530+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:18.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:18.530+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:18.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:18.521+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:18.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:18.522+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:18.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:18.522+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:18.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:18.522+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:18.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:18 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:18.522+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:19.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:18.791+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:19.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:18.791+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:19.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:18.791+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:19.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:18.791+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:19.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:18.757+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:19.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:18.757+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:19.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:18.757+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:19.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:18 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:18.757+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:19.625 INFO:journalctl@ceph.osd.7.smithi134.stdout:Jul 21 22:43:19 smithi134 bash[136105]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-7 2023-07-21T22:43:19.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:19.545+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:19.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:19.545+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:19.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:19.545+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:19.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:19.545+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:19.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:19.540+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:19.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:19.540+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:19.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:19.540+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:19.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:19.540+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:19.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:19 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:19.540+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:20.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:19.748+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:20.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:19.748+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:20.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:19.748+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:20.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:19 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:19.748+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:20.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:19.786+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:20.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:19.786+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:20.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:19.786+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:20.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:19 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:19.786+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:20.718 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-21T22:43:20.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:20.798 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-21T22:43:20.799 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-21T22:43:20.799 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.8 2023-07-21T22:43:20.830 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:20.567+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:20.830 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:20.567+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:20.830 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:20.567+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:20.830 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:20.567+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:20.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:20.529+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:20.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:20.529+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:20.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:20.529+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:20.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:20.529+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:20.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:20 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:20.529+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:21.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:20 smithi194 systemd[1]: Stopping Ceph osd.8 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:21.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:20.706+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:21.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:20.706+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:21.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:20.706+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:21.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:20.706+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:21.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:20 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:20.706+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:21.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:20.778+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:21.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:20.778+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:21.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:20.778+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:21.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:20 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:20.778+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:21.551 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.109+0000 7f4b9a66e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:21.552 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.109+0000 7f4b9a66e700 -1 osd.8 133 *** Got signal Terminated *** 2023-07-21T22:43:21.552 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.109+0000 7f4b9a66e700 -1 osd.8 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:21.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:21.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:21.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:21.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:21.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:21.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:21.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:21.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:21.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:21.511+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:21.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.601+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:21.834 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.601+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:21.834 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.601+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:21.834 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:21 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:21.601+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:22.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:21.722+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:22.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:21.722+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:22.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:21.722+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:22.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:21.722+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:22.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:21 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:21.722+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:22.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:21.747+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:22.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:21.747+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:22.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:21.747+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:22.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:21 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:21.747+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:22.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:22.637+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:22.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:22.637+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:22.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:22.637+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:22.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:22.637+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:22.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:22.532+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:22.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:22.532+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:22.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:22.532+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:22.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:22.532+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:22.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:22.532+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:23.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:22.697+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:23.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:22.697+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:23.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:22.697+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:23.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:22.697+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:23.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:22 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:22.697+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:23.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:22.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:23.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:22.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:23.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:22.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:23.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:22 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:22.699+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:24.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:23.657+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:24.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:23.657+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:24.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:23.657+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:24.083 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:23.657+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:24.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:23.547+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:24.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:23.547+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:24.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:23.547+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:24.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:23.547+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:24.084 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:23 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:23.547+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:24.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:23.655+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:24.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:23.655+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:24.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:23.655+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:24.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:23.655+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:24.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:23 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:23.655+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:24.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:23.679+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:24.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:23.679+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:24.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:23.679+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:24.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:23 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:23.679+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:24.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:24.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:24.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:24.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:24.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:24.502+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:24.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:24.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:24.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:24.834 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:24.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:24.834 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:24 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:24.608+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:25.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:25.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:25.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:25.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:25.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:25.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:24 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:24.688+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:25.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:24.694+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:25.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:24.694+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:25.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:24.694+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:25.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:24.694+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:25.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:24 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:24.694+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:25.832 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:25.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.913262+0000 front 2023-07-21T22:42:30.913115+0000 (oldest deadline 2023-07-21T22:42:53.813258+0000) 2023-07-21T22:43:25.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:25.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:40.116095+0000 front 2023-07-21T22:42:40.116130+0000 (oldest deadline 2023-07-21T22:43:04.816256+0000) 2023-07-21T22:43:25.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:25.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.717442+0000 front 2023-07-21T22:42:47.717499+0000 (oldest deadline 2023-07-21T22:43:09.417445+0000) 2023-07-21T22:43:25.833 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8[115122]: 2023-07-21T22:43:25.599+0000 7f4b96c64700 -1 osd.8 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:54.118603+0000 front 2023-07-21T22:42:54.118573+0000 (oldest deadline 2023-07-21T22:43:18.218659+0000) 2023-07-21T22:43:25.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:25.465+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:25.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:25.465+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:25.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:25.465+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:25.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:25.465+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:25.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:25 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:25.465+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:26.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:26.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:26.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:26.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:26.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:26.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:25 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:25.719+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:26.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:25.742+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:26.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:25.742+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:26.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:25.742+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:26.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:25.742+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:26.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:25 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:25.742+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:26.771 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:26.445+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:26.771 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:26.445+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:26.772 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:26.445+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:26.772 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:26.445+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:26.772 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:26.445+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:27.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:27.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:27.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:27.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:27.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:27.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:26 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:26.710+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:27.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:26.745+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:27.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:26.745+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:27.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:26.745+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:27.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:26.745+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:27.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:26 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:26.745+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:27.333 INFO:journalctl@ceph.osd.8.smithi194.stdout:Jul 21 22:43:26 smithi194 bash[134681]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-8 2023-07-21T22:43:27.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:27.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:27.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:27.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:27.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:27.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:27 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:27.410+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:28.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:28.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:28.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:28.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:28.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:28.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:27 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:27.677+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:28.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:27.764+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:28.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:27.764+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:28.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:27.764+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:28.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:27.764+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:28.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:27 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:27.764+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:28.397 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-21T22:43:28.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:28.478 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-21T22:43:28.478 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-21T22:43:28.478 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.9 2023-07-21T22:43:28.761 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 systemd[1]: Stopping Ceph osd.9 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:28.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:28.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:28.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:28.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:28.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:28.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:28 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:28.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:29.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:28 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:28.669+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:29.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.721+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.721+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.721+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.721+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.721+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.805+0000 7f1310014700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:29.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.805+0000 7f1310014700 -1 osd.9 133 *** Got signal Terminated *** 2023-07-21T22:43:29.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:28 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:28.805+0000 7f1310014700 -1 osd.9 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:29.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:29.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:29.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:29.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:29.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:29.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:29 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:29.401+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:30.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:30.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:30.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:30.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:30.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:30.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:29 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:29.619+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:30.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:29.729+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:30.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:29.729+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:30.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:29.729+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:30.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:29.729+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:30.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:29 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:29.729+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:30.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:30.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:30.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:30.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:30.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:30.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:30.390+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:30.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:30.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:30.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:30.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:30.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:30.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:30 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:30.573+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:31.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:30 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:30.687+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:31.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:30 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:30.687+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:31.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:30 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:30.687+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:31.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:30 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:30.687+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:31.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:30 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:30.687+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:31.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:31.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:31.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:31.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:31.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:31.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:31 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:31.430+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:31.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:31.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:31.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:31.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:31.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:31.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:31 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:31.574+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:32.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:31 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:31.681+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:32.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:31 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:31.681+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:32.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:31 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:31.681+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:32.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:31 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:31.681+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:32.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:31 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:31.681+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:32.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:32.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:32.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:32.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:32.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:32.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:32 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:32.459+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:32.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:32.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:32.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:32.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:32.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:32.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:32 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:32.530+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:33.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:33.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:33.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:33.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:33.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:33.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:32 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:32.731+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:09.816966+0000 front 2023-07-21T22:43:09.816909+0000 (oldest deadline 2023-07-21T22:43:32.117076+0000) 2023-07-21T22:43:33.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:33.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:33.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:33.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:33.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:33.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:33 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:33.472+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:33.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:33.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:33 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:33.562+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:34.186 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:20.105242+0000 front 2023-07-21T22:42:20.105270+0000 (oldest deadline 2023-07-21T22:42:48.906070+0000) 2023-07-21T22:43:34.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:28.906816+0000 front 2023-07-21T22:42:28.906665+0000 (oldest deadline 2023-07-21T22:42:53.606851+0000) 2023-07-21T22:43:34.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:37.708105+0000 front 2023-07-21T22:42:37.708135+0000 (oldest deadline 2023-07-21T22:43:01.208138+0000) 2023-07-21T22:43:34.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:46.811453+0000 front 2023-07-21T22:42:46.811479+0000 (oldest deadline 2023-07-21T22:43:10.911614+0000) 2023-07-21T22:43:34.187 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.314981+0000 front 2023-07-21T22:43:01.314773+0000 (oldest deadline 2023-07-21T22:43:24.214693+0000) 2023-07-21T22:43:34.188 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:33 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9[125983]: 2023-07-21T22:43:33.765+0000 7f130ae07700 -1 osd.9 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:09.816966+0000 front 2023-07-21T22:43:09.816909+0000 (oldest deadline 2023-07-21T22:43:32.117076+0000) 2023-07-21T22:43:34.686 INFO:journalctl@ceph.osd.9.smithi093.stdout:Jul 21 22:43:34 smithi093 bash[164277]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-9 2023-07-21T22:43:34.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:34.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:34.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:34.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:34.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:34.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:34.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:35.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:35.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:34 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:34.607+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:35.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-21T22:43:35.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:35.819 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-21T22:43:35.819 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-21T22:43:35.819 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.10 2023-07-21T22:43:35.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:35.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:35.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:35.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:35.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:35.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:35 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:35.440+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:36.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:36.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:35.613+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:36.126 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:35 smithi134 systemd[1]: Stopping Ceph osd.10 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:36.624 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.131+0000 7f2eeef72700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:36.625 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.131+0000 7f2eeef72700 -1 osd.10 133 *** Got signal Terminated *** 2023-07-21T22:43:36.625 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.131+0000 7f2eeef72700 -1 osd.10 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:36.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:36.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:36.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:36.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:36.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:36.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:36 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:36.444+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:37.124 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:37.125 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:36 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:36.594+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:37.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:37.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:37.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:37.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:37.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:37.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:37 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:37.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:37.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:37.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:37 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:37.564+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:38.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:38.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:38.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:38.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:38.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:38.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:38 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:38.438+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:38.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:38.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:38.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:38.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:38.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:38.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:38.876 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:38 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:38.532+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:39.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:39.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:39.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:39.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:39.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:39.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:39 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:39.466+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:39.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:39.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:39 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:39.519+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:40.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:40.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:40.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:40.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:40.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:40.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:40 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:40.432+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:40.874 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:22.537474+0000 front 2023-07-21T22:42:22.537512+0000 (oldest deadline 2023-07-21T22:42:48.937877+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:29.439009+0000 front 2023-07-21T22:42:29.439035+0000 (oldest deadline 2023-07-21T22:42:54.739150+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:39.341057+0000 front 2023-07-21T22:42:39.341092+0000 (oldest deadline 2023-07-21T22:43:03.441101+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:43.441712+0000 front 2023-07-21T22:42:43.441839+0000 (oldest deadline 2023-07-21T22:43:09.341698+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.743334+0000 front 2023-07-21T22:42:55.743379+0000 (oldest deadline 2023-07-21T22:43:19.843566+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:42:59.844243+0000 front 2023-07-21T22:42:59.844137+0000 (oldest deadline 2023-07-21T22:43:23.944066+0000) 2023-07-21T22:43:40.875 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:40 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10[119398]: 2023-07-21T22:43:40.522+0000 7f2eead67700 -1 osd.10 133 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-07-21T22:43:10.146995+0000 front 2023-07-21T22:43:10.146934+0000 (oldest deadline 2023-07-21T22:43:33.046937+0000) 2023-07-21T22:43:41.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:41.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:41 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:41.447+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:41.956 INFO:journalctl@ceph.osd.10.smithi134.stdout:Jul 21 22:43:41 smithi134 bash[136364]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-10 2023-07-21T22:43:42.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:42.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:42 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:42.403+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:43.213 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-21T22:43:43.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:43.254 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-21T22:43:43.254 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-21T22:43:43.255 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.11 2023-07-21T22:43:43.582 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 systemd[1]: Stopping Ceph osd.11 for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:43.583 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.407+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:44.082 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.580+0000 7f5cea034700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:44.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.580+0000 7f5cea034700 -1 osd.11 133 *** Got signal Terminated *** 2023-07-21T22:43:44.083 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:43 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:43.580+0000 7f5cea034700 -1 osd.11 133 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:43:44.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:44.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:44 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:44.360+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:45.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:45.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:45 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:45.391+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:46.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:46.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:46.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:46.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:46.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:46.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:46.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:46 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:46.352+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:47.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:47.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:47.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:47 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:47.323+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2023-07-21T22:43:23.168156+0000 front 2023-07-21T22:43:23.168194+0000 (oldest deadline 2023-07-21T22:43:47.268196+0000) 2023-07-21T22:43:48.832 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-07-21T22:42:23.456751+0000 front 2023-07-21T22:42:23.456733+0000 (oldest deadline 2023-07-21T22:42:47.556670+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6804 osd.1 since back 2023-07-21T22:42:30.358451+0000 front 2023-07-21T22:42:30.358380+0000 (oldest deadline 2023-07-21T22:42:55.658340+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2023-07-21T22:42:38.559106+0000 front 2023-07-21T22:42:38.559182+0000 (oldest deadline 2023-07-21T22:43:01.359989+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-07-21T22:42:47.761824+0000 front 2023-07-21T22:42:47.761795+0000 (oldest deadline 2023-07-21T22:43:10.061718+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6812 osd.4 since back 2023-07-21T22:42:55.362830+0000 front 2023-07-21T22:42:55.362956+0000 (oldest deadline 2023-07-21T22:43:16.963551+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2023-07-21T22:43:01.664485+0000 front 2023-07-21T22:43:01.664555+0000 (oldest deadline 2023-07-21T22:43:26.964553+0000) 2023-07-21T22:43:48.833 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.134:6820 osd.7 since back 2023-07-21T22:43:17.367102+0000 front 2023-07-21T22:43:17.367089+0000 (oldest deadline 2023-07-21T22:43:40.867171+0000) 2023-07-21T22:43:48.834 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:48 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11[120626]: 2023-07-21T22:43:48.353+0000 7f5ce4626700 -1 osd.11 133 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2023-07-21T22:43:23.168156+0000 front 2023-07-21T22:43:23.168194+0000 (oldest deadline 2023-07-21T22:43:47.268196+0000) 2023-07-21T22:43:49.494 INFO:journalctl@ceph.osd.11.smithi194.stdout:Jul 21 22:43:49 smithi194 bash[134950]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-osd-11 2023-07-21T22:43:50.753 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-21T22:43:50.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:50.793 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-21T22:43:50.793 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-21T22:43:50.794 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.a 2023-07-21T22:43:51.150 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:43:50 smithi093 systemd[1]: Stopping Ceph mds.a for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:51.436 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:43:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: 2023-07-21T22:43:51.110+0000 7fc35e1ca700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:51.436 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:43:51 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a[131348]: 2023-07-21T22:43:51.110+0000 7fc35e1ca700 -1 mds.a *** got signal Terminated *** 2023-07-21T22:43:53.593 INFO:journalctl@ceph.mds.a.smithi093.stdout:Jul 21 22:43:53 smithi093 bash[164541]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-a 2023-07-21T22:43:53.600 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-21T22:43:53.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:53.637 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-21T22:43:53.637 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-21T22:43:53.637 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.d 2023-07-21T22:43:53.936 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:43:53 smithi093 systemd[1]: Stopping Ceph mds.d for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:54.436 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:43:53 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-d[130345]: 2023-07-21T22:43:53.967+0000 7fc7d58e1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:54.436 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:43:53 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-d[130345]: 2023-07-21T22:43:53.967+0000 7fc7d58e1700 -1 mds.d *** got signal Terminated *** 2023-07-21T22:43:55.856 INFO:journalctl@ceph.mds.d.smithi093.stdout:Jul 21 22:43:55 smithi093 bash[164680]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-d 2023-07-21T22:43:55.862 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-21T22:43:55.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:43:55.896 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-21T22:43:55.896 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-21T22:43:55.896 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.g 2023-07-21T22:43:56.186 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:43:55 smithi093 systemd[1]: Stopping Ceph mds.g for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:43:56.686 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:43:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: 2023-07-21T22:43:56.226+0000 7f6f3e851700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:43:56.686 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:43:56 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g[130963]: 2023-07-21T22:43:56.226+0000 7f6f3e851700 -1 mds.g *** got signal Terminated *** 2023-07-21T22:44:00.686 INFO:journalctl@ceph.mds.g.smithi093.stdout:Jul 21 22:44:00 smithi093 bash[164821]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-g 2023-07-21T22:44:00.790 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-21T22:44:00.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:00.823 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-21T22:44:00.823 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-21T22:44:00.824 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.j 2023-07-21T22:44:01.186 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:44:00 smithi093 systemd[1]: Stopping Ceph mds.j for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:01.686 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:44:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-j[130004]: 2023-07-21T22:44:01.157+0000 7f44c5ca8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:01.686 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:44:01 smithi093 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-j[130004]: 2023-07-21T22:44:01.157+0000 7f44c5ca8700 -1 mds.j *** got signal Terminated *** 2023-07-21T22:44:05.656 INFO:journalctl@ceph.mds.j.smithi093.stdout:Jul 21 22:44:05 smithi093 bash[164967]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-j 2023-07-21T22:44:05.732 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-21T22:44:05.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:05.769 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-21T22:44:05.769 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-21T22:44:05.769 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.b 2023-07-21T22:44:06.124 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:44:05 smithi134 systemd[1]: Stopping Ceph mds.b for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:06.874 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:44:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: 2023-07-21T22:44:06.375+0000 7f414bbe3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:06.874 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:44:06 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b[123193]: 2023-07-21T22:44:06.375+0000 7f414bbe3700 -1 mds.b *** got signal Terminated *** 2023-07-21T22:44:09.831 INFO:journalctl@ceph.mds.b.smithi134.stdout:Jul 21 22:44:09 smithi134 bash[136629]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-b 2023-07-21T22:44:09.838 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-21T22:44:09.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:09.875 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-21T22:44:09.876 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-21T22:44:09.876 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.e 2023-07-21T22:44:10.124 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:44:09 smithi134 systemd[1]: Stopping Ceph mds.e for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:10.624 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:44:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: 2023-07-21T22:44:10.208+0000 7f0ba5a81700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:10.624 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:44:10 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e[122511]: 2023-07-21T22:44:10.208+0000 7f0ba5a81700 -1 mds.e *** got signal Terminated *** 2023-07-21T22:44:13.521 INFO:journalctl@ceph.mds.e.smithi134.stdout:Jul 21 22:44:13 smithi134 bash[136784]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-e 2023-07-21T22:44:13.709 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-21T22:44:13.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:13.745 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-21T22:44:13.745 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-21T22:44:13.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.h 2023-07-21T22:44:14.122 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:44:13 smithi134 systemd[1]: Stopping Ceph mds.h for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:14.374 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:44:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: 2023-07-21T22:44:14.083+0000 7f204c762700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:14.374 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:44:14 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h[122853]: 2023-07-21T22:44:14.083+0000 7f204c762700 -1 mds.h *** got signal Terminated *** 2023-07-21T22:44:17.782 INFO:journalctl@ceph.mds.h.smithi134.stdout:Jul 21 22:44:17 smithi134 bash[136926]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-h 2023-07-21T22:44:17.787 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-21T22:44:17.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:17.820 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-21T22:44:17.820 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-21T22:44:17.821 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.k 2023-07-21T22:44:18.124 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:44:17 smithi134 systemd[1]: Stopping Ceph mds.k for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:18.624 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:44:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: 2023-07-21T22:44:18.158+0000 7f408a26e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:18.624 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:44:18 smithi134 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k[122169]: 2023-07-21T22:44:18.158+0000 7f408a26e700 -1 mds.k *** got signal Terminated *** 2023-07-21T22:44:21.624 INFO:journalctl@ceph.mds.k.smithi134.stdout:Jul 21 22:44:21 smithi134 bash[137072]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-k 2023-07-21T22:44:21.718 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-21T22:44:21.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:21.752 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-21T22:44:21.753 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-21T22:44:21.753 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.c 2023-07-21T22:44:22.082 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:44:21 smithi194 systemd[1]: Stopping Ceph mds.c for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:22.582 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:44:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:44:22.149+0000 7feab795f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:22.582 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:44:22 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c[124224]: 2023-07-21T22:44:22.149+0000 7feab795f700 -1 mds.c *** got signal Terminated *** 2023-07-21T22:44:25.783 INFO:journalctl@ceph.mds.c.smithi194.stdout:Jul 21 22:44:25 smithi194 bash[135223]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-c 2023-07-21T22:44:25.790 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-21T22:44:25.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:25.828 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-21T22:44:25.829 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-21T22:44:25.829 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.f 2023-07-21T22:44:26.082 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:44:25 smithi194 systemd[1]: Stopping Ceph mds.f for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:26.582 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:44:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:44:26.197+0000 7f86927ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:26.582 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:44:26 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f[123884]: 2023-07-21T22:44:26.197+0000 7f86927ef700 -1 mds.f *** got signal Terminated *** 2023-07-21T22:44:29.581 INFO:journalctl@ceph.mds.f.smithi194.stdout:Jul 21 22:44:29 smithi194 bash[135367]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-f 2023-07-21T22:44:29.769 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-21T22:44:29.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:29.803 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-21T22:44:29.803 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-21T22:44:29.804 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.i 2023-07-21T22:44:30.082 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:44:29 smithi194 systemd[1]: Stopping Ceph mds.i for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:30.582 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:44:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: 2023-07-21T22:44:30.147+0000 7fa834d1e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:30.582 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:44:30 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i[124563]: 2023-07-21T22:44:30.147+0000 7fa834d1e700 -1 mds.i *** got signal Terminated *** 2023-07-21T22:44:33.832 INFO:journalctl@ceph.mds.i.smithi194.stdout:Jul 21 22:44:33 smithi194 bash[135508]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-i 2023-07-21T22:44:33.862 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-21T22:44:33.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:33.895 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-21T22:44:33.896 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-21T22:44:33.896 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.l 2023-07-21T22:44:34.294 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:44:33 smithi194 systemd[1]: Stopping Ceph mds.l for de1cec7a-2812-11ee-9b34-001a4aab830c... 2023-07-21T22:44:34.582 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:44:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: 2023-07-21T22:44:34.255+0000 7fa717c58700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:44:34.582 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:44:34 smithi194 ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l[123543]: 2023-07-21T22:44:34.255+0000 7fa717c58700 -1 mds.l *** got signal Terminated *** 2023-07-21T22:44:37.812 INFO:journalctl@ceph.mds.l.smithi194.stdout:Jul 21 22:44:37 smithi194 bash[135653]: ceph-de1cec7a-2812-11ee-9b34-001a4aab830c-mds-l 2023-07-21T22:44:37.901 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-de1cec7a-2812-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-21T22:44:37.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:44:37.936 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-21T22:44:37.937 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:44:39.843 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:44:41.705 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:44:43.624 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:44:43.661 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:44:43.697 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:44:43.726 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-21T22:44:43.729 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi093/crash 2023-07-21T22:44:43.730 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:44:43.777 INFO:teuthology.orchestra.run.smithi093.stderr:tar: /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:44:43.778 INFO:teuthology.orchestra.run.smithi093.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:44:43.779 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi134/crash 2023-07-21T22:44:43.780 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:44:43.847 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:44:43.847 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:44:43.850 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi194/crash 2023-07-21T22:44:43.852 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:44:43.896 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:44:43.897 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:44:43.898 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-21T22:44:43.899 DEBUG:teuthology.orchestra.run.smithi093:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-21T22:44:43.937 INFO:teuthology.orchestra.run.smithi093.stderr:grep: /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-21T22:44:43.940 INFO:tasks.cephadm:Compressing logs... 2023-07-21T22:44:43.941 DEBUG:teuthology.orchestra.run.smithi093:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:44:43.983 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:44:43.986 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:44:44.019 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:44:44.019 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2023-07-21T22:44:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-21T22:44:44.022 INFO:teuthology.orchestra.run.smithi093.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log : No such file or directory 2023-07-21T22:44:44.023 INFO:teuthology.orchestra.run.smithi093.stderr: 2023-07-21T22:44:44.024 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-21T22:44:44.024 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-21T22:44:44.024 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-21T22:44:44.024 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-21T22:44:44.024 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-21T22:44:44.025 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:44:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-21T22:44:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-21T22:44:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-21T22:44:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-21T22:44:44.027 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-21T22:44:44.027 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:44:44.032 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-21T22:44:44.033 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-21T22:44:44.034 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-21T22:44:44.034 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-21T22:44:44.034 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:44:44.034 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-21T22:44:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 92.5% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-21T22:44:44.038 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-21T22:44:44.039 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-21T22:44:44.039 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-21T22:44:44.039 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.0.log: /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-21T22:44:44.040 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-21T22:44:44.040 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-21T22:44:44.040 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-21T22:44:44.040 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-21T22:44:44.041 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:44:44.041 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-21T22:44:44.041 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-21T22:44:44.041 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.j.log: /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-21T22:44:44.042 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.l.log: 92.5% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-21T22:44:44.043 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:44:44.043 INFO:teuthology.orchestra.run.smithi093.stderr: 94.6% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-21T22:44:44.263 INFO:teuthology.orchestra.run.smithi194.stderr: 91.5% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:44:44.318 INFO:teuthology.orchestra.run.smithi134.stderr: 91.4% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:44:44.440 INFO:teuthology.orchestra.run.smithi194.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:44:44.706 INFO:teuthology.orchestra.run.smithi093.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:44:44.925 INFO:teuthology.orchestra.run.smithi093.stderr: 91.6% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:44:45.726 INFO:teuthology.orchestra.run.smithi093.stderr: 89.7% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-21T22:44:47.057 INFO:teuthology.orchestra.run.smithi093.stderr: 90.4% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-21T22:44:47.515 INFO:teuthology.orchestra.run.smithi194.stderr: 91.9% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-21T22:44:53.009 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-21T22:45:07.994 INFO:teuthology.orchestra.run.smithi093.stderr: 95.2% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-21T22:45:10.037 INFO:teuthology.orchestra.run.smithi134.stderr: 95.2% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-21T22:45:13.746 INFO:teuthology.orchestra.run.smithi134.stderr: 95.1% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-21T22:45:14.779 INFO:teuthology.orchestra.run.smithi134.stderr: 95.2% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-21T22:45:25.825 INFO:teuthology.orchestra.run.smithi093.stderr: 93.7% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-21T22:45:26.049 INFO:teuthology.orchestra.run.smithi093.stderr: 93.9% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-21T22:45:30.339 INFO:teuthology.orchestra.run.smithi134.stderr: 93.9% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-21T22:45:31.621 INFO:teuthology.orchestra.run.smithi093.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-21T22:45:33.565 INFO:teuthology.orchestra.run.smithi093.stderr: 93.7% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-21T22:45:36.076 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-21T22:45:40.828 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-21T22:45:45.245 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-21T22:45:47.948 INFO:teuthology.orchestra.run.smithi194.stderr: 95.3% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-21T22:46:26.331 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-21T22:46:30.444 INFO:teuthology.orchestra.run.smithi194.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-21T22:46:44.130 INFO:teuthology.orchestra.run.smithi194.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-21T22:46:45.290 INFO:teuthology.orchestra.run.smithi194.stderr: 93.8% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-21T22:48:34.601 INFO:teuthology.orchestra.run.smithi093.stderr: 93.1% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-21T22:48:34.603 INFO:teuthology.orchestra.run.smithi093.stderr: 2023-07-21T22:48:34.603 INFO:teuthology.orchestra.run.smithi093.stderr:real 3m50.606s 2023-07-21T22:48:34.604 INFO:teuthology.orchestra.run.smithi093.stderr:user 4m46.919s 2023-07-21T22:48:34.604 INFO:teuthology.orchestra.run.smithi093.stderr:sys 0m11.174s 2023-07-21T22:48:59.121 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-21T22:48:59.124 INFO:teuthology.orchestra.run.smithi134.stderr: 2023-07-21T22:48:59.124 INFO:teuthology.orchestra.run.smithi134.stderr:real 4m15.125s 2023-07-21T22:48:59.125 INFO:teuthology.orchestra.run.smithi134.stderr:user 5m12.618s 2023-07-21T22:48:59.125 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m12.656s 2023-07-21T22:52:26.901 INFO:teuthology.orchestra.run.smithi194.stderr: 93.0% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-21T22:53:09.884 INFO:teuthology.orchestra.run.smithi194.stderr: 93.1% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-21T22:54:12.141 INFO:teuthology.orchestra.run.smithi194.stderr: 93.1% -- replaced with /var/log/ceph/de1cec7a-2812-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-21T22:54:12.153 INFO:teuthology.orchestra.run.smithi194.stderr: 2023-07-21T22:54:12.154 INFO:teuthology.orchestra.run.smithi194.stderr:real 9m28.155s 2023-07-21T22:54:12.154 INFO:teuthology.orchestra.run.smithi194.stderr:user 11m56.826s 2023-07-21T22:54:12.154 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m29.417s 2023-07-21T22:54:12.154 INFO:tasks.cephadm:Archiving logs... 2023-07-21T22:54:12.155 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi093/log 2023-07-21T22:54:12.157 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:54:47.777 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi134/log 2023-07-21T22:54:47.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:55:27.024 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi194/log 2023-07-21T22:55:27.028 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:56:57.159 INFO:tasks.cephadm:Removing cluster... 2023-07-21T22:56:57.160 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force 2023-07-21T22:56:58.254 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force 2023-07-21T22:56:59.382 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid de1cec7a-2812-11ee-9b34-001a4aab830c --force 2023-07-21T22:57:01.334 INFO:tasks.cephadm:Removing cephadm ... 2023-07-21T22:57:01.334 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:57:01.356 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:57:01.378 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:57:01.400 INFO:tasks.cephadm:Teardown complete 2023-07-21T22:57:01.401 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-21T22:57:01.412 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-21T22:57:01.413 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:57:01.416 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:57:01.421 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:57:01.560 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-21T22:57:01.574 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-21T22:57:01.575 DEBUG:teuthology.orchestra.run.smithi093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:57:01.578 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:57:01.582 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:57:01.594 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-07-21T22:57:01.597 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-07-21T22:57:01.598 INFO:teuthology.orchestra.run.smithi194.stderr:bash: ntpq: command not found 2023-07-21T22:57:01.599 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:57:01.599 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-07-21T22:57:01.599 INFO:teuthology.orchestra.run.smithi093.stdout:^- hv01.front.sepia.ceph.com 3 6 377 30 +182us[ +182us] +/- 68ms 2023-07-21T22:57:01.600 INFO:teuthology.orchestra.run.smithi093.stdout:^* hv02.front.sepia.ceph.com 3 8 377 157 -1128us[-1114us] +/- 75ms 2023-07-21T22:57:01.600 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 91 +1984us[+1984us] +/- 91ms 2023-07-21T22:57:01.600 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:57:01.626 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:57:01.627 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2023-07-21T22:57:01.627 INFO:teuthology.orchestra.run.smithi194.stdout:^- hv01.front.sepia.ceph.com 3 6 377 31 -395us[ -154us] +/- 68ms 2023-07-21T22:57:01.627 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -1687us[-1447us] +/- 77ms 2023-07-21T22:57:01.628 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +1409us[+1650us] +/- 59ms 2023-07-21T22:57:01.628 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:57:01.633 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:57:01.633 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-07-21T22:57:01.634 INFO:teuthology.orchestra.run.smithi134.stdout:^- hv01.front.sepia.ceph.com 3 6 377 30 -76us[ -76us] +/- 68ms 2023-07-21T22:57:01.634 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1392us[-1374us] +/- 77ms 2023-07-21T22:57:01.634 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +1727us[+1727us] +/- 59ms 2023-07-21T22:57:01.634 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:57:01.634 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-21T22:57:01.645 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-21T22:57:01.646 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-21T22:57:01.657 DEBUG:teuthology.orchestra.run.smithi093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:57:01.811 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:57:01.958 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:57:02.143 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:57:02.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:02.182 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:57:02.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:02.217 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:57:02.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:02.249 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-21T22:57:02.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-21T22:57:02.271 INFO:teuthology.task.internal:Duration was 4076.565940 seconds 2023-07-21T22:57:02.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-21T22:57:02.284 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-21T22:57:02.285 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:57:02.288 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:57:02.291 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:57:02.358 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:57:02.379 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:57:02.405 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:57:02.652 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-21T22:57:02.652 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi093.front.sepia.ceph.com 2023-07-21T22:57:02.653 DEBUG:teuthology.orchestra.run.smithi093:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:57:02.679 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2023-07-21T22:57:02.680 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:57:02.705 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2023-07-21T22:57:02.705 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:57:02.744 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-21T22:57:02.744 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:57:02.747 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:57:02.749 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:57:02.849 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-21T22:57:02.850 DEBUG:teuthology.orchestra.run.smithi093:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:57:02.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:57:02.859 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:57:02.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-21T22:57:02.993 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-21T22:57:02.994 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:57:03.055 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:57:03.083 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:57:03.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-21T22:57:03.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.159 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.161 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.185 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2023-07-21T22:57:03.186 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = core 2023-07-21T22:57:03.200 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2023-07-21T22:57:03.231 DEBUG:teuthology.orchestra.run.smithi093:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:03.266 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:03.290 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:57:03.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:57:03.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-21T22:57:03.350 INFO:teuthology.task.internal:Transferring archived files... 2023-07-21T22:57:03.352 DEBUG:teuthology.misc:Transferring archived files from smithi093:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi093 2023-07-21T22:57:03.352 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:57:03.883 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi134 2023-07-21T22:57:03.883 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:57:03.952 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722/remote/smithi194 2023-07-21T22:57:03.953 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:57:04.018 INFO:teuthology.task.internal:Removing archive directory... 2023-07-21T22:57:04.019 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:57:04.025 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:57:04.034 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:57:04.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-21T22:57:04.086 INFO:teuthology.task.internal:Not uploading archives. 2023-07-21T22:57:04.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-21T22:57:04.102 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-21T22:57:04.103 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:57:04.108 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:57:04.113 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:57:04.125 INFO:teuthology.orchestra.run.smithi093.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:57 /home/ubuntu/cephtest 2023-07-21T22:57:04.129 INFO:teuthology.orchestra.run.smithi134.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:57 /home/ubuntu/cephtest 2023-07-21T22:57:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:57 /home/ubuntu/cephtest 2023-07-21T22:57:04.134 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-21T22:57:04.148 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-21T22:57:04.189 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T22:57:04.220 DEBUG:teuthology.nuke:shortname: smithi093 2023-07-21T22:57:04.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:57:04.249 DEBUG:teuthology.nuke:shortname: smithi134 2023-07-21T22:57:04.249 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:57:04.254 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.017564', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:57:04.282 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.019731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:57:04.287 DEBUG:teuthology.nuke:shortname: smithi194 2023-07-21T22:57:04.287 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:57:04.321 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347722', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:41:35.021060', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:57:04.344 INFO:teuthology.orchestra.console:Power off smithi093 2023-07-21T22:57:04.345 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:57:04.363 INFO:teuthology.orchestra.console:Power off smithi134 2023-07-21T22:57:04.364 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:57:04.375 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:04.389 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:04.418 INFO:teuthology.orchestra.console:Power off smithi194 2023-07-21T22:57:04.419 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:57:04.454 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:08.403 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:08.432 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:08.485 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:12.520 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:12.545 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:12.600 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:12.812 INFO:teuthology.orchestra.console:Power off for smithi194 completed 2023-07-21T22:57:16.636 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:16.659 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:57:16.850 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2023-07-21T22:57:16.877 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2023-07-21T22:57:16.982 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 4076.5659399032593 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-21T22:57:16.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T22:57:17.150 INFO:teuthology.run:FAIL